abu@software-lab.de

A PicoLisp Tutorial

(c) Software Lab. Alexander Burger

This document demonstrates some aspects of the PicoLisp system in detail and example. For a general description of the PicoLisp kernel please look at the PicoLisp Reference.

This is not a Lisp tutorial, as it assumes some basic knowledge of programming, Lisp, and even PicoLisp. Please read these sections before coming back here: Introduction and The PicoLisp Machine. This tutorial concentrates on the specificities of PicoLisp, and its differences with other Lisp dialects.

Now let's start

If not stated otherwise, all examples assume that PicoLisp was started from a global installation (see Installation) from the shell prompt as

$ pil +
:

It loads the PicoLisp base system and the debugging environment, and waits for you to enter input lines at the interpreter prompt (:). You can terminate the interpreter and return to the shell at any time, by either hitting the Ctrl-D key, or by executing the function (bye).

Input editing is done via the readline(3) library. You will want to configure it according to your taste via your "~/.inputrc" file. Useful value for PicoLisp are

set keyseq-timeout 40
set blink-matching-paren on
TAB: menu-complete
C-y: menu-complete-backward
In addition to the above, I (preferring vi-style) do also have
set editing-mode vi
set keymap vi-command
v: ""

Table of content

If you are new to PicoLisp, you might want to read the following sections in the given order, as some of them assume knowledge about previous ones. Otherwise just jump anywhere you are interested in.


Browsing

PicoLisp provides some functionality for inspecting pieces of data and code within the running system.

Basic tools

The really basic tools are of course available and their name alone is enough to know: print, size ...

But you will appreciate some more powerful tools like:

Inspect a symbol with show

The most commonly used tool is probably the show function. It takes a symbolic argument, and shows the symbol's name (if any), followed by its value, and then the contents of the property list on the following lines (assignment of such things to a symbol can be done with set, setq, and put).

: (setq A '(This is the value))  # Set the value of 'A'
-> (This is the value)
: (put 'A 'key1 'val1)           # Store property 'key1'
-> val1
: (put 'A 'key2 'val2)           # and 'key2'
-> val2
: (show 'A)                      # Now 'show' the symbol 'A'
A (This is the value)
   key2 val2
   key1 val1
-> A

show accepts an arbitrary number of arguments which are processed according to the rules of get, resulting in a symbol which is showed then.

: (put 'B 'a 'A)        # Put 'A' under the 'a'-property of 'B'
-> A
: (setq Lst '(A B C))   # Create a list with 'B' as second argument
-> (A B C)
: (show Lst 2 'a)       # Show the property 'a of the 2nd element of 'Lst'
A (This is the value)   # (which is 'A' again)
   key2 val2
   key1 val1
-> A

Inspect and edit symbols in-memory

If you pass one or more symbols as a list to vi, they are written to a temporary file in a format similar to show, and Vip is started with that file.

: (vi '(A B))

The Vip window will look like

A (This is the value)
key1 val1
key2 val2

(=======)

B NIL
a A  # (This is the value)

(=======)

A convenient shortcut is the non-evaluating version v of vi. An equivalent call to the above is:

(v A B)

Now you can modify values or properties. You should not touch the parenthesized hyphens, as they serve as delimiters. If you position the cursor on the first character of a symbol name and type 'K' ("Keyword lookup"), the editor will be restarted with that symbol added to the editor window. 'Q' (for "Quit") will bring you back to the previous view.

If you exit Vip with e.g. ":x", any changes you made in your editing session will be communicated back to the REPL.

In-memory editing is also very useful to browse in a database. You can follow the links between objects with 'K', and even - e.g. for low-level repairs - modify the data (but only if you are really sure about what you are doing, and don't forget to commit when you are done).

Built-in pretty print with pp

The pretty-print function pp takes a symbol that has a function defined (or two symbols that specify message and class for a method definition), and displays that definition in a formatted and indented way.

: (pp 'pretty)
(de pretty (X N)
   (setq N (abs (space (or N 0))))
   (while (and (pair X) (== 'quote (car X)))
      (prin "'")
      (pop 'X) )
   (cond
      ...
      (T (prtty0 X N)) ) )
-> pretty

The style is the same as we use in source files:

Inspect elements one by one with more

more is a simple tool that displays the elements of a list one by one. It stops after each element and waits for input. If you just hit ENTER, more continues with the next element, otherwise (usually I type a dot (.) followed by ENTER) it terminates.

: (more (1 2 3 4 5 6))
1                          # Hit ENTER
2   .                      # Hit '.' and ENTER
-> T                       # stopped

Optionally more takes a function as a second argument and applies that function to each element (instead of the default print). Here, often show or pp (see below) is used.

: (more '(A B))            # Step through 'A' and 'B'
A
B
-> NIL
: (more '(A B) show)       # Step through 'A' and 'B' with 'show'
A (This is the value)      # showing 'A'
   key2 val2
   key1 val1
                           # Hit ENTER
B NIL                      # showing 'B'
   a A
-> NIL

Search through available symbols with what

The what function returns a list of all internal symbols in the system which match a given pattern (with '@' wildcard characters).

: (what "prin@")
-> (prin print prinl print> printsp println)

Search through values or properties of symbols with who

The function who returns "who contains that", i.e. a list of symbols that contain a given argument somewhere in their value or property list.

: (who 'print)
-> (query _pretty spPrt prtty1 prtty2 prtty3 pretty ("syms>" . "+Buffer")
msg more show view (print> . +Date) rules select (print> . +relation) pico)

A dotted pair indicates either a method definition or a property entry. So (print> . +relation) denotes the print> method of the +relation class.

who can be conveniently combined with more and pp:

: (more (who 'print) pp)
(de query ("Q" "Dbg")  # Pretty-print these functions one by one
   (use "R"
      (loop
         (NIL (prove "Q" "Dbg"))
         (T (=T (setq "R" @)) T)
         (for X "R"
            (space)
            (print (car X))
            (print '=)
            (print (cdr X))
            (flush) )
         (T (line)) ) ) )

(de pretty (X N)
   ...

The argument to who may also be a pattern list (see match):

: (who '(print @ (less (val @))))
-> (show)

: (more (who '(% @ 7)) pp)
(de day (Dat Lst)
   (when Dat
      (get
         (or Lst *DayFmt)
         (inc (% (inc Dat) 7)) ) ) )

(de _week (Dat)
   (/ (- Dat (% (inc Dat) 7)) 7) )

Find what classes can accept a given message with can

The function can returns a list which indicates which classes can accept a given message. Again, this list is suitable for iteration with pp:

: (can 'del>)                                   # Which classes accept 'del>' ?
-> ((del> . +List) (del> . +Entity) (del> . +relation))

: (more (can 'del>) pp)                         # Inspect the methods with 'pp'
(dm (del> . +List) (Obj Old Val)
   (and ((<> Old Val) (delete Val Old)) )

(dm (del> . +Entity) (Var Val)
   (when
      (and
         Val
         (has> (meta This Var) Val (get This Var)) )
      (let Old (get This Var)
         (rel>
            (meta This Var)
            This
            Old
            (put This Var (del> (meta This Var) This Old @)) )
         (when (asoq Var (meta This 'Aux))
            (relAux This Var Old (cdr @)) )
         (upd> This Var Old) ) ) )

(dm (del> . +relation) (Obj Old Val)
   (and ((<> Old Val) Val) )

Inspect dependencies with dep

dep shows the dependencies in a class hierarchy. That is, for a given class it displays the tree of its (super)class(es) above it, and the tree of its subclasses below it.

To view the complete hierarchy of input fields, we start with the root class +relation:

: (dep '+relation)
+relation
   +Bag
   +Any
   +Blob
   +Link
      +Joint
   +Bool
   +Symbol
      +String
   +Number
      +Time
      +Date
-> +relation

If we are interested in +Link:

: (dep '+Link)
   +relation
+Link
   +Joint
-> +Link

This says that +Link is a subclass of +relation, and has a single subclass (+Joint).


Defining Functions

Most of the time during programming is spent defining functions (or methods). In the following we will concentrate on functions, but most will be true for methods as well except for using dm instead of de.

Functions with no argument

The notorious "Hello world" function must be defined:

: (de hello ()
   (prinl "Hello world") )
-> hello

The () in the first line indicates a function without arguments. The body of the function is in the second line, consisting of a single statement. The last line is the return value of de, which here is the defined symbol. From now on we will omit the return values of examples when they are unimportant.

Now you can call this function this way:

: (hello)
Hello world

Functions with one argument

A function with an argument might be defined this way:

: (de hello (X)
   (prinl "Hello " X) )
# hello redefined
-> hello

PicoLisp informs you that you have just redefined the function. This might be a useful warning in case you forgot that a bound symbol with that name already existed.

: (hello "world")
Hello world
: (hello "Alex")
Hello Alex

Preventing arguments evaluation and variable number of arguments

Normally, PicoLisp evaluates the arguments before it passes them to a function:

: (hello (+ 1 2 3))
Hello 6
: (setq A 1  B 2)       # Set 'A' to 1 and 'B' to 2
-> 2
: (de foo (X Y)         # 'foo' returns the list of its arguments
   (list X Y) )
-> foo
: (foo A B)             # Now call 'foo' with 'A' and 'B'
-> (1 2)                # -> We get a list of 1 and 2, the values of 'A' and 'B'

In some cases you don't want that. For some functions (setq for example) it is better if the function gets all arguments unevaluated, and can decide for itself what to do with them.

For such cases you do not define the function with a list of parameters, but give it a single atomic parameter instead. PicoLisp will then bind all (unevaluated) arguments as a list to that parameter.

: (de foo X
   (list (car X) (cadr X)) )        # 'foo' lists the first two arguments

: (foo A B)                         # Now call it again
-> (A B)                            # -> We don't get '(1 2)', but '(A B)'

: (de foo X
   (list (car X) (eval (cadr X))) ) # Now evaluate only the second argument

: (foo A B)
-> (A 2)                            # -> We get '(A 2)'

Mixing evaluated arguments and variable number of unevaluated arguments

As a logical consequence, you can combine these principles. To define a function with 2 evaluated and an arbitrary number of unevaluated arguments:

: (de foo (X Y . Z)     # Evaluate only the first two args
   (list X Y Z) )

: (foo A B C D E)
-> (1 2 (C D E))        # -> Get the value of 'A' and 'B' and the remaining list

Variable number of evaluated arguments

More common, in fact, is the case where you want to pass an arbitrary number of evaluated arguments to a function. For that, PicoLisp recognizes the symbol @ as a single atomic parameter and remembers all evaluated arguments in an internal frame. This frame can then be accessed sequentially with the args, next, arg and rest functions.

: (de foo @
   (list (next) (next)) )     # Get the first two arguments

: (foo A B)
-> (1 2)

Again, this can be combined:

: (de foo (X Y . @)
   (list X Y (next) (next)) ) # 'X' and 'Y' are fixed arguments

: (foo A B (+ 3 4) (* 3 4))
-> (1 2 7 12)                 # All arguments are evaluated

These examples are not very useful, because the advantage of a variable number of arguments is not used. A function that prints all its evaluated numeric arguments, each on a line followed by its squared value:

: (de foo @
   (while (args)                    # Check if there are some args left
      (let N (next)
         (println N (* N N)) ) ) )

: (foo (+ 2 3) (- 7 1) 1234 (* 9 9))
5 25
6 36
1234 1522756
81 6561
-> 6561

This next example shows the behaviour of args and rest:

: (de foo @
   (while (args)
      (println (next) (args) (rest)) ) )
: (foo 1 2 3)
1 T (2 3)
2 T (3)
3 NIL NIL

Finally, it is possible to pass all these evaluated arguments to another function, using pass:

: (de foo @
   (pass println 9 8 7)       # First print all arguments preceded by 9, 8, 7
   (pass + 9 8 7) )           # Then add all these values

: (foo (+ 2 3) (- 7 1) 1234 (* 9 9))
9 8 7 5 6 1234 81             # Printing ...
-> 1350                       # Return the result

Anonymous functions without the lambda keyword

There's no distinction between code and data in PicoLisp, quote will do what you want (see also this FAQ entry).
: ((quote (X) (* X X)) 9)
-> 81
: (setq f '((X) (* X X)))  # This is equivalent to (de f (X) (* X X))
-> ((X) (* X X))
: f
-> ((X) (* X X))
: (f 3)
-> 9


Debugging

There are two major ways to debug functions (and methods) at runtime: Tracing and single-stepping.

In this section we will use the REPL to explore the debugging facilities, but in the Scripting section, you will learn how to launch PicoLisp scripts with some selected functions debugged:

$ pil app/file1.l -"trace 'foo" -main -"debug 'bar" app/file2.l +

Tracing

Tracing means letting functions of interest print their name and arguments when they are entered, and their name again and the return value when they are exited.

For demonstration, let's define the unavoidable factorial function:

(de fact (N)
   (if (=0 N)
      1
      (* N (fact (dec N))) ) )

With trace we can put it in trace mode:

: (trace 'fact)
-> fact

Calling fact now will display its execution trace.

: (fact 3)
 fact : 3
  fact : 2
   fact : 1
    fact : 0
    fact = 1
   fact = 1
  fact = 2
 fact = 6
-> 6

As can be seen here, each level of function call will indent by an additional space. Upon function entry, the name is separated from the arguments with a colon (:), and upon function exit with an equals sign (=) from the return value.

trace works by modifying the function body, so generally it works only for functions defined as lists (lambda expressions, see Evaluation). Tracing a built-in function (SUBR) is possible, however, when it is a function that evaluates all its arguments.

So let's trace the functions =0 and *:

: (trace '=0)
-> =0
: (trace '*)
-> *

If we call fact again, we see the additional output:

: (fact 3)
 fact : 3
  =0 : 3
  =0 = NIL
  fact : 2
   =0 : 2
   =0 = NIL
   fact : 1
    =0 : 1
    =0 = NIL
    fact : 0
     =0 : 0
     =0 = 0
    fact = 1
    * : 1 1
    * = 1
   fact = 1
   * : 2 1
   * = 2
  fact = 2
  * : 3 2
  * = 6
 fact = 6
-> 6

To reset a function to its untraced state, call untrace:

: (untrace 'fact)
-> fact
: (untrace '=0)
-> =0
: (untrace '*)
-> *

or simply use mapc:

: (mapc untrace '(fact =0 *))
-> *

Single-stepping

Single-stepping means to execute a function step by step, giving the programmer an opportunity to look more closely at what is happening. The function debug inserts a breakpoint into each top-level expression of a function. When the function is called, it stops at each breakpoint, displays the expression it is about to execute next (this expression is also stored into the global variable ^) and enters a read-eval-loop. The programmer can then

Thus, in the simplest case, single-stepping consists of just hitting ENTER repeatedly to step through the function.

To try it out, let's look at the stamp system function. You may need to have a look at

to understand this definition.
: (pp 'stamp)
(de stamp (Dat Tim)
   (and (=T Dat) (setq Dat (date T)))
   (default Dat (date) Tim (time T))
   (pack (dat$ Dat "-") " " (tim$ Tim T)) )
-> stamp
: (debug 'stamp)                       # Debug it
-> T
: (stamp)                              # Call it again
(and (=T Dat) (setq Dat (date T)))     # stopped at first expression
!                                      # ENTER
(default Dat (date) Tim (time T))      # second expression
!                                      # ENTER
(pack (dat$ Dat "-") " " (tim$ ...     # third expression
! Tim                                  # inspect 'Tim' variable
-> 41908
! (time Tim)                           # convert it
-> (11 38 28)
!                                      # ENTER
-> "2004-10-29 11:38:28"               # done, as there are only 3 expressions

Now we execute it again, but this time we want to look at what's happening inside the second expression.

: (stamp)                              # Call it again
(and (=T Dat) (setq Dat (date T)))
!                                      # ENTER
(default Dat (date) Tim (time T))
!                                      # ENTER
(pack (dat$ Dat "-") " " (tim$ ...     # here we want to look closer
! (d)                                  # debug this expression
-> T
!                                      # ENTER
(dat$ Dat "-")                         # stopped at first subexpression
! (e)                                  # evaluate it
-> "2004-10-29"
!                                      # ENTER
(tim$ Tim T)                           # stopped at second subexpression
! (e)                                  # evaluate it
-> "11:40:44"
!                                      # ENTER
-> "2004-10-29 11:40:44"               # done

The breakpoints still remain in the function body. We can see them when we pretty-print it:

: (pp 'stamp)
(de stamp (Dat Tim)
   (! and (=T Dat) (setq Dat (date T)))
   (! default Dat (date) Tim (time T))
   (! pack
      (! dat$ Dat "-")
      " "
      (! tim$ Tim T) ) )
-> stamp

To reset the function to its normal state, call unbug:

: (unbug 'stamp)

Often, you will not want to single-step a whole function. Just use v (see above) to insert a single breakpoint (the exclamation mark followed by a space) as CAR of an expression, and run your program. Execution will then stop there as described above; you can inspect the environment and continue execution with ENTER when you are done.


Functional I/O

Input and output in PicoLisp is functional, in the sense that there are not variables assigned to file descriptors, which need then to be passed to I/O functions for reading, writing and closing. Instead, these functions operate on implicit input and output channels, which are created and maintained as dynamic environments.

Standard input and standard output are the default channels. Try reading a single expression:

: (read)
(a b c)        # Console input
-> (a b c)

To read from a file, we redirect the input with in. Note that comments and whitespace are automatically skipped by read:

: (in "@lib.l" (read))
-> (de task (Key . Prg) (nond (...

The skip function can also be used directly. To get the first non-white character in the file with char:

: (in "@lib.l" (skip "#") (char))
-> "("

from searches through the input stream for given patterns. Typically, this is not done with Lisp source files (there are better ways), but for a simple example let's extract all items immediately following fact in the file,

: (in "@lib.l" (while (from "nond") (println (read))))
(Prg (del (assoc Key *Run) '*Run))
((pair "X") (or (pair (getd "X")) (expr "X")))
("Prg" (caar (idx "Var" "K")))

or the word following "(de " with till:

: (in "@lib.l" (from "(de ") (till " " T))
-> "task"

To read the contents of a whole file (or the rest of it starting from the current position):

: (in "f.l" (till NIL T))
-> "... file contents ..."

With line, a line of characters is read, either into a single transient symbol (the type used by PicoLisp for strings),

: (in "@doc/tut.html" (line T))
-> "<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN" "http://..."

or into a list of symbols (characters):

: (in "@doc/tut.html" (line))
-> ("<" "!" "D" "O" "C" "T" "Y" "P" "E" " " "H" "T" "M" "L" ...

line is typically used to read tabular data from a file. Additional arguments can split the line into fixed-width fields, as described in the reference manual. If, however, the data are of variable width, delimited by some special character, the split function can be used to extract the fields. A typical way to import the contents of such a file is:

(in '("bin/utf2" "importFile.txt")              # Pipe: Convert to UTF-8
   (until (eof)                                 # Process whole file
      (let L (split (line) "\t")                # TAB-delimited data
         ...                                    # process them

Some more examples with echo:

(in "a"                                         # Copy the first 40 Bytes
   (out "b"                                     # from file "a" to file "b"
      (echo 40) ) )

(in "@doc/tut.html"                             # Show the HTTP-header
   (line)
   (echo "<body>") )

(out "file.mac"                                 # Convert to Macintosh
   (in "file.txt"                               # from Unix or DOS format:
      (while (char)
         (prin
            (case @
               ("\r" NIL)                       # ignore CR
               ("\n" "\r")                      # convert LF to CR
               (T @) ) ) ) ) )                  # otherwise no change

(out "c"                                        # Merge the contents of "a"
   (in "b"                                      # and "b" into "c"
      (in "a"
         (while (read)                          # Read an item from "a",
            (println @ (in -1 (read))) ) ) ) )  # print it with an item from "b"


Scripting

There are two possibilities to get the PicoLisp interpreter into doing useful work: via command line arguments, or as a stand-alone script.

Command line arguments for the PicoLisp interpreter

The command line can specify either files for execution, or arbitrary Lisp expressions for direct evaluation (see Invocation): if an argument starts with a hyphen, it is evaluated, otherwise it is loaded as a file. A typical invocation might look like:

$ pil app/file1.l -main app/file2.l +

It loads the debugging environment, an application source file, calls the main function, and then loads another application source. In a typical development and debugging session, this line is often modified using the shell's history mechanisms, e.g. by inserting debugging statements:

$ pil app/file1.l -"trace 'foo" -main -"debug 'bar" app/file2.l +

Another convenience during debugging and testing is to put things into the command line (shell history) which would otherwise have to be done each time in the application's user interface:

$ pil app/file1.l -main app/file2.l -go -'login "name" "password"' +

The final production release of an application usually includes a shell script, which initializes the environment, does some bookkeeping and cleanup, and calls the application with a proper command line. It is no problem if the command line is long and complicated.

For small utility programs, however, this is overkill. Enter full PicoLisp scripts.

PicoLisp scripts

It is better to write a single executable file using the mechanisms of "interpreter files". If the first two characters in an executable file are "#!", the operating system kernel will pass this file to an interpreter program whose pathname is given in the first line (optionally followed by a single argument). This is fast and efficient, because the overhead of a subshell is avoided.

Let's assume you installed PicoLisp in the directory "/home/foo/pil21/", and put links to the executable and the installation directory as:

$ ln -s /home/foo/pil21 /usr/lib/picolisp
$ ln -s /usr/lib/picolisp/bin/picolisp /usr/bin
Then a simple hello-world script might look like:
#!/usr/bin/picolisp /usr/lib/picolisp/lib.l
(prinl "Hello world!")
(bye)

If you write this into a text file, and use chmod to set it to "executable", it can be executed like any other command. Note that (because # is the comment character in PicoLisp) the first line will not be interpreted, and you can still use that file as a normal command line argument to PicoLisp (useful during debugging).

Grab command line arguments from PicoLisp scripts

The fact that a hyphen causes evaluation of command line arguments can be used to implement command line options. The following script defines two functions a and f, and then calls (load T) to process the rest of the command line (which otherwise would be ignored because of the (bye) statement):

#!/usr/bin/picolisp /usr/lib/picolisp/lib.l

(de a ()
   (println '-a '-> (opt)) )

(de f ()
   (println '-f '-> (opt)) )

(load T)
(bye)
(opt retrieves the next command line option)

Calling this script (let's say we named it "testOpts") gives:

$ ./testOpts -f abc
-f -> "abc"
$ ./testOpts -a xxx  -f yyy
-a -> "xxx"
-f -> "yyy"

We have to be aware of the fact, however, that the aggregation of arguments like

$ ./testOpts -axxx  -fyyy

or

$ ./testOpts -af yyy

cannot be achieved with this simple and general mechanism of command line processing.

Run scripts from arbitrary places on the host file system

Utilities are typically used outside the context of the PicoLisp environment. All examples above assumed that the current working directory is the PicoLisp installation directory, which is usually all right for applications developed in that environment. Command line file arguments like "app/file1.l" will be properly found.

To allow utilities to run in arbitrary places on the host file system, the concept of home directory substitution was introduced. The interpreter remembers internally at start-up the pathname of its first argument (usually "lib.l"), and substitutes any leading "@" character in subsequent file names with that pathname. Thus, to run the above example in some other place, simply write:

$ /home/foo/pil21/pil @app/file1.l -main @app/file2.l +

that is, supply a full path name to the initial command (here 'pil'), or put it into your PATH variable, and prefix each file which has to be loaded from the PicoLisp home directory with a @ character. "Normal" files (not prefixed by @) will be opened or created relative to the current working directory as usual.

Stand-alone scripts will often want to load additional modules from the PicoLisp environment, beyond the "lib.l" we provided in the first line of the hello-world script. Typically, at least a call to

(load "@lib/misc.l")

(note the home directory substitution) will be included near the beginning of the script.

As a more complete example, here is a script which extracts the date, name and size of the latest official PicoLisp release version from the download web site, and prints it to standard output:

#!/usr/bin/picolisp /usr/lib/picolisp/lib.l

(load "@lib/misc.l" "@lib/http.l")

(use (@Date @Name @Size)
   (when
      (match
         '(@Date ~(chop " - <a href=\"") @Name "\"" ">"
             @Name ~(chop "</a> (") @Size )
         (client "software-lab.de" 80 "down.html"
            (from "Release Archive")
            (from "<li>")
            (till ",") ) )
      (prinl @Name)
      (prinl @Date " -- " @Size) ) )

(bye)