abu@software-lab.de

Perfection is attained
not when there is nothing left to add
but when there is nothing left to take away.
(Antoine de Saint-Exupéry)

The PicoLisp Reference

(c) Software Lab. Alexander Burger

This document describes the concepts, data types, and kernel functions of the PicoLisp system.

This is not a Lisp tutorial. For an introduction to Lisp, a traditional Lisp book like "Lisp" by Winston/Horn (Addison-Wesley 1981) is recommended. Note, however, that there are significant differences between PicoLisp and Maclisp (and even greater differences to Common Lisp).

Please take a look at the PicoLisp Tutorial for an explanation of some aspects of PicoLisp, and scan through the list of Frequently Asked Questions (FAQ).


Introduction

PicoLisp is the result of a language design study, trying to answer the question "What is a minimal but useful architecture for a virtual machine?". Because opinions differ about what is meant by "minimal" and "useful", there are many answers to that question, and people might consider other solutions more "minimal" or more "useful". But from a practical point of view, PicoLisp has proven to be a valuable answer to that question.

First of all, PicoLisp is a virtual machine architecture, and then a programming language. It was designed in a "bottom up" way, and "bottom up" is also the most natural way to understand and to use it: Form Follows Function.

PicoLisp has been used in several commercial and research programming projects since 1988. Its internal structures are simple enough, allowing an experienced programmer always to fully understand what's going on under the hood, and its language features, efficiency and extensibility make it suitable for almost any practical programming task.

In a nutshell, emphasis was put on four design objectives. The PicoLisp system should be

Simple
The internal data structure should be as simple as possible. Only one single data structure is used to build all higher level constructs.
Unlimited
There are no limits imposed upon the language due to limitations of the virtual machine architecture. That is, there is no upper bound in symbol name length, number digit counts, stack depth, or data structure and buffer sizes, except for the total memory size of the host machine.
Dynamic
Behavior should be as dynamic as possible ("run"-time vs. "compile"-time). All decisions are delayed until runtime where possible. This involves matters like memory management, dynamic symbol binding, and late method binding.
Practical
PicoLisp is not just a toy of theoretical value. It is in use since 1988 in actual application development, research and production.


The PicoLisp Machine

An important point in the PicoLisp philosophy is the knowledge about the architecture and data structures of the internal machinery. The high-level constructs of the programming language directly map to that machinery, making the whole system both understandable and predictable.

This is similar to assembly language programming, where the programmer has complete control over the machine.


The Cell

The PicoLisp virtual machine is both simpler and more powerful than most current (hardware) processors. At the lowest level, it is constructed from a single data structure called "cell":

         +-----+-----+
         | CAR | CDR |
         +-----+-----+

A cell is a pair of 64-bit machine words, which traditionally are called CAR and CDR in the Lisp terminology. These words can represent either a numeric value (scalar) or the address of another cell (pointer). All higher level data structures are built out of cells.

The type information of higher level data is contained in the pointers to these data. Assuming the implementation on a byte-addressed physical machine, and the pointer size being 8 bytes, each cell has a size of 16 bytes. Therefore, the pointer to a cell must point to a 16-byte boundary (a number which is a multiple of 16), and its bit-representation will look like:

      xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx0000

(the 'x' means "don't care"). For the individual data types, the pointer is adjusted to point to other parts of a cell, in effect setting some of the lower four bits to non-zero values. These bits are then used by the interpreter to determine the data type.

In any case, bit(0) - the least significant of these bits - is reserved as a mark bit for garbage collection.

Initially, all cells in the memory are unused (free), and linked together to form a "free list". To create higher level data types at runtime, cells are taken from that free list, and returned by the garbage collector when they are no longer needed. All memory management is done via that free list; there are no additional buffers, string spaces or special memory areas, with two exceptions:


Data Types

On the virtual machine level, PicoLisp supports

They are all built from the single cell data structure, and all runtime data cannot consist of any other types than these three.

The following diagram shows the complete data type hierarchy, consisting of the three base types and the symbol variations:

                       cell
                        |
            +-----------+-----------+
            |           |           |
         Number       Symbol       Pair
                        |
                        |
   +--------+-----------+-----------+
   |        |           |           |
  NIL   Internal    Transient    External


Numbers

A number can represent a signed integral value of arbitrary size. Internally, numeric values of up to 60 bits are stored in "short" numbers,

      xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxS010
i.e. the value is directly represented in the pointer, and doesn't take any heap space.

Numbers larger than that are "big" numbers, stored in heap cells. The CARs of one or more cells hold the number's "digits" (64 bits each), with the least significant digit first, while the CDRs point to the remaining digits.

         Bignum
         |
         V
      +-----+-----+
      | DIG |  |  |
      +-----+--+--+
               |
               V
            +-----+-----+
            | DIG |  |  |
            +-----+--+--+
                     |
                     V
                  +-----+-----+
                  | DIG | CNT |
                  +-----+-----+
The CDR of the final cell holds the remaining bits in a short number.

The pointer to a big number points into the middle of the CAR, with an offset of 4 from the cell's start address, and the sign bit in bit(3):

      xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxS100

Thus, a number is recognized by the interpreter when either bit(1) is non-zero (a short number) or bit(2) is non-zero (a big number).


Symbols

A symbol is more complex than a number. Each symbol has a value, and optionally a name and an arbitrary number of properties. The CDR of a symbol cell is also called VAL, and the CAR points to the symbol's tail. As a minimum, a symbol consists of a single cell, and has no name or properties:

            Symbol
            |
            V
      +-----+-----+
      |  /  | VAL |
      +-----+-----+

That is, the symbol's tail is empty (ZERO, as indicated by the '/' character).

The pointer to a symbol points to the CDR of the cell, with an offset of 8 bytes from the cell's start address. Therefore, the bit pattern of a symbol will be:

      xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx1000

Thus, a symbol is recognized by the interpreter when bit(3) is non-zero. (It should also be understood that both bit(2) and bit(1) must be zero, thus avoiding confusion with the number types.)

A property is a key-value pair, represented by a cons pair in the symbol's tail. This is called a "property list". The property list may be terminated by a number (short or big) representing the symbol's name. In the following example, a symbol with the name "abcdefghijklmno" has three properties: A KEY/VAL pair, a cell with only a KEY, and another KEY/VAL pair.

            Symbol
            |
            V
      +-----+-----+                                +----------+---------+
      |  |  | VAL |                                |'hgfedcba'|'onmlkji'|
      +--+--+-----+                                +----------+---------+
         | tail                                       ^
         |                                            |
         V                                            | name
         +-----+-----+     +-----+-----+     +-----+--+--+
         |  |  |  ---+---> | KEY |  ---+---> |  |  |  |  |
         +--+--+-----+     +-----+-----+     +--+--+-----+
            |                                   |
            V                                   V
            +-----+-----+                       +-----+-----+
            | VAL | KEY |                       | VAL | KEY |
            +-----+-----+                       +-----+-----+

Each property in a symbol's tail is either a symbol (like the single KEY above, then it represents the boolean value T) or a cons pair with the property key in its CDR and the property value in its CAR. In both cases, the key should be a symbol, because searches in the property list are performed using pointer comparisons.

The name of a symbol is stored as a number at the end of the tail. It contains the characters of the name in UTF-8 encoding, using between one and seven bytes in a short number, or eight bytes in a bignum cell. The first byte of the first character, for example, is stored in the lowest 8 bits of the number.

All symbols have the above structure, but depending on scope and accessibility there are actually four types of symbols: NIL, internal, transient and external symbols.


NIL

NIL is a special symbol which exists exactly once in the whole system. It is used

For that, NIL has a special structure:

      NIL:  /
            |
            V
      +-----+-----+-----+-----+
      |'LIN'|  /  |  /  |  /  |
      +-----+-----+-----+-----+

The reason for that structure is NIL's dual nature both as a symbol and as a list:

These requirements are fulfilled by the above structure.


Internal Symbols

Internal symbols are all those "normal" symbols, as they are used for function definitions and variable names. They are "interned" into an index structure, so that it is possible to find an internal symbol by searching for its name.

There cannot be two different symbols with the same name in the same namespace.

Initially, a new internal symbol's VAL is NIL.


Transient Symbols

Transient symbols are only interned into an index structure for a certain time (e.g. while reading the current source file), and are released after that. That means, a transient symbol cannot be accessed then by its name, and there may be several transient symbols in the system having the same name.

Transient symbols are used

Initially, a new transient symbol's VAL is that symbol itself.

A transient symbol without a name can be created with the box or new functions.


External Symbols

External symbols reside in a database file (or other resources, see *Ext), and are loaded into memory - and written back to the file - dynamically as needed, and transparently to the programmer. They are kept in memory ("cached") as long as they are accessible ("referred to") from other parts of the program, or when they were modified but not yet written to the database file (by commit).

The interpreter recognizes external symbols internally by an additional tag bit in the tail structure.

There cannot be two different external symbols with the same name. External symbols are maintained in index structures while they are loaded into memory, and have their external location (disk file and block offset) directly coded into their names (more details here).

Initially, a new external symbol's VAL is NIL, unless otherwise specified at creation time.


Lists

A list is a sequence of one or more cells (cons pairs), holding numbers, symbols, or cons pairs.

      |
      V
      +-----+-----+
      | any |  |  |
      +-----+--+--+
               |
               V
               +-----+-----+
               | any |  |  |
               +-----+--+--+
                        |
                        V
                        ...

Lists are used in PicoLisp to implement composite data structures like arrays, trees, stacks or queues.

In contrast to lists, numbers and symbols are collectively called "Atoms".

Typically, the CDR of each cell in a list points to the following cell, except for the last cell which points to NIL. If, however, the CDR of the last cell points to an atom, that cell is called a "dotted pair" (because of its I/O syntax with a dot '.' between the two values).


Memory Management

The PicoLisp interpreter has complete knowledge of all data in the system, due to the type information associated with every pointer. Therefore, an efficient garbage collector mechanism can easily be implemented. PicoLisp employs a simple but fast mark-and-sweep garbage collector.

As the collection process is very fast (in the order of milliseconds per megabyte), it was not necessary to develop more complicated, time-consuming and error-prone garbage collection algorithms (e.g. incremental collection). A compacting garbage collector is also not necessary, because the single cell data type cannot cause heap fragmentation.


Programming Environment

Lisp was chosen as the programming language, because of its clear and simple structure.

In some previous versions, a Forth-like syntax was also implemented on top of a similar virtual machine (Lifo). Though that language was more flexible and expressive, the traditional Lisp syntax proved easier to handle, and the virtual machine can be kept considerably simpler. PicoLisp inherits the major advantages of classical Lisp systems like

In the following, some concepts and peculiarities of the PicoLisp language and environment are described.


Installation

PicoLisp supports two installation strategies: Local and Global.

Normally, if you didn't build PicoLisp yourself but installed it with your operating system's package manager, you will have a global installation. This allows system-wide access to the executable and library/documentation files.

To get a local installation, you can directly download the PicoLisp tarball, and follow the instructions in the INSTALL file.

A local installation will not interfere in any way with the world outside its directory. There is no need to touch any system locations, and you don't have to be root to install it. Many different versions - or local modifications - of PicoLisp can co-exist on a single machine.

Note that you are still free to have local installations along with a global installation, and invoke them explicitly as desired.

Most examples in the following apply to a global installation.


Invocation

When PicoLisp is invoked from the command line, an arbitrary number of arguments may follow the command name.

By default, each argument is the name of a file to be executed by the interpreter. If, however, the argument's first character is a hyphen '-', then the rest of that argument is taken as a Lisp function call (without the surrounding parentheses), and a hyphen by itself as an argument stops evaluation of the rest of the command line (it may be processed later using the argv and opt functions). This whole mechanism corresponds to calling (load T).

A special case is if the last argument is a single '+'. This will switch on debug mode (the *Dbg global variable) and discard the '+'. In that case, a file "~/.pil/rc" (if it exists) will be loaded, which can contain arbitrary statements and definitions (e.g. to initialize the readline(3) history).

As a convention, PicoLisp source files have the extension ".l".

Note that the PicoLisp executable itself does not expect or accept any command line flags or options (except the '+', see above). They are reserved for application programs.

The simplest and shortest invocation of PicoLisp does nothing, and exits immediately by calling bye:

$ picolisp -bye
$

In interactive mode, the PicoLisp interpreter (see load) will also exit when Ctrl-D is entered:

$ picolisp
:           # Typed Ctrl-D
$

To start up the standard PicoLisp environment, several files should be loaded. The most commonly used things are in "lib.l" and in a bunch of other files, which are in turn loaded by "ext.l". Thus, a typical call would be:

$ picolisp lib.l ext.l

The recommended way, however, is to call the "pil" shell script, which includes "lib.l" and "ext.l". Given that your current project is loaded by some file "myProject.l" and your startup function is main, your invocation would look like:

$ pil myProject.l -main

For interactive development it is recommended to enable debugging mode, to get readline(3) line history, single-stepping, tracing and other debugging utilities.

$ pil myProject.l -main +

This is - in a local installation - equivalent to

$ ./pil myProject.l -main +

In any case, the directory part of the first file name supplied (normally, the path to "lib.l" as called by 'pil') is remembered internally as the PicoLisp Home Directory. This path is later automatically substituted for any leading "@" character in file name arguments to I/O functions (see path).


Input/Output

In Lisp, each internal data structure has a well-defined external representation in human-readable format. All kinds of data can be written to a file, and restored later to their original form by reading that file.

For all input functions besides wr, rd and echo the input is assumed to be valid UTF-8, consisting only of characters allowed in picolisp symbol names.

In normal operation, the PicoLisp interpreter continually executes an infinite "read-eval-print loop". It reads one expression at a time, evaluates it, and prints the result to the console. Any input into the system, like data structures and function definitions, is done in a consistent way no matter whether it is entered at the console or read from a file.

Comments can be embedded in the input stream with the hash # character. Everything up to the end of that line will be ignored by the reader.

: (* 1 2 3)  # This is a comment
-> 6

A comment spanning several lines (a block comment) may be enclosed between #{ and }#. Block comments may be nested.

Here is the I/O syntax for the individual PicoLisp data types (numbers, symbols and lists) and for read-macros:


Numbers

A number consists of an arbitrary number of digits ('0' through '9'), optionally preceded by a sign character ('+' or '-'). Legal number input is:

: 7
-> 7
: -12345678901245678901234567890
-> -12345678901245678901234567890

Fixpoint numbers can be input by embedding a decimal point '.', and setting the global variable *Scl appropriately:

: *Scl
-> 0

: 123.45
-> 123
: 456.78
-> 457

: (setq *Scl 3)
-> 3
: 123.45
-> 123450
: 456.78
-> 456780

Thus, fixpoint input simply scales the number to an integer value corresponding to the number of digits in *Scl.

Formatted output of scaled fixpoint values can be done with the format and round functions:

: (format 1234567890 2)
-> "12345678.90"
: (format 1234567890 2 "." ",")
-> "12,345,678.90"


Symbols

The reader is able to recognize the individual symbol types from their syntactic form. A symbol name should - of course - not look like a legal number (see above).

In general, symbol names are case-sensitive. car is not the same as CAR.


NIL

Besides the standard form, NIL is also recognized as (), [] or "".

: NIL
-> NIL
: ()
-> NIL
: ""
-> NIL

Output will always appear as NIL.


Internal Symbols

Internal symbol names can consist of any printable (non-whitespace) character, except for the following meta characters:

   "  '  (  )  ,  [  ]  `  { } ~

It is possible, though, to include these special characters into symbol names by escaping them with a backslash '\'.

The dot '.' has a dual nature. It is a meta character when standing alone, denoting a dotted pair, but can otherwise be used in symbol names.

As a rule, anything not recognized by the reader as another data type will be returned as an internal symbol.


Transient Symbols

A transient symbol is anything surrounded by double quotes '"'. With that, it looks like - and can be used as - a string constant in other languages. However, it is a real symbol, and may be assigned a value or a function definition, and properties.

Initially, a transient symbol's value is that symbol itself, so that it does not need to be quoted for evaluation:

: "This is a string"
-> "This is a string"

However, care must be taken when assigning a value to a transient symbol. This may cause unexpected behavior:

: (setq "This is a string" 12345)
-> 12345
: "This is a string"
-> 12345

The name of a transient symbol can contain any character except the null-byte. Control characters can be written with a preceding hat '^' character. A hat or a double quote character can be escaped with a backslash '\', and a backslash itself has to be escaped with another backslash.

: "We^Ird\\Str\"ing"
-> "We^Ird\\Str\"ing"
: (chop @)
-> ("W" "e" "^I" "r" "d" "\\" "S" "t" "r" "\"" "i" "n" "g")

The combination of a backslash followed by 'b', 'e', 'n', 'r' or 't' is replaced with backspace ("^H"), escape ("^["), newline ("^J"), return ("^M") or TAB ("^I"), respectively.

: "abc\tdef\r"
-> "abc^Idef^M"

A decimal number between two backslashes can be used to specify any unicode character directly.

: "äöü\8364\xyz"
-> "äöü€xyz"

A backslash in a transient symbol name at the end of a line discards the newline, and continues the name in the next line. In that case, all leading spaces and tabs in that line are discarded, to allow proper source code indentation.

: "abc\
   def"
-> "abcdef"
: "x \
   y \
   z"
-> "x y z"

The index for transient symbols is local when loading a source file. With that mechanism, it is possible to create symbols with a local access scope, not accessible from other parts of the program.

A special case of transient symbols are anonymous symbols. These are symbols without name (see box, box? or new). They print as a dollar sign ($) followed by a decimal digit string (actually their machine address).


External Symbols

External symbol names are surrounded by braces ('{' and '}'). The characters of the symbol's name itself identify the physical location of the external object. This is the number of the database file minus 1 in hax notation (i.e. hexadecimal/alpha notation, where '@' is zero, 'A' is 1 and 'O' is 15 (from "alpha" to "omega")), immediately followed (without a hyphen) the starting block in octal ('0' through '7').

The database file is omitted for the first (default) file.


Lists

Lists are surrounded by parentheses ('(' and ')').

(A) is a list consisting of a single cell, with the symbol A in its CAR, and NIL in its CDR.

(A B C) is a list consisting of three cells, with the symbols A, B and C respectively in their CAR, and NIL in the last cell's CDR.

(A . B) is a "dotted pair", a list consisting of a single cell, with the symbol A in its CAR, and B in its CDR.

PicoLisp has built-in support for reading and printing simple circular lists. If the dot in a dotted-pair notation is immediately followed by a closing parenthesis, it indicates that the CDR of the last cell points back to the beginning of that list.

: (let L '(a b c) (conc L L))
-> (a b c .)
: (cdr '(a b c .))
-> (b c a .)
: (cddddr '(a b c .))
-> (b c a .)

A similar result can be achieved with the function circ. Such lists must be used with care, because many functions won't terminate or will crash when given such a list.


Read-Macros

Read-macros in PicoLisp are special forms that are recognized by the reader, and modify its behavior. Note that they take effect immediately while reading an expression, and are not seen by the eval in the main loop.

The most prominent read-macro in Lisp is the single quote character "'", which expands to a call of the quote function. Note that the single quote character is also printed instead of the full function name.

: '(a b c)
-> (a b c)
: '(quote . a)
-> 'a
: (cons 'quote 'a)   # (quote . a)
-> 'a
: (list 'quote 'a)   # (quote a)
-> '(a)

A comma (,) will cause the reader to collect the following data item into an idx tree in the global variable *Uni, and to return a previously inserted equal item if present. This makes it possible to create a unique list of references to data which do normally not follow the rules of pointer equality. If the value of *Uni is T, the comma read macro mechanism is disabled.

A single backquote character "`" will cause the reader to evaluate the following expression, and return the result.

: '(a `(+ 1 2 3) z)
-> (a 6 z)

A tilde character ~ inside a list will cause the reader to evaluate the following expression, and (destructively) splice the result into the list.

: '(a b c ~(list 'd 'e 'f) g h i)
-> (a b c d e f g h i)

When a tilde character is used to separate two symbol names (without surrounding whitespace), the first is taken as a namespace to look up the second.

: 'libA~foo  # Look up 'foo' in namespace 'libA'
-> libA~foo  # "foo" is not interned in the current namespace

Reading libA~foo is equivalent to switching the current namespace search order to libA only (with symbols), reading the symbol foo, and then switching back to the original search order.

%~foo temporarily switches the search order to the CDR of the current namespace list.

Brackets ('[' and ']') can be used as super parentheses. A closing bracket will match the innermost opening bracket, or all currently open parentheses.

: '(a (b (c (d]
-> (a (b (c (d))))
: '(a (b [c (d]))
-> (a (b (c (d))))

Finally, reading the sequence '{}' will result in a new anonymous symbol with value NIL, equivalent to a call to box without arguments.

: '({} {} {})
-> ($177066763035351 $177066763035353 $177066763035355)
: (mapcar val @)
-> (NIL NIL NIL)


Evaluation

PicoLisp tries to evaluate any expression encountered in the read-eval-print loop. Basically, it does so by applying the following three rules:

: 1234
-> 1234        # Number evaluates to itself
: *Pid
-> 22972       # Symbol evaluates to its VAL
: (+ 1 2 3)
-> 6           # List is evaluated as a function call

For the third rule, however, things get a bit more involved. First - as a special case - if the CAR of the list is a number, the whole list is returned as it is:

: (1 2 3 4 5 6)
-> (1 2 3 4 5 6)

This is not really a function call, but just a convenience to avoid having to quote simple data lists. The interpreter needs to check it anyway, and returning the list (instead of throwing an error) is a lot faster than calling the quote function.

Otherwise, if the CAR is a symbol or a list, PicoLisp tries to obtain an executable function from that, by either using the symbol's value, or by evaluating the list.

What is an executable function? Or, said in another way, what can be applied to a list of arguments, to result in a function call? A legal function in PicoLisp is

either
a number. When a number is used as a function, it is simply taken as a pointer to executable code that will be called with the list of (unevaluated) arguments as its single parameter. It is up to that code to evaluate the arguments, or not. Some functions do not evaluate their arguments (e.g. quote) or evaluate only some of their arguments (e.g. setq).
or
a lambda expression. A lambda expression is a list, whose CAR is either a symbol or a list of symbols, and whose CDR is a list of expressions. Note: In contrast to other Lisp implementations, the symbol LAMBDA itself does not exist in PicoLisp but is implied from context.

A few examples should help to understand the practical consequences of these rules. In the most common case, the CAR will be a symbol defined as a function, like the * in:

: (* 1 2 3)    # Call the function '*'
-> 6

Inspecting the VAL of * gives

: *            # Get the VAL of the symbol '*'
-> 67318096

The VAL of * is a number. In fact, it is the numeric representation of a function pointer, i.e. a pointer to executable code. This is the case for all built-in functions of PicoLisp.

Other functions in turn are written as Lisp expressions:

: (de foo (X Y)            # Define the function 'foo'
   (* (+ X Y) (+ X Y)) )
-> foo
: (foo 2 3)                # Call the function 'foo'
-> 25
: foo                      # Get the VAL of the symbol 'foo'
-> ((X Y) (* (+ X Y) (+ X Y)))

The VAL of foo is a list. It is the list that was assigned to foo with the de function. It would be perfectly legal to use setq instead of de:

: (setq foo '((X Y) (* (+ X Y) (+ X Y))))
-> ((X Y) (* (+ X Y) (+ X Y)))
: (foo 2 3)
-> 25

If the VAL of foo were another symbol, that symbol's VAL would be used instead to search for an executable function.

As we said above, if the CAR of the evaluated expression is not a symbol but a list, that list is evaluated to obtain an executable function.

: ((intern (pack "c" "a" "r")) (1 2 3))
-> 1

Here, the intern function returns the symbol car whose VAL is used then. It is also legal, though quite dangerous, to use the code-pointer directly:

: *
-> 67318096
: ((* 2 33659048) 1 2 3)
-> 6
: ((quote . 67318096) 1 2 3)
-> 6
: ((quote . 1234) (1 2 3))
Segmentation fault

When an executable function is defined in Lisp itself, we call it a lambda expression. A lambda expression always has a list of executable expressions as its CDR. The CAR, however, must be a either a list of symbols, or a single symbol, and it controls the evaluation of the arguments to the executable function according to the following rules:

When the CAR is a list of symbols
For each of these symbols an argument is evaluated, then the symbols are bound simultaneously to the results. The body of the lambda expression is executed, then the VAL's of the symbols are restored to their original values. This is the most common case, a fixed number of arguments is passed to the function.
As a special case, a single-level list of symbols may be passed instead of a single parameter here, resulting in a restricted form of destructuring bind.
Otherwise, when the CAR is the symbol @
All arguments are evaluated and the results kept internally in a list. The body of the lambda expression is executed, and the evaluated arguments can be accessed sequentially with the args, next, arg and rest functions. This allows to define functions with a variable number of evaluated arguments.
Otherwise, when the CAR is a single symbol
The symbol is bound to the whole unevaluated argument list. The body of the lambda expression is executed, then the symbol is restored to its original value. This allows to define functions with unevaluated arguments. Any kind of interpretation and evaluation of the argument list can be done inside the expression body.

In all cases, the return value is the result of the last expression in the body.

: (de foo (X Y Z)                   # CAR is a list of symbols
   (list X Y Z) )                   # Return a list of all arguments
-> foo
: (foo (+ 1 2) (+ 3 4) (+ 5 6))
-> (3 7 11)                         # all arguments are evaluated
: (de foo @                         # CAR is the symbol '@'
   (list (next) (next) (next)) )    # Return the first three arguments
-> foo
: (foo (+ 1 2) (+ 3 4) (+ 5 6))
-> (3 7 11)                         # all arguments are evaluated
: (de foo X                         # CAR is a single symbol
   X )                              # Return the argument
-> foo
: (foo (+ 1 2) (+ 3 4) (+ 5 6))
-> ((+ 1 2) (+ 3 4) (+ 5 6))        # the whole unevaluated list is returned

Note that these forms can also be combined. For example, to evaluate only the first two arguments, bind the results to X and Y, and bind all other arguments (unevaluated) to Z:

: (de foo (X Y . Z)                 # CAR is a list with a dotted-pair tail
   (list X Y Z) )                   # Return a list of all arguments
-> foo
: (foo (+ 1 2) (+ 3 4) (+ 5 6))
-> (3 7 ((+ 5 6)))                  # Only the first two arguments are evaluated

Or, a single argument followed by a variable number of arguments:

: (de foo (X . @)                   # CAR is a dotted-pair with '@'
   (println X)                      # print the first evaluated argument
   (while (args)                    # while there are more arguments
      (println (next)) ) )          # print the next one
-> foo
: (foo (+ 1 2) (+ 3 4) (+ 5 6))
3                                   # X
7                                   # next argument
11                                  # and the last argument
-> 11

In general, if more than the expected number of arguments is supplied to a function, these extra arguments will be ignored. Missing arguments default to NIL.


Shared Libraries

Analogous to built-in functions (which are written in PilSrc (based on LLVM)) in the interpreter kernel), PicoLisp functions may also be defined in shared object files (called "DLLs" on some systems). The coding style, register usage, argument passing etc. follow the same rules as for normal built-in functions.

Note that this has nothing to do with external (e.g. third-party) library functions called with native.

When the interpreter encounters a symbol supposed to be called as a function, without a function definition, but with a name of the form "lib:sym", then - instead of throwing an "undefined"-error - it tries to locate a shared object file with the name lib.so and a function sym, and stores a pointer to this code in the symbol's value. From that point, this symbol lib:sym keeps that function definition, and is undistinguishable from built-in functions. Future calls to this function do not require another library search.

A consequence of this lookup mechanism, however, is the fact that such symbols cannot be used directly in a function-passing context (i.e. "apply" them) like

(apply + (1 2 3))
(mapcar inc (1 2 3))

These calls work because + and inc already have a (function) value at this point. Applying a shared library function like

(apply ext:Base64 (1 2 3))

works only if ext:Base64 was either called before (and thus automatically received a function definition), or was fetched explicitly with (getd 'ext:Base64).

Therefore, it is recommended to always apply such functions by passing the symbol itself and not just the value:

(apply 'ext:Base64 (1 2 3))


Coroutines

Coroutines are independent execution contexts. They may have multiple entry and exit points, and preserve their environment (stack, symbol bindings, namespaces, catch/throw and I/O frames) between invocations.

A coroutine is identified by a tag. This tag can be passed to other functions, and (re)invoked as needed. In this regard coroutines are similar to "continuations" in other languages.

Tags may be of any type (pointer equality is used for comparison), but symbolic tags are more efficient for large numbers of coroutines. They cache a pointer to the internal data structure in a property with key zero (which is inaccessible with the put and get functions).

A coroutine is created by calling co. Its prg body will be executed, and unless yield is called at some point, the coroutine will "fall off" at the end and disappear.

The initial value of This is bound and preserved in the coroutine environment.

When yield is called, control is either transferred back to the caller, or to some other - explicitly specified, and already running - coroutine.

A coroutine is stopped and disposed when

Before using many coroutines, make sure you have sufficient stack space, e.g. by calling

$ ulimit -s unlimited

Without that, the stack limit in Linux is typically 8 MiB.


Interrupt

During the evaluation of an expression, the PicoLisp interpreter can be interrupted at any time by hitting Ctrl-C. It will then enter the breakpoint routine, as if ! were called.

Hitting ENTER at that point will continue evaluation, while Ctrl-D or (quit) will abort evaluation and return the interpreter to the top level. See also debug, e, ^ and *Dbg

Other interrupts may be handled by alarm, sigio, *Hup, *Winch, *Sig[12], *TStp[12] and *Term.


Error Handling

When a runtime error occurs, execution is stopped and an error handler is entered.

The error handler resets the I/O channels to the console, and displays the location (if possible) and the reason of the error, followed by an error message. That message is also stored in the global *Msg, and the location of the error in ^. If the VAL of the global *Err is non-NIL it is executed as a prg body. If the standard input is from a terminal, a read-eval-print loop (with a question mark "?" as prompt) is entered (the loop is exited when an empty line is input). Then all pending finally expressions are executed, all variable bindings restored, and all files closed. If the standard input is not from a terminal, the interpreter terminates. Otherwise it is reset to its top-level state.

: (de foo (A B) (badFoo A B))       # 'foo' calls an undefined symbol
-> foo
: (foo 3 4)                         # Call 'foo'
!? (badFoo A B)                     # Error handler entered
badFoo -- Undefined
? A                                 # Inspect 'A'
-> 3
? B                                 # Inspect 'B'
-> 4
?                                   # Empty line: Exit
:

Errors can be caught with catch, if a list of substrings of possible error messages is supplied for the first argument. In such a case, the matching substring (or the whole error message if the substring is NIL) is returned.

An arbitrary error can be thrown explicitly with quit.


@ Result

In certain situations, the result of the last evaluation is stored in the VAL of the symbol @. This can be very convenient, because it often makes the assignment to temporary variables unnecessary.

This happens in two - only superficially similar - situations:

load
In read-eval loops, the last three results which were printed at the console are available in @@@, @@ and @, in that order (i.e the latest result is in @).
: (+ 1 2 3)
-> 6
: (/ 128 4)
-> 32
: (- @ @@)        # Subtract the last two results
-> 26
Flow functions
Flow functions store the non-nil results of controlling expressions, and logic functions their non-nil results, in @.
: (while (read) (println 'got: @))
abc            # User input
got: abc       # print result
123            # User input
got: 123       # print result
NIL
-> 123

: (setq L (1 2 3 4 5 1 2 3 4 5))
-> (1 2 3 4 5 1 2 3 4 5)
: (and (member 3 L) (member 3 (cdr @)) (set @ 999))
-> 999
: L
-> (1 2 3 4 5 1 2 999 4 5)

Functions with controlling expressions are case, casq, prog1, prog2, and the bodies of *Run tasks.

Functions with conditional expressions are and, cond, do, for, if, if2, ifn, loop, nand, nond, nor, not, or, state, unless, until, when and while.

@ is generally local to functions and methods, its value is automatically saved upon function entry and restored at exit.


Comparing

In PicoLisp, it is legal to compare data items of arbitrary type. Any two items are either

Identical
They are the same memory object (pointer equality). For example, two internal symbols with the same name are identical. And short numbers (up to 60 bits plus sign) are also equivalent to "pointer"-equal.
Equal
They are equal in every respect (structure equality), but need not to be identical. Examples are numbers with the same value, transient symbols with the same name or lists with equal elements.
Or they have a well-defined ordinal relationship
Numbers are comparable by their numeric value, strings by their name, and lists recursively by their elements (if the CAR's are equal, their CDR's are compared). For differing types, the following rule applies: Numbers are less than symbols, and symbols are less than lists. As special cases, NIL is always less than anything else, and T is always greater than anything else.

To demonstrate this, sort a list of mixed data types:

: (sort '("abc" T (d e f) NIL 123 DEF))
-> (NIL 123 DEF "abc" (d e f) T)

See also max, min, rank, <, =, > etc.


OO Concepts

PicoLisp comes with built-in object oriented extensions. There seems to be a common agreement upon three criteria for object orientation:

Encapsulation
Code and data are encapsulated into objects, giving them both a behavior and a state. Objects communicate by sending and receiving messages.
Inheritance
Objects are organized into classes. The behavior of an object is inherited from its class(es) and superclass(es).
Polymorphism
Objects of different classes may behave differently in response to the same message. For that, classes may define different methods for each message.

PicoLisp implements both objects and classes with symbols. Object-local data are stored in the symbol's property list, while the code (methods) and links to the superclasses are stored in the symbol's VAL (encapsulation).

In fact, there is no formal difference between objects and classes (except that objects usually are anonymous symbols containing mostly local data, while classes are named internal symbols with an emphasis on method definitions). At any time, a class may be assigned its own local data (class variables), and any object can receive individual method definitions in addition to (or overriding) those inherited from its (super)classes.

PicoLisp supports multiple inheritance. The VAL of each object is a (possibly empty) association list of message symbols and method bodies, concatenated with a list of classes. When a message is sent to an object, it is searched in the object's own method list, and then (with a left-to-right depth-first search) in the tree of its classes and superclasses. The first method found is executed and the search stops. The search may be explicitly continued with the extra and super functions.

Thus, which method is actually executed when a message is sent to an object depends on the classes that the object is currently linked to (polymorphism). As the method search is fully dynamic (late binding), an object's type (i.e. its classes and method definitions) can be changed even at runtime!

While a method body is being executed, the global variable This is set to the current object, allowing the use of the short-cut property functions =:, : and ::.


Database

On the lowest level, a PicoLisp database is just a collection of external symbols. They reside in a database file, and are dynamically swapped in and out of memory. Only one database can be open at a time (pool).

In addition, further external symbols can be specified to originate from arbitrary sources via the *Ext mechanism.

Whenever an external symbol's value or property list is accessed, it will be automatically fetched into memory, and can then be used like any other symbol. Modifications will be written to disk only when commit is called. Alternatively, all modifications since the last call to commit can be discarded by calling rollback.

Note that a property with the key NIL is a volatile property, which is held only in memory and not written to disk on commit, and discarded by rollback. Volatile properties can be used by applications for any kind of temporary data.


Transactions

In the typical case there will be multiple processes operating on the same database. These processes should be all children of the same parent process, which takes care of synchronizing read/write operations and heap contents. Then a database transaction is normally initiated by calling (dbSync), and closed by calling (commit 'upd). Short transactions, involving only a single DB operation, are available in functions like new! and methods like put!> (by convention with an exclamation mark), which implicitly call (dbSync) and (commit 'upd) themselves.

A transaction proceeds through five phases:

  1. dbSync waits to get a lock on the root object *DB. Other processes continue reading and writing meanwhile.
  2. dbSync calls sync to synchronize with changes from other processes. We hold the shared lock, but other processes may continue reading.
  3. We make modifications to the internal state of external symbols with put>, set>, lose> etc. We - and also other processes - can still read the DB.
  4. We call (commit 'upd). commit obtains an exclusive lock (no more read operations by other processes), writes an optional transaction log, and then all modified symbols. As upd is passed to 'commit', other processes synchronize with these changes.
  5. Finally, all locks are released by 'commit'.


Entities / Relations

The symbols in a database can be used to store arbitrary information structures. In typical use, some symbols represent nodes of search trees, by holding keys, values, and links to subtrees in their VAL's. Such a search tree in the database is called index.

For the most part, other symbols in the database are objects derived from the +Entity class.

Entities depend on objects of the +relation class hierarchy. Relation-objects manage the property values of entities, they define the application database model and are responsible for the integrity of mutual object references and index trees.

Relations are stored as properties in the entity classes, their methods are invoked as daemons whenever property values in an entity are changed. When defining an +Entity class, relations are defined - in addition to the method definitions of a normal class - with the rel function. Predefined relation classes include


Pilog (PicoLisp Prolog)

A declarative language is built on top of PicoLisp, that has the semantics of Prolog, but uses the syntax of Lisp.

For an explanation of Prolog's declarative programming style, an introduction like "Programming in Prolog" by Clocksin/Mellish (Springer-Verlag 1981) is recommended.

Facts and rules can be declared with the be function. For example, a Prolog fact 'likes(john,mary).' is written in Pilog as:

(be likes (John Mary))

and a rule 'likes(john,X) :- likes(X,wine), likes(X,food).' is in Pilog:

(be likes (John @X) (likes @X wine) (likes @X food))

As in Prolog, the difference between facts and rules is that the latter ones have conditions, and usually contain variables.

A variable in Pilog is any symbol starting with an at-mark character ("@"), i.e. a pat? symbol. The symbol @ itself can be used as an anonymous variable: It will match during unification, but will not be bound to the matched values.

The cut operator of Prolog (usually written as an exclamation mark (!)) is the symbol T in Pilog.

An interactive query can be done with the ? function:

(? (likes John @X))

This will print all solutions, waiting for user input after each line. If Ctrl-D or a non-empty line is typed, it will terminate.

Pilog can be called from Lisp and vice versa:


Naming Conventions

It was necessary to introduce - and adhere to - a set of conventions for PicoLisp symbol names. Because all (internal) symbols have a global scope, and each symbol can only have either a value or function definition, it would otherwise be very easy to introduce name conflicts. Besides this, source code readability is increased when the scope of a symbol is indicated by its name.

These conventions are not hard-coded into the language, but should be so into the head of the programmer. Here are the most commonly used ones:

For example, a local variable could easily overshadow a function definition:

: (de max-speed (car)
   (.. (get car 'speeds) ..) )
-> max-speed

Inside the body of max-speed (and all other functions called during that execution) the kernel function car is redefined to some other value, and will surely crash if something like (car Lst) is executed. Instead, it is safe to write:

: (de max-speed (Car)            # 'Car' with upper case first letter
   (.. (get Car 'speeds) ..) )
-> max-speed

Note that there are also some strict naming rules (as opposed to the voluntary conventions) that are required by the corresponding kernel functionalities, like:

With that, the last of the above conventions (local functions start with an underscore) is not really necessary, because true local scope can be enforced with transient symbols.

The symbols T and NIL are global constants, so care should be taken not to bind them to some other value by mistake:

(de foo (R S T)
   ...

However, lint will issue a warning in such a case.


Breaking Traditions

PicoLisp does not try very hard to be compatible with traditional Lisp systems. If you are used to some other Lisp dialects, you may notice the following differences:

Case Sensitivity
PicoLisp distinguishes between upper case and lower case characters in symbol names. Thus, CAR and car are different symbols, which was not the case in traditional Lisp systems.
QUOTE
In traditional Lisp, the QUOTE function returns its first unevaluated argument. In PicoLisp, on the other hand, quote returns all (unevaluated) argument(s).
LAMBDA
The LAMBDA function, in some way at the heart of traditional Lisp, is completely missing (and quote is used instead).
PROG
The PROG function of traditional Lisp, with its GOTO and ENTER functionality, is also missing. PicoLisp's prog function is just a simple sequencer (as PROGN in some Lisps).
Function/Value
In PicoLisp, a symbol cannot have a value and a function definition at the same time. Though this is a disadvantage at first sight, it allows a completely uniform handling of functional data.


Function Reference

This section provides a reference manual for the kernel functions, and some extensions. See the thematically grouped list of indexes below.

Though PicoLisp is a dynamically typed language (resolved at runtime, as opposed to statically (compile-time) typed languages), many functions can only accept and/or return a certain set of data types. For each function, the expected argument types and return values are described with the following abbreviations:

The primary data types:

Other (derived) data types

Arguments evaluated by the function in the "normal" way are quoted (prefixed with the single quote character "'"). Other arguments are either not evaluated, or may be evaluated depending on the context.

For example, the function setq evaluates every second argument (giving any kind of value), while it does not evaluate the others (vars, here typically symbols). This could be specified as:

   (setq var1 'any1 var2 'any2 ..) -> any

A dotted pair notation in the argument list like (... 'any . prg) indicates an unevaluated list of further arguments.

Arguments in brackets '[' and ']' are optional.

A B C D E F G H I J K L M N O P Q R S T U V W X Y Z Other

Symbol Functions
new sym str char name nsp sp? pat? fun? all symbols -symbols private local export import all* intern extern qsym loc box? str? ext? touch zap length size format chop pack glue pad align center text wrap pre? sub? low? upp? lowc uppc fold val getd set setq def de dm recur undef redef daemon patch swap xchg on off onOff zero one default expr subr let let? use buf accu push push1 push1q pop ++ shift cut del queue fifo rid enum enum? idx lup cache locale dirname basename
Property Access
put get prop ; =: : :: putl getl wipe meta
Predicates
atom pair circ? lst? num? sym? flg? sp? pat? fun? box? str? ext? bool not == n== = <> =0 =1 =T n0 nT < <= > >= match full
Arithmetics
+ - * / % */ ** inc dec >> rev lt0 le0 ge0 gt0 abs bit? & | x| sqrt seed hash rand max min length size accu format pad money round bin oct hex hax
List Processing
car cdr caar cadr cdar cddr caaar caadr cadar caddr cdaar cdadr cddar cdddr caaaar caaadr caadar caaddr cadaar cadadr caddar cadddr cdaaar cdaadr cdadar cdaddr cddaar cddadr cdddar cddddr nth con cons conc circ rot list need range full make made chain link yoke copy mix append delete delq replace insert remove place strip split reverse flip trim clip head tail stem fin last member memq mmeq sect diff index offset prior assoc rassoc asoq rasoq flood rank sort uniq group length size bytes val set xchg push push1 push1q pop ++ shift cut queue fifo idx balance depth get fill apply
Control Flow
load args next arg rest pass quote as lit eval run macro curry def de dm recur recurse undef box new type isa method meth send try super extra with bind job let let? use and or nand nor xor bool not nil t prog prog1 prog2 if if2 ifn when unless cond nond case casq state while until loop do at for catch throw finally co yield ! e $ call ipid opid kill quit task fork detach pipe later timeout tasks abort bye
Mapping
apply pass fun maps map mapc maplist mapcar mapcon mapcan filter extract seek find pick fully cnt sum maxi mini fish by
Input/Output
path in out err ctl input output fd tty ipid opid pipe any sym str load hear tell key poll peek char skip eol eof from till line format scl read print println printsp prin prinl msg space beep tab flush rewind ext rd pr wr wait sync echo info file dir open close port listen accept host connect udp script once finish rc acquire release tmp pretty pp show view here prEval mail
Object Orientation
*Class class dm rel var var: new type isa method meth send try object extend super extra with This can dep
Database
pool pool2 journal id blk seq lieu lock commit rollback mark free dbck tree genKey genStrKey useKey +relation +Any +Bag +Bool +Number +Date +Time +Symbol +String +Link +Joint +Blob +Hook +Hook2 +index +Key +Ref +Ref2 +Idx +Sn +Fold +IdxFold +Aux +UB +Dep +List +Need +Mis +Alt +Swap +Entity blob dbSync new! set! put! inc! blob! upd dbs db: db aux collect search forall rel request request! obj create root fetch store count leaf minKey maxKey init step scan iter ubIter prune zapTree chkTree db/3 db/4 db/5 val/3 lst/3 map/3 isa/2 same/3 bool/3 range/3 head/3 fold/3 part/3 tolr/3 select/3 remote/2 revolve/2
Pilog
prove -> unify be clause repeat asserta assertz retract rules goal fail pilog solve query ? repeat/0 fail/0 true/0 not/1 call/1 or/2 nil/1 equal/2 different/2 append/3 member/2 delete/3 permute/2 uniq/2 asserta/1 assertz/1 retract/1 clause/2 show/1 for/2 for/3 for/4 db/3 db/4 db/5 val/3 lst/3 map/3 isa/2 same/3 bool/3 range/3 head/3 fold/3 part/3 tolr/3 select/3 remote/2
Debugging
pretty pp show loc *Dbg help docs doc more less what who can dep debug -debug d unbug u v vi trace -trace untrace traceAll proc hd bench bt lint lintAll select
System Functions
cmd argv opt version gc tty raw alarm sigio kids protect heap stack adr byte env trail up sys date time usec stamp dat$ $dat datSym datStr strDat expDat day week ultimo tim$ $tim telStr expTel locale allowed allow pwd cd chdir ctty info dir dirname basename errno native %@ struct lisp exec call kill quit task fork pipe timeout mail assert test bye
Globals
NIL pico *CPU *OS *DB T *Solo *PPid *Pid @ @@ @@@ This *Prompt *Dbg *Zap *Scl *Rule *Class *Dbs *Run *Hup *Sig1 *Sig2 *TStp1 *TStp2 *Term ^ *Err *Msg *Uni *Adr *Allow *Fork *Bye


Download

The PicoLisp system can be downloaded from the PicoLisp Download page.