Toy Lisp Interpreter

Here is a toy lisp interpreter, written in c. A novel feature of the interpreter is that it includes a function for creating new special forms.


Files

lisp.c

Toy lisp interpreter with static scoping, written in c. If there is a file called "predefined.scm" in the directory that contains the interpreter executable, the interpreter loads and interprets this file.

lisp-dynamic.c

Toy lisp interpreter with dynamic scoping, written in c.

predefined.scm

Predefined functions and special forms.

lisp.scm

Toy lisp interpreter, written in lisp. The interpreter is capable of interpreting itself.


Read-eval-write loop

The lisp interpreter consists of three main subroutines: read, eval, and write.

Read subroutine

The input to the lisp interpreter takes the form of strings called symbolic expressions (s-expressions), which are generated by the following grammar:
<s-expression> --> ' <s-expression>
               --> <list>
               --> integer
               --> symbol

<list> --> ( <series> )
       --> ( <series> . <s-expression> )

<series> --> <s-expression> <series>
         -->
The read subroutine reads in an s-expression and converts it into a lisp object consisting of CONS cells, INTS, and SYMS.

Eval subroutine

The eval subroutine evaluates the input lisp object and produces an output lisp object. Lisp objects are evaluated according to the following rules:
  • INTS, PRIMS, COMPS, and SPECS evaluate to themselves.
  • SYMs evaluate to the lisp objects to which they are bound.
  • CONS cells (lists) are evaluated as follows. First, the car of the list is evaluated. If the result is a function (a PRIM or a COMP), then the remaining elements of the list are evaluated and the function is applied to them. If the result is a special form (a SPEC), then the function corresponding to the special form is applied to a two-element list whose first element contains the cdr of the original list and whose second element is the calling environment.

For example, consider a function f that takes two arguments. The s-expression "(f x y)" is evaluated by evaluating x and y to obtain arguments X and Y, and then applying the function named by f to these arguments to obtain the result.

In general, I will use lowercase letters to denote the elements of the list and uppercase letters to denote the results of evaluating these elements. For example, suppose we define symbols three and four:

--> (define three 3)
three
--> (define four 4)
four
In the expression "(+ three four)", x is "three", y is "four", X is "3", and Y is "4".

Write subroutine

The write subroutine converts the output lisp object into a string, which it displays on the screen. Note that although the input lisp object consists only of CONS cells, INTS, and SYMS, the output lisp object may consist of lisp objects of any type, including PRIMS, COMPS, and SPECS. When the output lisp object is converted to a string, PRIMS, COMPS, and SPECS are converted to "[primitive function]", "[compound function]", and "[special form]", respectively. For example:
--> +
[primitive function]
--> (lambda x x)
[compound function]
--> quote
[special form]
--> (cons + quote)
([primitive function] . [special form])


Types of lisp objects

There are six types of lisp objects:
Type CONS

A CONS cell is an ordered pair of lisp objects. The first lisp object is called the car of the CONS cell, and the second lisp object is called the cdr of the CONS cell. Note that the first or second lisp object may itself be a CONS cell, so lisp objects can be viewed as binary trees in which the non-terminal nodes are CONS cells and the terminal nodes are atoms (an atom is anything that is not a CONS cell).

Type INT

An INT is an integer.

Type SYM

A SYM is a symbol (also called a variable), and can be bound to another lisp object. A SYM can be thought of as naming the s-expression to which it is bound.

Type PRIM

A PRIM is a primitive function; that is, a function that is predefined by the lisp interpreter.

Type COMP

A COMP is a compound function; that is, a function that is built out of primitive functions and other compound functions.

Type SPEC

A SPEC is a special form; unlike a function, it does not evaluate all of its arguments.


Lists and atoms

Lisp objects are divided into two classes: atoms and lists. An atom is anything that is not a CONS cell. A list is either a CONS cell or the empty list, which is a SYM. Thus, the empty list is both an atom and a list, and is the only lisp object to have this property.

Lists are subdivided into two classes: proper lists and improper lists. A list is said to be proper if it the empty list or if the cdr of the list is a proper list; otherwise the list is said to be improper.

Examples of proper lists: "(1 2 3)", "((1 2) 3)", "((1 . 2) 3)"

Examples of improper lists: "(1 2 . 3)"


Predefined symbols

There are a number of SYMS that are predefined by the LISP interpreter; these include the symbols "()", "#t", and "#f", the names of the primitive functions, and the names of the predefined special forms. The symbols "()", "#t", and "#f" are bound to "()", "#t", and "()", respectively, the names of the primitive functions are bound to their respective functions, and the names of the predefined special forms are bound to their respective special forms.


Static versus dynamic scoping

For static scoping, a function is evaluated by extending the environment in which the function's lambda expression was defined. For dynamic scoping, a function is evaluated by extending the environment in which the function is called.

Here are some examples that illustrate the difference between static and dynamic scoping. For static scoping,

--> (define bletch 1)
bletch
--> (define add-bletch (lambda (x) (+ x bletch)))
add-bletch
--> ((lambda (bletch) (add-bletch 3)) 5)
4
Here is the same example with dynamic scoping:
--> (define bletch 1)
bletch
--> (define add-bletch (lambda (x) (+ x bletch)))
add-bletch
--> ((lambda (bletch) (add-bletch 3)) 5)
8
A second example with static scoping:
--> (define bletch 1)
bletch
--> ((lambda (bletch) (define add-bletch (lambda (x) (+ x bletch)))) 7)
add-bletch
--> ((lambda (bletch) (add-bletch 3)) 5)
10
Here is the second example with dynamic scoping:
--> (define bletch 1)
bletch
--> ((lambda (bletch) (define add-bletch (lambda (x) (+ x bletch)))) 7)
add-bletch
--> ((lambda (bletch) (add-bletch 3)) 5)
8


Function for creating new special forms

A novel feature of the lisp interpreter is that it includes a function called "special" for creating new special forms. The s-expression "(special f)" returns a special form, where F is required to be a function of two arguments. A list whose first element evaluates to this special form is evaluated by applying F to a two-element list whose first element is the cdr of the list and whose second element is the calling environment. For example:
--> ((special (lambda x x)) a b c)
((a b c) ())
--> (let ((bletch 42)) ((special (lambda x x)) a b c))
((a b c) ((bletch . 42)))
To illustrate this function, I will show how it can be used to define some common special forms.

We can use "special" to define the "quote" special form as follows:

--> (define quote-new (special (lambda (sexp env) (car sexp))))
quote-new
--> (quote-new a)
a

We can use "special" to define the "cond" special form as follows:

--> (define eval-cond
  (lambda (sexp env)
    (if (null? sexp) ()
        (if (eval (first (first sexp)) env)
            (eval (second (first sexp)) env)
            (eval-cond (cdr sexp) env)))))
eval-cond
--> (define cond (special eval-cond))
cond
--> (cond (#f 1) (#t 2))
2


Primitive functions

The following primitive functions are predefined by the lisp interpreter (lowercase letters denote the unevaluated arguments; uppercase letters denote the results of evaluating these arguments):
(+ x1 ... xn) : X1, ..., Xn must be of type INT

Return the sum of X1, X2, ..., Xn. Examples:
--> (+ 7 5)
12
--> (+ 1 2 3)
6
--> (+ 3)
3

(- x1 ... xn) : X1, ..., Xn must be of type INT

Return X1 minus the sum of X2, ..., Xn. Examples:
--> (- 7 5)
2
--> (- 1 2 3)
-4
--> (- 3)
3

(* x1 ... xn) : X1, ..., Xn must be of type INT

Return the product of X1, ..., Xn. Examples:
--> (* 7 5)
35
--> (* 2 3 4)
24
--> (* 3)
3

(= x y) : X and Y must be of type INT

If X and Y are equal then return #t, otherwise return #f. Examples:
--> (= 42 42)
#t
--> (= 0 42)
()

(car list) : LIST must be of type CONS

Return the first element of LIST. Examples:
--> (car '(1 2 3))
1
--> (car '(a . b))
a

(cdr list) : LIST must be of type CONS

Return the list that is obtained by removing the first element from LIST. Examples:
--> (cdr '(1 2 3))
(2 3)
--> (cdr '(a . b))
b
--> (cdr '(1))
()

(cons x y)

Return a CONS cell whose first element is X and whose second element is Y. Examples:
--> (cons 1 2)
(1 . 2)
--> (cons 'a '(1 2 3))
(a 1 2 3)

(eval x env) : ENV must be an association list

Evaluate X in environment ENV and return the result. Examples:
--> (eval 'bletch '((bletch . 42)))
42
--> (eval '(lambda quote quote) (list (cons 'lambda *) (cons 'quote 3)))
9

(apply f list) : F must be a function, LIST must be a list of arguments for F

Apply F to LIST and return the result. Example:
--> (apply + '(1 2 3))
6

(assoc key alist) : KEY must be an atom, ALIST must be an association list

Compare KEY with each dotted pair in alist in sequence, starting with the first. If KEY matches the first element of a dotted pair then return that dotted pair, otherwise return the empty list. Examples:
--> (assoc 'b '((a . 1) (b . 2) (c . 3)))
(b . 2)
--> (assoc 'z '((a . 1) (b . 2) (c . 3)))
()
--> (assoc 'a '((a . 1) (a . 2) (a . 3)))
(a . 1)
(eqv? x y)

If X and Y are the same lisp object then return #t, otherwise return #f. Examples:
--> (eqv? 'a 'a)
#t
--> (eqv? 12 (+ 7 5))
#t
--> (eqv? '(1 2) '(1 2))
()
(type-of x)

Return the type code of X, where CONS cells, INTS, SYMS, PRIMS, COMPS, and SPECS are assigned type codes 0, 1, 2, 3, 4, and 5, respectively. Examples:
--> (type-of '(1 2 3))
0
--> (type-of 42)
1
--> (type-of 'bletch)
2
--> (type-of ())
2
--> (type-of #t)
2
--> (type-of #f)
2
--> (type-of +)
3
--> (type-of (lambda (x y) (* x y)))
4
--> (type-of quote)
5
--> (type-of (special cons))
5

(special f): F must be a function that takes two arguments

Return a special form. The special form is evaluated by applying F to a two-element list whose first element is a list of the unevaluated arguments and whose second element is the calling environment. Examples:
--> ((special (lambda x x)) (+ 1 2) cons)
(((+ 1 2) cons) ())
--> ((lambda (a b) ((special (lambda x x)) bletch foo)) 1 2)
((bletch foo) ((a . 1) (b . 2)))


Predefined special forms

The following special forms are predefined by the lisp interpreter (lowercase letters denote the unevaluated arguments; uppercase letters denote the results of evaluating these arguments):
(quote x)

Return x. Examples:
--> '(1 2 3)
(1 2 3)
--> (quote (1 2 3))
(1 2 3)
--> 'bletch
bletch

(lambda args x) : args must be either a SYM or a list of SYMs

Return a function. Examples:
--> ((lambda (x y) (* x y)) 3 4)
12
--> ((lambda x x) 1 2 3)
(1 2 3)

(if x y z)

If X is true then return Y, otherwise return Z.

(define name x) : name must be a SYM

Bind name to X, return name.


Commands

The following commands are defined by the lisp interpreter:
:q
Quit the program.

:e
Print out the current environment.

:t
Toggle trace mode; this shows the individual steps taken by the lisp interpreter in evaluating lambda expressions.

:m
Print the total number of objects that have been generated.

:n
Print out the list of SYMS.


Recursive local functions

It requires some trickery to define recursive local functions. The following example illustrates one way to define such functions:
(let ((fact-helper (lambda (f n) (if (= n 0) 1 (* n (f f (- n 1)))))))
  (let ((fact (lambda (n) (fact-helper fact-helper n))))
    (fact 5)))
Here "fact", the factorial function, is defined as a recursive local function. Here is another way to do this:
((lambda x (apply (car x) x))
 (lambda (f n) (if (= n 0) 1 (* n (f f (- n 1))))) 5)

John McCarthy defined a special form called "label" that makes it much simpler to define recursive local functions. Here is how "label" can be used to define "fact" as a recursive local function:

((label fact (lambda (n) (if (= n 0) 1 (* n (fact (- n 1)))))) 5)
The "label" special form is defined in
predefined.scm.


Diagonalization

Consider the problem of finding a s-expression called "fixed-point" such that "fixed-point" and "(eval-top fixed-point)" evaluate to the same thing This can be viewed as the lisp equivalent of finding a program that prints itself out.

The problem can be trivially solved by simply binding "fixed-point" to itself:

--> (define fixed-point 'fixed-point)
fixed-point
--> (eval-top fixed-point)
fixed-point
Suppose, however, that we rule out this type of solution. The problem can be solved using a very general method that makes use of diagonalization.

Define a symbol "diag" as follows:

--> (define diag '(lambda (x) (list x (list (quote quote) x))))
diag
Note that "diag" has the following property:
(eval-top ((eval-top diag) f)) = ((eval-top f) f),
where "f" is an arbitrary s-expression. For example,
--> (eval-top ((eval-top diag) 'list))
(list)
--> (list 'list)
(list)
Also,
--> (eval-top ((eval-top diag) '(lambda (x) (cons x x))))
((lambda (x) (cons x x)) lambda (x) (cons x x))
--> ((lambda (x) (cons x x)) '(lambda (x) (cons x x)))
((lambda (x) (cons x x)) lambda (x) (cons x x))
Now suppose we take "f" to be "diag". Then
(eval-top ((eval-top diag) diag)) = ((eval-top diag) diag).
Thus, "((eval-top diag) diag)" has the desired property:
--> (define fixed-point ((eval-top diag) diag))
fixed-point
--> fixed-point
((lambda (x) (list x (list (quote quote) x))) (quote (lambda (x) (list x (list (quote quote) x)))))
--> (eval-top fixed-point)
((lambda (x) (list x (list (quote quote) x))) (quote (lambda (x) (list x (list (quote quote) x)))))


David Boozer

Last modified 25 January 2009
boozer at caltech dot edu