mattias.sun@gmail.com

The 'httpGate' Proxy Server

(c) Software Lab. Mattias Sundblad

This document describes the httpGate utility which is included in the PicoLisp distribution.

For basic information about the PicoLisp system please look at the PicoLisp Reference and the PicoLisp Tutorial.


Purpose

httpGate is a central element of the PicoLisp application server architecture. Its purpose is to perform the following tasks:

Basic functionality

A HTTP request to port 80, respectively 443, of the form


   http[s]://server.org/12345/path/file

is forwarded to a server on localhost listening on port 12345, to ask for the resource "path/file".

If httpGate was started with a config file, and that file contains an entry for "app", then also the following request is accepted:


   http[s]://server.org/app/path/file

In that case, the "app" server process is started automatically (if it is not already running) listening on port 12345, and the request is forwarded as above.

Only GET and POST requests will be forwarded, and only to ports >= 1024. The main httpGate process then forks two child processes, one for each direction. These child processes terminate automatically if the connection is idle for more than 7 minutes.

Building httpGate

httpGate is delivered in source code form with the picoLisp distribution. To build the program you need a c compiler and related header files. Development headers for OpenSSL are also needed. On Debian based systems, the prerequisites can be fulfilled by running apt-get install build-essential and apt-get install openssl-dev.

Next, go to the 'src' directory in the distribution and run make gate. When this is done, there should be an httpGate executable in the 'bin' directory.

Running httpGate

The simplest way to run httpGate is to start it with an explicit port argument:


   bin/httpGate 80 8080
   bin/httpGate 443 8080 pem/www.domain.key,pem/domain.crt

When started in this way, httpGate forwards requests from port 80 and 443 respectively to a PicoLisp application on port 8080. This form has a drawback though, since it only allows for a single application to be handled. Usually, there are many PicoLisp applications running on the same machine, and we need httpGate to forward requests to all of them.

To handle several applications, start httpGate with a "names" config file:


      bin/httpGate 80 names
      bin/httpGate 443 names pem/www.domain.key,pem/domain.crt

httpGate needs to be started as root, but application servers should run under normal user accounts. The easiest way to start httpGate automatically is to add lines like the ones above to '/etc/rc.local'.

Configuring httpGate

The "names" config file

The "names" config file contains one line per application server. Each line holds six whitespace separated tokens, for example:


   app 12345 tom /home/tom log/app ./pil app/main.l lib/app.l -main -go -wait

  1. "app" is the name of the application, and the key to this line.
  2. "12345" is the port where this server should listen at.
  3. "tom" is the user under whose ID the server should run.
  4. "/home/tom" is the working directory where the server should start.
  5. "log/app" is a log file to redirect stdout/stderr to.
  6. The rest of the line ".pil app/main.l ..." is the command to start the application.

Empty lines, and lines starting with a "#", are ignored. If the key in a config file record is the special name "@", then it denotes the default application for this machine. URLs without name will be forwarded to that port. Optional tokens (e.g. log files) or empty arguments to the commands must be written as single caret (^) characters to denote empty strings. Double or single quotes are not parsed.

If the port is zero, then a single additional token is expected which should denote an URL to redirect the request to:


   app 0 https://domain/foo/bar
This will cause httpGate to respnd with "301 Moved Permanently" and "Location: https://domain/foo/bar".

Balanced names file

If the config file contains many (hundreds or thousands) entries, then it is recommended to sort it with the 'balance' utility. This may greatly accelerate name (key) lookup at runtime. For that, put the above config lines into a file "config". The tool 'balance' can be built - together with httpGate - with


   (cd src; make tools gate)

The following command will create a balanced "names" file:


   cat config | bin/balance -sort > names

The "void" file

If the local application server cannot be connected on the requested port (typically because a session timed out), and a file with the name "void" exists in the current working directory (token 4 in the config line), then the contents of that file (normally HTML) are sent as response to the client.

Reloading the configuration

When the config file is modified, it can be reloaded by sending SIGHUP to all running top-level httpGate processes:


   $ sudo pkill -HUP -P1 httpGate

Another possibility is to restart httpGate(s). This is not a problem, and can be done also while the server is in production.

Just kill the top level httpGate parent process. This is not harmful, because existing user sessions are handled by pairs of child processes, which continue to run (until they terminate normally) even if their parent is stopped. Note that this is different from PicoLisp DB applications, where the parent should *never* be hard-stopped (eg. with 'kill -9 <pid>') while child processes are running ('kill <pid>' is OK though, because the parent takes care of stopping the children).

An example for stopping and restarting a running httpGate is:


   (let L
      # Build list of all httpGate parents (i.e. on 80 and 443)
      (make
         (in '("sudo" "pgrep" "-P1" "httpGate")
            (while (read)
               (link @) ) ) )
      # Stop them
      (for P L
         (call "sudo" "kill" P) )
      # Wait until all are gone
      (while (find '((P) (kill P 0)) L)
         (wait 200) )
      # Start new
      (call "sudo" "bin/httpGate" 80 "names")
      (call "sudo" "bin/httpGate" 443 "names" "pem/...") )

Keep-alive and retirement

Applications should call


   (retire 20)

before they call 'server'. This causes the parent server process to terminate automatically 20 minutes after the last child process (user session) terminated. It will be started by httpGate again on demand. User sessions in turn terminate automatically after 5 minutes (if nobody logged in) or 1 hour (if a user is logged in), unless JavaScript is enabled in the client browser and the application calls


   (<ping> 7)

in its main 'action' function. In that case, the user session will not terminate until the user closes the last window or tab to this application.