How to Configure Apache to Talk to Lisp

There are several webservers implemented entirely in Lisp. You may be excited about the prospect of using them. But, if you've tried using them for part of a large site, you may have realized that Apache does certain things better, and it would be nice if you could somehow use Lisp for only part of the site. Or, you may be advocating that Lisp be used for a part of your corporate website, but the company understandably does not want to redo the entire site to use a Lisp server. Perhaps it's just easier to sell people on the idea of Lisp if it fits in with other things instead of trying to replace everything.

One solution is to run two webservers: the ordinary Apache one, and the special Lisp one. Anytime Apache receives a request that should be handled by Lisp, it opens a connection to the Lisp webserver so that it can be done. Here, I show you how to create exactly this setup, using the simple rule that any file ending in the extension .lisp will be handled by Lisp.

You will still need to configure the Lisp server to know where the directory tree is, and what code to run. How that's done depends on which server it is. It will see the request as being for exactly the same URL that Apache saw. The Lisp server will only receive requests for .lisp files, so those are all it needs to handle. It can also receive options after the filename, as in:

/advent/foo.lisp?search=fee+fie+foe+foo&btnOK=true

If you are implementing your own Lisp server, rather than using one of the existing ones, you may find it slightly easier to use mod_lisp, instead of the approach I describe here. Or not: the only work that saves you is parsing the http headers, which is trivial.

The paths and filenames I use here are correct for Mac OS X; if you're using some other system, you'll have to figure out where Apache is installed and what the correct paths are. Happily, if you're using some other system you probably know how to do that. I hope so, anyway, because I can't advise you.

Backup the Config File

Open a new shell. On the Mac, that's Terminal.app. Type the commands:

bash$ cd /etc/httpd/
bash$ sudo cp httpd.conf httpd.conf.my.backup.2005.january

That filename is just a suggestion, of course. The reason for putting the date in it is so that if you accidentally leave it there and find it two years later, you'll know it's not a recent backup and that you can safely delete it. Well, that always happens to me...

Editing the Config File

I edit my config files using Emacs; in recent versions, you can edit a file as root by using the command C-x C-f /sudo:root@localhost:/etc/httpd/httpd.conf. If you're comfortable in Vi, you can use it from the terminal: sudo vi /etc/httpd/httpd.conf. Otherwise, you're on your own; I don't know how people edit the config files on a Mac if they don't know how to use Emacs or Vi.

Edit One: Make Sure mod_proxy and mod_rewrite are Loaded

Skim through the file until you find where all the LoadModule commands are. Look for the line that starts with LoadModule rewrite_module and make sure it's uncommented (doesn't have a # at the start of it). It probably already is, definitely if you're using a Mac. If it's commented, delete the # to uncomment it.

Now, in that same part of the file, look for a line that starts with LoadModule proxy_module. This one probably is commented out, because proxies are a security risk unless you know what you're doing. Uncomment it.

Continue scrolling down and you will find lines starting with AddModule mod_rewrite.c and AddModule mod_proxy.c. Make sure these are both uncommented, as well.

Edit Two: Configure mod_proxy

Search the file for the line <IfModule mod_proxy.c>. Again, it's probably commented out because you probably didn't have proxying turned on before. Uncomment it, and also uncomment the line </IfModule>, which will be shortly after.

Do not uncomment the line ProxyRequests On. That would be a major security risk, and is not necessary for what we're doing.

Add the following line, anywhere between the IfModule tags:

    ProxyPassReverse / http://localhost:8080/

If you can't find the mod_proxy.c section anywhere in the file, you'll have to add one. Put it near the end of the file, after all the other IfModule lines:

<IfModule mod_proxy.c>
    ProxyPassReverse / http://localhost:8080/
</IfModule>

What this does is make sure that if the Lisp webserver sends any redirect messages which refer to itself, they will be modified before they're sent to the client, so that the redirect will still be going through Apache.

Edit Three: Configure mod_rewrite

Now search for the <IfModule mod_rewrite.c> section. At the very end of it, add the line:

    RewriteRule ^(.*\.lisp(\?.*)?)$ http://localhost:8080$0 [P]

This is if the Lisp server is running on port 8080; if it's running on a different port, you can change that. You can also change the hostname if it's running remotely, on a different machine.

If there is no mod_rewrite.c section, you'll have to add one:

<IfModule mod_rewrite.c>
    RewriteEngine On
    RewriteRule ^(.*\.lisp(\?.*)?)$ http://localhost:8080$0 [P]
</IfModule>

What this does is create a rule saying that any filename ending in .lisp should be transparently proxied over to the Lisp server. Apache will open a connection to the Lisp server, and send the request, acting as if it were a client. When the Lisp server sends the response back to Apache, Apache will send the response on to the original client. The original client never needs to know or care that Lisp was involved.

Edit Four: Configure mod_dir

Search for the <IfModule mod_dir.c> section. It should look something like this:

<IfModule mod_dir.c>
    DirectoryIndex index.html index.cgi index.pl
</IfModule>

At the end of the DirectoryIndex line, add index.lisp.

What this does is inform Apache that if a directory contains an index.lisp file, it should use the Lisp proxy for that directory, instead of producing a directory listing itself.

Reload the Config File

Save the file and close it. Now you just need to tell Apache to read your changes. In the terminal, type the command:

bash$ sudo kill -1 `cat /var/run/httpd.pid`

You will need to give your password. What this command does is send a HUP signal to the already-running copy of httpd, telling it that you've changed httpd.conf and would it please put the changes into effect.

Usage and Sunshine

Anytime a user tries to load a file such as http://you.com/somewhere/script.lisp, the Lisp server will receive a request, such as GET /somewhere/script.lisp. For maximum flexibility, it does this regardless of whether or not the file script.lisp and the directory /somewhere/ actually exist, so it's up to the Lisp server to generate a 404 response if it so chooses.

If a user tries to load a directory index, such as http://you.com/elsewhere/, and the file index.lisp exists in the directory elsewhere/, then Lisp will receive the request GET /elsewhere/index.lisp. For this to work, the file does have to actually exist, although it may be empty.

There is a peculiarity here which you do need to be alert for. Because Lisp receives the unedited URL, not the local path, if you want the Lisp webserver to read and execute that file, you need to take whatever steps the Lisp server you use requires to ensure that it can find the file. This probably means setting up a directory mapping telling it that the web path /somewhere/ maps to the directory /Users/you/Sites/somewhere/ on your machine, or whatever's appropriate.

Another perfectly acceptable way of doing things, instead of setting up Lisp to find the same file as Apache sees, is to have the Lisp server use its own local tree, entirely different. Or, perhaps your server doesn't even use real files for its pages at all, and they exist only as in-memory data structures. Apache doesn't need to be able to find the files Lisp uses, although it does need to be able to find an index.lisp file when you're using one.

Now you have Apache set up to communicate with a Lisp webserver. You can do whatever fantastic things you want in Lisp, and you'll never need to mess around with confusing Apache config files again! Isn't that great?

Future Directions

Apache has some rather nice capabilities, such as authenticated access, SSL, and cache control. It would be nice if there were a way to take advantage of these when running a Lisp server. Probably some of them "just work" with no further effort, and some of them shouldn't be too difficult to set up. Experiment is required. Please get in touch if you get anything along these lines working, or would like help in doing so.