HomeInternet Applications

mod_pipe

This Apache module provides a mean for communication between Apache server and an external process using Unix pipes.
The idea behind is inspired by Oleg Kiselyov's "Writing agents in sh: conversing through a pipe" and mod_pipe depends on his tool exec_with_piped.

The mod_pipe is designed for the inclusion of Scheme language instructions in the Apache-processed HTML files.
Using mod_pipe Scheme interpreter may be started once and when acts as Scheme servlet engine.
Such an architecture reduces the per-request overhead, as the expensive start-up process is not required.
Thus it possible, for example, to load a bulk of data at the interpreters start and query this data later including the reply in the pre-build HTML pages. The example provided includes in the HTML page the value of the variable scm-interpreter which is bound at the Scheme interpreter initialization time.

This module may be also used for more general purposes, as the mod_pipe.c itself just provides some mechanism for communication and doesn't depend on a process on the other end of the pipes.

Content handler looks for a pair of special tags ("<?" and "?>" by default) and passes the nested string (so-called "request" or "mod_pipe instruction") to the external process (which is usually Scheme interpreter) through a pipe "from_mod_pipe". This external process have to process the request received and to return the result as a string (so called "reply") to a mod_pipe via another pipe, "to_mod_pipe". This reply substitutes Scheme instruction in the processed HTML file.


Scheme interpreter have to be started with its input and output redirected. Example (using Gambit):
exec_with_piped /tmp/from_mod_pipe "gsi -:u >/tmp/to_mod_pipe"

The u option forces all I/O on the standard input and output to be unbuffered. Otherwise you have to include something like (flush-output) into the Scheme code in order to force output from Gambit.
The example above is simplified a bit, because for the practical needs it is necessary to suppress excessive output from the interpreter, such as prompts, unwanted return values, etc. Some interpreter-specific settings are provided for such a purpose.


Installation:

  1. Download the mod_pipe source distribution mod_pipe-src-15.tgz and auxiliary tools mod_pipe-misc-15.tgz.
  2. Unpack the mod_pipe source distribution into the Apache source tree (version 1.3.12 was tested)
  3. Compile and install Apache.
    Where is nothing special here, please refer to its manual for more information, the instructions below are just a shortcut.
    1. Run the configure script from the Apache distribution top-level directory. The installation path may be specified here: 
      ./configure --prefix=/usr/local/apache
    2. cd to the src subdirectory
    3. Modify the Configuration file (add the Module ... directive to the end of file). For example: 
      Module mod_pipe_module modules/mod_pipe/mod_pipe.o
    4. Run ./Configure
    5. Run make. This will build your httpd executable.
    6. Run make install from the Apache distribution top-level directory. (You may have to be a root for this).
    7. cd to /usr/local/apache/conf. Add directives for new module activation into the srm.conf file. For example: 
           <Location /mod_pipe>
       SetHandler pipe-handler
     </Location>
      (Supposing that the subdirectory mod_pipe exists in Apache DocumentRoot directory)
    Important: In case of mod_pipe source file modification just the steps 5,6 and 8 have to be repeated. Do not run configure scripts again!
  4. Create with mknod p command two pipes necessary. The mkpipes script provided may be used as example.
  5. Install Oleg Kiselyov's exec_with_piped tool. Just place it somewhere in your PATH under the name exec_with_piped. It is included in mod_pipe-src-15.tgz as C source and binary for Linux/Intel/libc6. The compilation is straightforward: 
    gcc exec_with_piped.c -o exec_with_piped
  6. Start up the Scheme interpreter using one of *-scmd scripts provided or manually: 
    exec_with_piped /tmp/from_mod_pipe "YOUR_INTERPRETER > /tmp/to_mod_pipe
  7. Start up Apache: Run apachectl start from the newly installed Apache bin directory (/usr/local/apache/bin in the example above)
  8. mod_pipe-misc-15.tgz contains sample configuration file and test page index.html which have to be placed into the DocumentRoot/mod_pipe directory
  9. You may browse the http://hostname:8080/mod_pipe/index.html

Hints/Bugs/Limitations:

  • Both the request and reply buffers are limited to 32K. It may be changed at the compilation time adjusting two constants at the very beginning of the mod_pipe.c file.
  • If the external process needs the CR after the instruction (as the Bigloo and Guile interpreters do) than it is necessary to add PipeAppendNewLine on directive into httpd.conf or insert newline in the HTML file before the closing tag.
  • Both the pipes used have to be created in advance with mknod p command
  • Control-C is not a good way to stop the interpreter started with exec_with_piped, you have to use stopscm script or kill exec_with_piped.
  • The example provided uses two pipes and trace file in the /tmp directory. It doesn't look like a most secure place in case of the practical installation :-)
  • As the current version of mod_pipe ignores HTML comments, it is necessary to replace mod_pipe instruction's bracket with the HTML comment brackets in order to comment out the mod_pipe instruction.
    Scheme comments may be used inside mod_pipe instruction.
  • The mod_pipe is waiting for some reply (at least one byte) after the every instruction sent to the interpreter.

mod_pipe was successfully tested on Linux/Intel using:

  • Bigloo 2.2b
  • GambitC 3.0
  • Guile 1.3.3
  • MzScheme 103
  • SCSH 0.5.2

Interpreter tuning

Settings for the Schemes mentioned above are included in the mod_pipe-misc-15.tgz file.

This file also contains sample configuration files and two simple scripts, mkpipes and stopscm. Please note that the settings provided for some verbose interpreters (Gambit, SCSH and bigloo) intentionally suppress all the messages from the interpreters read-eval-print-loop, and all the output have to be done via side-effects, using 'display', 'write', etc.

It looks like this paradigm is more portable one for mod_pipe applications, as otherwise returned values like 'void', 'unspecified', etc. may be annoying enough.

The same approach may be used with Guile and MzScheme, but as it was no need to silence them for such a purpose their REPL still may print the value returned, if any.

This behavior may be changed, if necessary: MzScheme, Gambit and Guile already have two versions of the init files, with and without REPL messages.

The file mod_pipe/index.html included in the mod_pipe-misc-15.tgz contains the example which will inform you about the method of REPL message handling.

If your Scheme is not one of the mentioned above it is still a good idea to look at the settings provided as they are simple enough and (hopefully) may be used as a starting point for your settings.


License:

mod_pipe is provided under MIT-style Open Source License.


Victor Anikin shares the credit for mod_pipe code and documentation.
HomeInternet Applications