Working Around JSVC's Logging Limitations

JSVC is a popular option for people using Tomcat as their web container. The main advantage of JSVC is that it allows downgrading the user running a process (since most Linux systems require the root user to open a port below 1024), and also acts as a watchdog to restart the JVM if it crashes. However one big problem with JSVC is that it can only write the output of the JVM it's hosting to two files on the filesystem corresponding to stdout and stderr. This is problematic since it doesn't allow for log rotation or any other form of redirection.

At Kikini, we created a logging solution to append log statements into SimpleDB so that logs from all our machines end up in a central location, unbounded by normal filesystem limits, and easily query-able against and monitored, allowing us to react quickly to diagnose problems. The simplest way to use our logger is to redirect the output from the target process to the stdin of our logging process. However JVSC makes this rather difficult since it is hard-coded to only write to files on the filesystem.

Fortunately we have a trick up our sleeve in the form of UNIX named pipes, which can use as a target for JSVC to write to and a source for the logger to read from:

mkfifo pipe.out
mkfifo pipe.err
/usr/bin/startlogger.sh STDOUT < pipe.out
/usr/bin/startlogger.sh STDERR < pipe.err
/usr/bin/jsvc -outfile pipe.out -errfile pipe.err ...

Now JSVC will start up, and write into the pipes we created, which will be redirected into the mylogger processes.