Using Nginx as your main web server for multiple Trac Projects

This recipe below describes some setups where ​Nginx is your main webserver. Nginx is an Apache replacement by ​Igor Sysoev. First set up Nginx as your main web server and then set up multiple instances of Tracd Web Server. Trac has an embedded webserver that is included in Trac and called Tracd. Then you have Nginx serve requests to your Tracd web server instances.

Why you should use Tracd behind Nginx

This is a setup where Nginx acts as a load balancer. Tracd is lightweight and fast and it is easy to get working with Trac. You can start multiple instances of the Tracd web server on different ports for different Trac projects. Nginx as your main webserver can serve requests to these instances of the Tracd web server. This also works for multiple Trac Projects on multiple vhosts. In short it's fast, lightweight, and easy to set up.

Here are the steps you need to take:

1. Set up Nginx as your main webserver on port 80.
2. Start multiple instances of the Tracd embedded web server on different ports for each Trac project.
3. Configure Nginx to serve requests to your various running instances of Tracd webserver.

Using Tracd with Nginx in Cluster Mode

When using Trac and Apache with multiple vhosts, and multiple Trac sites per vhost, and upgrading from Subversion 1.2.3 to 1.4, we hit this bug:

• apache/mod_python occasionally segfaults.
• apache/mod_python was causing strange occasional delays, likely related to the segfaults.

Caveat: Only use this with PostgreSQL. If you want to do this, but are on SQLite, then use Pacopablos ​Sqlite-to-Pg script. We use it here, and it's great.

Tracd - Trac's light and fast embedded web server

Run multiple tracd instances. This offers a speed benefit if you use ​FasterFox, as well as good concurrency responsiveness.

Using the Gentoo init system, it was easy to create simple init scripts, which are attached to this page. Here is a simplified example:

Multi-Site tracd Startup: works with Trac-0.10 and up:

tracd -d -p 3050 --pidfile=/var/live/run/tracd.3050 --protocol=http -e /var/live/trac

Single-Site tracd Startup: will not work with Trac before version 0.10:

tracd -d -p 3050 --pidfile=/var/live/run/tracd.3051 --protocol=http -s /var/live/trac/telecardia

Nginx

Install Nginx. All examples are based on Gentoo and the Gentoo package is under /etc/nginx.

Sample /etc/nginx/nginx.conf:

http {
  include         /etc/nginx/mime.types;
  default_type    application/octet-stream;

  include         /etc/nginx/nginx-defaults.conf;

  upstream live_trachosts_com {
          server  127.0.0.1:3050;
          server  127.0.0.1:3051;
          #[... up to the number of instance, or more, if you want to be free to add more ...]
  }
  
  server {
          listen          192.168.1.254:80;
          server_name     live.trachosts.com live;
  
          access_log      /var/log/nginx/live.access.log main;
          error_log       /var/log/nginx/live.error_log info;
  
          location / {
                  proxy_pass      http://live_trachosts_com;
                  include         /etc/nginx/proxy.conf;
                  # if your system doesn't have the proxy.conf file, add the following two lines to get redirects working:
                  # proxy_redirect on;
                  # proxy_set_header Host $host;
          }
  
  }

Nginx + SSL

Here is what we do for SSL in /etc/nginx/nginx.conf:

http {
  include         /etc/nginx/mime.types;
  default_type    application/octet-stream;

  include         /etc/nginx/nginx-defaults.conf;

  upstream live_trachosts_com {
          server  127.0.0.1:3050;
          server  127.0.0.1:3051;
          #[... up to the number of instance, or more, if you want to be free to add more ...]
  }
  
  server {
          listen          192.168.1.254:80;
          server_name     live.trachosts.com live;
  
          access_log      /var/log/nginx/live.access.log main;
          error_log       /var/log/nginx/live.error_log info;
  
          location / {
                  rewrite         ^/(.*)$ https://imrlive.com/$1 redirect;
          }
  
  }
  server {
          listen          192.168.1.254:443;
          server_name     live.trachosts.com live;
  
          access_log      /var/log/nginx/live.access.log main;
          error_log       /var/log/nginx/live.error_log info;
  
          ssl                  on;
          ssl_certificate      /etc/nginx/ssl/_nginx.cert;
          ssl_certificate_key  /etc/nginx/ssl/traclive.key;
          keepalive_timeout    70;
          add_header           Front-End-Https    on;

          location / {
                  proxy_pass      http://live_trachosts_com;
                  include         /etc/nginx/proxy.conf;
                  # my system doesn't have the proxy.conf file so I needed to add the following two lines to get redirects working:
                  # proxy_redirect on;
                  # proxy_set_header Host $host;
          }
  
  }
}

Static Content

Serving static files from htdocs dir ala /<site>/chrome/site aliases http://live.trachosts.com/myproj/chrome/site to /var/trachosts/trac/myproj/htdocs:

        location ~ /(.*?)/chrome/site/ {
                rewrite /(.*?)/chrome/site/(.*) /$1/htdocs/$2 break;
                root    /var/trachosts/trac;
        }

Subversion

This section can be skipped, if you're using tracd and start it as follows:

/usr/bin/python /usr/bin/tracd -d -p 3050 --basic-auth projec1,/var/www/trac/project1/db/users.htdigest,svn --pidfile=/var/www/trac/tracd.3050 --protocol=http -e /var/www/trac

We still need to get access to Subversion via Apache mod_dav_svn. I created a vhost in Apache for _only_ the svn URLs. Other people might not use this setup.

Listen 127.0.0.1:80
<VirtualHost *:80>
    ServerAdmin webmaster@trachosts.com

    # in order to support COPY and MOVE, etc - over https (443),
    # ServerName _must_ be the same as the nginx servername
    ServerName live.trachosts.com 
    UseCanonicalName on

    DocumentRoot "/var/www/live.trachosts.com/htdocs"

    <Directory "/var/www/live.trachosts.com/htdocs">
        Options FollowSymLinks
        AllowOverride None
        Order allow,deny
        Allow from all
    </Directory>

  # Subversion Configuration
  <IfModule mod_dav_svn.c>
      <Location /svn>
          DAV svn 
          SVNParentPath /var/live/svn
          AuthType Basic
          AuthName "Client Trac Sites - Subversion Repository"
          AuthUserFile /var/live/conf/users
           <IfModule !mod_authz_svn.c>
              AuthzSVNAccessFile /var/live/conf/svnauthz
          </IfModule>
          Require valid-user
      </Location>
  </IfModule>
</VirtualHost>

Add this to the server section of the Nginx config, in the :80 line, or the :443:

location /svn {
        proxy_pass      http://127.0.0.1:80;
        include         /etc/nginx/proxy.conf;
        set  $dest  $http_destination;
        if ($http_destination ~ "^https://(.+)") {
           set  $dest   http://$1;
        }
        proxy_set_header  Destination   $dest;
}

Script Examples

start-meta-site.sh:

INSTANCES="3050 3051 3052 3053 3054 3055 3056"
USER="apache"
VERSION="0.10.1"
#ENV="/var/live/trac"
PIDFILE="/var/live/run/tracd"
### Extra Args here, for instance --basic-auth
#ARGS="--basic-auth=${ENV},${ENV}/vccbob.pwd,FarQ"
ARGS="-e /var/live/trac"
PYTHON_EGG_CACHE="/var/live/egg_cache"

function start(){
    export PYTHON_EGG_CACHE
    for I in $INSTANCES; do
        su ${USER} -c "/usr/bin/tracd -d -p ${I} --pidfile=${PIDFILE}.${I} --protocol=http ${ARGS} ${ENV}"
        done
}

function stop(){
   for x in `ls ${PIDFILE}.*}`; do
       kill `cat ${x}`
       done
}

$1

start-single-site.sh

INSTANCES="3050 3051 3052"
USER="apache"
VERSION="0.10.1"
ENV="/var/lib/trac-single"
PIDFILE="/var/lib/trac-single/run"
### Extra Args here, for instance --basic-auth
#ARGS="--basic-auth=${ENV},${ENV}/vccbob.pwd,FarQ"
ARGS="--basic-auth=${ENV},${ENV}/vccbob.pwd,FarQ -s ${ENV}"
PYTHON_EGG_CACHE="${ENV}/egg_cache"

function start(){
    export PYTHON_EGG_CACHE
    for I in $INSTANCES; do
        su ${USER} -c "/usr/bin/tracd -d -p ${I} --pidfile=${PIDFILE}.${I} --protocol=http ${ARGS}"
        done
}

function stop(){
   for x in `ls ${PIDFILE}.*`; do
       kill `cat ${x}`
       done
}

$1

FreeBSD

Startup rc.d script for FreeBSD that allows you to run StandAlone server for multiple Trac Projects behind Nginx with ngx_http_proxy_module (proxy_pass) module. This script is better than the default FreeBSD rc.d script, because it supports multiple Trac Projects. Also it supports authentication for each project separately with Trac basic authentication.

/usr/local/etc/rc.d/trac:

#!/bin/sh
#
# Trac (http://trac.edgewall.org/) startup rc.d
# 
# (c) 2012 Stanislav Rudenko, sandel{at}ukr.net
#  Thanks to Alexey Tsvetnov, vorakl{at}fbsd.kiev.ua 
#   last update 11.03.12

# PROVIDE: tracd
# REQUIRE: LOGIN
# KEYWORD: shutdown

# 
# Add the following line to /etc/rc.conf to enable trac: 
#  trac_enable="YES"
#
# and optional: 
#  trac_data="/usr/local/www/trac" 
#  trac_config="/usr/local/etc/trac.conf"
#  trac_user="trac" 
#  trac_group="trac" 
#  trac_port="8000" 
#  trac_host="127.0.0.1" 
#  trac_pidfile="/var/run/trac/trac.pid"
#
# You _MUST_ create config file --> /usr/local/etc/trac.conf
# and add the following lines:
#
#  DIR_PROJECT_1="/usr/local/www/trac/myproj"
#  NAMEDIR_PROJECT_1="myproj"
#  PATH_PASSFILE_1="/usr/local/www/trac/.htpasswd_myproj"
#  NAME_REALM_1="My Project"
#
# (also you can increment variables to use multiply projects)
# (for example DIR_PROJECT_2,...,NAME_REALM_2 etc...)
#

. "/etc/rc.subr"

name="trac"
rcvar=`set_rcvar`
load_rc_config $name

# Set some defaults
trac_enable=${trac_enable:-"NO"}
trac_data=${trac_data:-"/usr/local/www/trac"}
trac_config=${trac_config:-"/usr/local/etc/trac.conf"}
trac_user=${trac_user:-"trac"}
trac_group=${trac_group:-"trac"}
trac_port=${trac_port:-"8000"}
trac_host=${trac_host:-"127.0.0.1"}
trac_pidfile=${trac_pidfile:-"/var/run/trac/trac.pid"}

start_cmd="trac_start"
start_precmd="trac_precmd"

required_files="/usr/local/bin/sudo"

pidfile=${trac_pidfile}

command="/usr/local/bin/tracd"
command_interpreter="/usr/local/bin/python2.5"

trac_precmd()
{
        if [ ! -s "${trac_config}" ]
         then
          trac_configexample
          return 1
        fi

        . "${trac_config}"


        piddir=$(/usr/bin/dirname "${trac_pidfile}")
        if [ ! -d ${piddir} ]
         then
          /bin/mkdir -p "${piddir}"
          /usr/sbin/chown ${trac_user}:${trac_group} "${piddir}"
        fi

        /usr/local/bin/sudo -u ${trac_user} /bin/sh -c "if [ -d \"${piddir}\" -a -w \"${piddir}\" ];then return 0;else return 1;fi"
        if [ $? -ne 0 ]
         then
         echo "Can't write .pid file to \"${piddir}\"!!! Change directory permissions!"
         return 1
        fi

        if [ -s "${trac_pidfile}" ]
         then
          /bin/ps -axp "$(/bin/cat ${trac_pidfile})" >/dev/null 2>&1
          if [ $? -eq 0 ]
           then
            echo "trac already running! (check ${trac_pidfile})"
            return 1
          fi
          /bin/rm -f "${trac_pidfile}"
         else
          trpid=$(/usr/bin/pgrep "/usr/local/bin/tracd")
          if [ -n "${trpid}" ]
           then
            echo "trac(${trpid}) already running, but can't find PID file!!!"
            return 1
          fi
        fi


        i=0
        while :
         do
          i=$(( $i + 1 ))
          [ -z "$(eval echo "\${DIR_PROJECT_${i}}")" ] && break

          NAMEDIR_PROJECT=$(eval echo "\${NAMEDIR_PROJECT_${i}}")
          PATH_PASSFILE=$(eval echo "\${PATH_PASSFILE_${i}}")
          NAME_REALM=$(eval echo "\${NAME_REALM_${i}}")
          DIR_PROJECT=$(eval echo "\${DIR_PROJECT_${i}}")

          basicauth="${basicauth} --basic-auth=\"${NAMEDIR_PROJECT},${PATH_PASSFILE},${NAME_REALM}\""
          projects="${projects} \"${DIR_PROJECT}\""
        done

        rc_flags="-d -p ${trac_port} -b ${trac_host} -e ${trac_data} --pidfile=${pidfile}${basicauth}${projects}"
}

trac_start()
{
        /bin/echo -n "Starting ${name}"

        eval /usr/local/bin/sudo -u ${trac_user} env PYTHON_EGG_CACHE=${trac_data}/.python-eggs ${command} ${rc_flags}

        /bin/echo "."
}

trac_configexample()
{
        echo ""
        echo "=========================================================="
        echo "You _MUST_ create config at /usr/local/etc/trac.conf"
        echo "and add the following lines:"
        echo ""
        echo "DIR_PROJECT_1=\"/usr/local/www/trac/myproj\""
        echo "NAMEDIR_PROJECT_1=\"myproj\""
        echo "PATH_PASSFILE_1=\"/usr/local/www/trac/.htpasswd_myproj\""
        echo "NAME_REALM_1=\"My Project\""
        echo "=========================================================="
        echo ""
}

run_rc_command "$1"

Handling authentication in Nginx

If you want to handle the authentication in Nginx rather than through Trac, that is also possible. Since you are proxying the tracd server from Nginx, you just have to tell Nginx to forward the authorization header to tracd, and use the same authentication scheme in both (Basic / Digest). Also, both Nginx and Trac must access the same password file, or an identical copy. As a simple example, let's assume you are using Basic authentication. Digest would be very similar.

This is the Nginx configuration snippet:

server {
        location / {
                proxy_pass http://localhost:8000;  # Replace localhost:8000 with your server:port
                auth_basic "Restricted";
                auth_basic_user_file htpasswd;     # Will effectively be /etc/nginx/htpasswd in Ubuntu, check your distribution
                proxy_pass_header Authorization;   # Here you tell Nginx to forward the Authorization header to tracd
        }
}

And then, you can start tracd with the following command if you use multi-project setup (notice the *):

tracd --port=8000 --hostname=127.0.0.1 --env-parent-dir=/home/trac --basic-auth="*,/etc/nginx/htpasswd,Restricted"

Or the following command if you run one tracd per project:

tracd --port=8000 --single-env /path/to/trac/environments/project --basic-auth="project,/etc/nginx/htpasswd,Restricted"

You can adjust those commands to your specific needs (daemonize, etc).

Todo

• Post the actual config files somewhere.

Questions

• is this possible with client certificate authentication?

---

See also TracFastCgi#SimpleNginxConfiguration1

          if [ $? -eq 0 ]
           then
            echo "trac already running! (check ${trac_pidfile})"
            return 1
          fi
          /bin/rm -f "${trac_pidfile}"
         else
          trpid=$(/usr/bin/pgrep "/usr/local/bin/tracd")
          if [ -n "${trpid}" ]
           then
            echo "trac(${trpid}) already running, but can't find PID file!!!"
            return 1
          fi
        fi


        i=0
        while :
         do
          i=$(( $i + 1 ))
          [ -z "$(eval echo "\${DIR_PROJECT_${i}}")" ]