Zero-downtime web server restarts with Flying Passenger Enterprise
Flying Passenger allows one to decouple Passenger's life time from the web server's life time, so that the web server can be independently restarted from Passenger, and from any of the application processes served by Passenger.
Table of contents
Normally, Passenger starts together with the web server, and shuts down together with the web server. The advantages of this default behavior is that it makes Passenger easy to administer: one only has to deal with the web server process and can expect all relevant processes to be cleaned up after a web server shut down. However this also brings about a disadvantage: every time one restarts the web server (e.g. to make a minor configuration change), Passenger and all its application processes also get restarted.
This problem is solved by 'Flying Passenger', which is an advanced mode of operation in Passenger that allows the web server to be independently restarted from Passenger. When this mode is enabled:
- One must start Passenger separately from the web server, namely by starting the Flying Passenger daemon. This daemon must - to an extent - be separately configured and managed from the web server.
- The web server must be configured to forward requests to the Flying Passenger daemon.
- You should beware of the caveats and limitations.
The Flying Passenger mode is off by default. You have to explicitly enable it.
There is also a distinct problem, namely that of restarting web applications without downtime after you have deployed application updates. That problem is solved by the rolling restarts feature, not by Flying Passenger. To achieve full zero-downtime, you will want to enable both rolling restarts and Flying Passenger.
- The Flying Passenger feature is only available in Passenger Enterprise. Buy Passenger Enterprise here.
- Passenger version 4.0.45 or higher installed on Apache.
Start the Flying Passenger daemon by invoking the
flying-passenger command. The only required option is
--socket-file. Depending on whether you wish to enable user account sandboxing, you have to start
flying-passenger with root privileges or not.
$ sudo flying-passenger --socket-file=/var/run/flying-passenger.sock I, [2013-06-14T09:10:13.095339 #77179] INFO -- : Welcome to Flying Passenger 5.0.11 I, [2013-06-14T09:10:13.095339 #77179] INFO -- : Starting PassengerAgent... I, [2013-06-14T09:10:13.097036 #77179] INFO -- : PassengerAgent started on PID 77181 ... I, [2013-06-14T09:10:13.129017 #77179] INFO -- : PassengerAgent initialized properly I, [2013-06-14T09:10:13.129127 #77179] INFO -- : Flying Passenger up and listening on /var/run/flying-passenger.sock!
Now configure the web server to make use of the Flying Passenger daemon by setting the
passenger_fly_with option to the socket filename:
After (re)starting the web server, Flying Passenger is fully operational.
You can test it by adding a virtual host for a web app:
<VirtualHost *:80> ServerName www.foo.local DocumentRoot /webapps/foo/public </VirtualHost>
Verify that it works by making an HTTP request to it:
$ curl http://www.foo.local/
Now let's verify that restarting the web server does not restart the just-spawned application process. Run
passenger-status to obtain the PID of the application process:
$ sudo passenger-status Version : 5.0.11 Date : 2015-06-24 12:47:13 +0200 Instance: XE6bilnV (Apache/2.4.12 Phusion_Passenger/5.0.11) ----------- General information ----------- Max pool size : 6 App groups : 1 Processes : 1 Requests in top-level queue : 0 ----------- Application groups ----------- /webapps/foo/public: App root: /webapps/foo Requests in queue: 0 * PID: 77283 Sessions: 0 Processed: 49991 Uptime: 6h 22m 3s CPU: 2% Memory : 110M Last used: 0s ago
As you can see, the PID of the application process is 77283. Now let's see what happens if we restart the web server:
$ sudo service apache2 restart $ sudo passenger-status
The application process should remain there, unchanged:
$ sudo passenger-status Version : 5.0.11 Date : 2015-06-24 12:47:45 +0200 Instance: XE6bilnV (Apache/2.4.12 Phusion_Passenger/5.0.11) ----------- General information ----------- Max pool size : 6 App groups : 1 Processes : 1 Requests in top-level queue : 0 ----------- Application groups ----------- /webapps/foo/public: App root: /webapps/foo Requests in queue: 0 * PID: 77283 Sessions: 0 Processed: 49992 Uptime: 6h 22m 3s CPU: 2% Memory : 110M Last used: 0s ago
Configuring Flying Passenger
Flying Passenger gets some configuration from Apache, but not all. In particular, most web server directives that are only valid in the global context, e.g. PassengerLogLevel, have no effect when using Flying Passenger. Instead, you are supposed to pass these configuration directives through command line options to the Flying Passenger daemon. Configuration directives that have no effect on Flying Passenger are documented as such. You can assume that configuration directives that are not documented as such, work fine on Flying Passenger.
For example, to achieve the same effect as setting
to 2, run the Flying Passenger daemon as follows:
$ sudo flying-passenger --socket-file=/var/run/flying-passenger.sock --log-level=2
Currently, not all configuration directives have a Flying Passenger equivalent. Run the following command to see an overview of available options:
$ flying-passenger --help
Managing the Flying Passenger daemon
The Flying Passenger daemon runs in the foreground by default. This is undesirable on server environments. You can make it go into the background by passing
$ sudo flying-passenger --socket-file=/var/run/flying-passenger.sock \ --daemonize --log-file=/var/log/flying-passenger.log \ --pid-file=/var/run/flying-passenger.pid
You can shut down a Flying Passenger daemon by sending SIGINT or SIGTERM to it:
$ kill `cat /var/run/flying-passenger.pid`
We recommend using daemontools or runit for managing the Flying Passenger daemon. These tools will allow automatically starting the Flying Passenger daemon at boot, and will automatically restart the daemon if it crashes. You can create and enable a daemontools/runit service as follows:
$ sudo mkdir /etc/service/flying-passenger $ sudo nano /etc/service/flying-passenger/run #!/bin/sh exec /path-to/flying-passenger \ --socket-file=/var/run/flying-passenger.sock \ --log-file=/var/log/flying-passenger.log \ --pid-file=/var/run/flying-passenger.pid
Immediately after creating the
run file, daemontools/runit automatically runs it to start the daemon. Note that the location (
/etc/service) depends on the OS or Linux distros. Sometimes it's
/service. Also note that we start the Flying Passenger daemon without
To shut down a daemontools/runit-managed daemon, you need to use
svc -d /etc/service/flying-passenger (daemontools) or
sv stop /etc/service/flying-passenger (runit) instead of sending a signal to the process.
Using Flying Passenger with MRI 1.8 or JRuby
Using Flying Passenger in combination with MRI Ruby 1.8 or with JRuby requires special attention. This is because the Flying Passenger daemon is written in Ruby, and requires proper
Process.spawn support, which neither MRI 1.8 nor JRuby support.
It is however possible to use Flying Passenger with MRI Ruby 1.8 and JRuby. You can't run the Flying Passenger daemon in MRI 1.8 or JRuby, but you can still run the web applications – hosted under Flying Passenger – in MRI 1.8 or JRuby.
First, edit your web server configuration file and specify a Ruby interpreter for your web applications. For example:
# Connect to the Flying Passenger daemon on the following socket PassengerFlyWith unix:/var/run/flying-passenger.sock ... <VirtualHost *:80> ServerName www.foo.com DocumentRoot /webapps/foo/public # Use JRuby for this web application PassengerRuby /opt/jruby/bin/jruby </VirtualHost>
Then you need to install a Ruby 1.9-compatible Ruby interpreter with POSIX spawn support, alongside JRuby/MRI 1.8. Ruby interpreters which can be used for running the Flying Passenger daemon include:
- MRI Ruby >= 1.9.
The following example demonstrates how you can install MRI Ruby 1.9 in parallel with your MRI Ruby 1.8 or JRuby installation.
Example for Debian/Ubuntu users:
# Install Ruby 1.9 sudo apt-get install -y ruby1.9.3 # Run the Flying Passenger daemon in Ruby 1.9 ruby1.9 -S flying-passenger --socket-file=/var/run/flying-passenger.sock
Example for RVM users:
# Install Ruby 1.9 rvm install 1.9.3 # Run the Flying Passenger daemon in Ruby 1.9 rvm-exec 1.9.3 ruby -S flying-passenger --socket-file=/var/run/flying-passenger.sock
The Flying Passenger daemon will now be run on Ruby 1.9, while the web application
www.foo.com will be run on JRuby.
Caveats and limitations
Beware of the following caveats and limitations when using Flying Passenger.
The Flying Passenger daemon is written in Ruby. It requires a Ruby interpreter with proper
Process#spawn support. At the time of writing, all Ruby interpreters in existance satisfy this requirement, except for MRI Ruby 1.8 and JRuby. See Using Flying Passenger with MRI 1.8 or JRuby for more information.
When you add a new application to the web server configuration, Flying Passenger will automatically pick up the application's settings and spawn this new application upon the first request to it. However it is not capable of automatically starting the new app before a request has been sent to it (i.e. PassengerPreStart-like behavior is not available in this case). As a workaround, you can send an HTTP request to your application after starting the daemon, which forces it to spawn application processes.
The Passenger module that is loaded into Apache, must be of the same version as the Flying Passenger daemon. Failing to meet this requirement may result in cryptic errors, or may result in certain features not working, until you've fixed the situation. When upgrading Passenger, you must restart both Apache and the Flying Passenger daemon.
The PassengerRoot directive has no effect. When using Flying Passenger, you are not supposed to set
When you remove an application from the web server configuration, Flying Passenger will not detect the removal and will not shut down the associated application processes. Killing the application processes will also not help, because Flying Passenger will restart them per the (now-removed, but still in the Flying Passenger daemon's memory) PassengerMinInstances setting. At the moment, there are two ways to get rid of those processes:
- Before removing the application from the web server configuration, explicitly set its
PassengerMinInstancesto 0. Next, send a request to it, which will cause the Flying Passenger daemon to take over the new
PassengerMinInstances 0option. You can then proceed with removing the application from the web server configuration, and restarting the web server. Finally, kill the PIDs associated to those application processes and remove the application configuration.
- Restart the Flying Passenger daemon.