In many VPS environments, it is often the case that you will have a number of small programs that you want to run persistently, whether these be small shell scripts, Node.js apps, or any large-sized packages.
Conventionally, you may write a init script for each of these programs, but this can quickly become time consuming to manage and isn’t always particularly transparent for newer users.
Supervisor is a process manager which makes managing a number of long-running programs a trivial task by providing a consistent interface through which they can be monitored and controlled.
Installation
Installation of Supervisor on both Ubuntu and Debian is incredibly simple, as prebuilt packages already exist within both distributions’ repositories.
As the root user, run the following command to install the Supervisor package:
1 | apt-get install supervisor |
Once this has completed, the supervisor daemon should already be started, as the prebuilt packages come with an init script that will also ensure the Supervisor is restarted after a system reboot. You can ensure this is the case by running:
1 | service supervisor restart |
Now that we have Supervisor installed, we can look at adding our first programs.
Adding a Program
New programs are given to Supervisor through configuration files, which inform it of the executable to run, any environmental variables, and how output should be handled.
Note: All programs run under Supervisor must be run in a non-daemonising mode (sometimes also called ‘foreground mode’). If, by default, the program forks and returns on startup, then you may need to consult the program’s manual to find the option to enable this mode, otherwise Supervisor will not be able to properly determine the status of the program.
For the sake of this article, we’ll assume we have a shell script we wish to keep persistently running that we have saved at /usr/local/bin/long.sh and looks like the following:
1 |
|
Then run it:
1 | chmod +x /usr/local/bin/long.sh |
In a practical sense, this script is clearly rather pointless, but it will allow us to cover the fundamentals of Supervisor configuration.
The program configuration files for Supervisor programs are found in the /etc/supervisor/conf.d directory, normally with one program per file and a .conf extension. A simple configuration for our script, saved at /etc/supervisor/conf.d/long_script.conf, would look like so:
1 | [program:long_script] |
We’ll look at the significance of each line and some of the tweaks that may be desirable for your program below:
1 | [program:long_script] |
The configuration begins by defining a program with the name ‘long_script’ and the full path to the program:
1 | autostart=true |
The next two lines define the basic automatic behaviour of the script under certain conditions.
The autostart option tells Supervisor that this program should be started when the system boots. Setting this to false will require a manual start command following any system shutdown.
autorestart defines how Supervisor should manage the program in the event it exits and has three options:
falsetells Supervisor not to ever restart the program after it exitstruetells Supervisor to always restart the program after it exitsunexpectedtells Supervisor to only restart the program if it exits with an unexpected error code (by default anything other than codes 0 or 2).
1 | stderr_logfile=/var/log/long.err.log |
The final two lines define the locations of the two main log files for the program. As suggested by the option names, stdout and stderr will be directed to the stdout_logfile and stderr_logfile locations respectively. The specified directory specified must exist before we start the program, as Supervisor will not attempt to create any missing directories.
The configuration we have created here is a minimal reasonable template for a Supervisor program. The documentation lists many more optional configuration options that are available to fine tune how the program is executed.
Once our configuration file is created and saved, we can inform Supervisor of our new program through the supervisorctl command. First we tell Supervisor to look for any new or changed program configurations in the /etc/supervisor/conf.d directory with:
1 | supervisorctl reread |
Followed by telling it to enact any changes with:
1 | supervisorctl update |
Any time you make a change to any program configuration file, running the two previous commands will bring the changes into effect.
At this point our program should now be running and we can check this is the case by looking at the output log file:
1 | $ tail /var/log/long.out.log |
Managing Programs
Once our programs are running, there will undoubtedly be a time when we want to stop, restart, or see their status. The supervisorctl program, which we first used above, also has an interactive mode through which we can issue commands to control our programs.
To enter the interactive mode, start supervisorctl with no arguments:
1 | $ supervisorctl |
When started, supervisorctl will initially print the status and uptime of all programs, followed by showing a command prompt. Entering help will reveal all of the available commands that we can use:
1 | supervisor> help |
Using the tail command, we can view the most recent entries in the stdout and stderr logs for our program:
1 | supervisor> tail long_script |
Using status we can view again the current execution state of each program after making any changes:
1 | supervisor> status |
Finally, once we are finished, we can exit supervisorctl with Ctrl-C or by entering quit into the prompt:
1 | supervisor> quit |
And that’s it! You’ve mastered the basics of managing persistent programs through Supervisor and extending this to your own programs should be a relatively simple task.