Setting Up Python and Supervisor on CentOS

CentOS default repository is very limited, and even if you install EPEL you will get old packages, in my case I needed to install Supervisor to manage my Django application, after trying to do it manually and through EPEL I ended up with the following setup.

Install Needed Package

sudo yum install python-setuptools
sudo easy_install pip
sudo pip install supervisor

Setup Supervisor

We’ve already installed “Supervisor” globally, but we need to create its configuration, luckily it comes with default config:

echo_supervisord_conf > supervisord.conf
sudo cp supervisord.conf /etc/supervisord.conf
sudo mkdir /etc/supervisord.d/
sudo vi /etc/supervisord.conf
:
[include]
files = /etc/supervisord.d/*.conf
:

Next we need to set “Supervisor” to run automatically every time you restart your machine, we need to create /etc/rc.d/init.d/supervisord with the following content:

sudo vi /etc/rc.d/init.d/supervisord
#!/bin/sh
#
# /etc/rc.d/init.d/supervisord
#
# Supervisor is a client/server system that
# allows its users to monitor and control a
# number of processes on UNIX-like operating
# systems.
#
# chkconfig: - 64 36
# description: Supervisor Server
# processname: supervisord

# Source init functions
. /etc/rc.d/init.d/functions

prog="supervisord"

prefix="/usr/"
exec_prefix="${prefix}"
prog_bin="${exec_prefix}/bin/supervisord"
PIDFILE="/var/run/$prog.pid"

start()
{
       echo -n $"Starting $prog: "
       daemon $prog_bin --pidfile $PIDFILE
       [ -f $PIDFILE ] && success $"$prog startup" || failure $"$prog startup"
       echo
}

stop()
{
       echo -n $"Shutting down $prog: "
       [ -f $PIDFILE ] && killproc $prog || success $"$prog shutdown"
       echo
}

case "$1" in

 start)
   start
 ;;

 stop)
   stop
 ;;

 status)
       status $prog
 ;;

 restart)
   stop
   start
 ;;

 *)
   echo "Usage: $0 {start|stop|restart|status}"
 ;;

esac

Then make sure CentOS knows about it:

sudo chmod +x /etc/rc.d/init.d/supervisord
sudo chkconfig --add supervisord
sudo chkconfig supervisord on
sudo service supervisord start

Sample Supervisor App

Here is a sample of Django App to be controlled and monitored by Supervisor, just put it:

sudo vi /etc/supervisord.d/my_django_cms.conf
[program:my_django_cms]
directory=/var/www/dashboard/www
command=/home/rayed/.virtualenvs/dev/bin/gunicorn apps.wsgi:application -b 127.0.0.1:8080 --workers 8 --max-requests 1000
# UNIX Socket version (better with Nginx)
#command=/home/rayed/.virtualenvs/dev/bin/gunicorn apps.wsgi:application -b unix:/tmp/my_django_cms.sock --workers 8  --max-requests 1000
environment=DJANGO_ENV="prod"
user=rayed
autostart=true
autorestart=true
redirect_stderr=True

After that:

 sudo supervisorctl add my_django_cms
 sudo supervisorctl start my_django_cms

Setting Apache to Proxy to Gunicorn

To add virtual host to Apache that forward dynamic content to Gunicorn:

sudo vi /etc/httpd/conf.d/my_django_cms.conf
NameVirtualHost *:80
<VirtualHost *:80>
    ServerName dj.example.com
    ServerAdmin webmaster@example.com
    DocumentRoot /var/www/my_django_cms/www
    RewriteEngine on
    ProxyPreserveHost On
    RewriteCond $1 !^/(favicon\.ico|robots\.txt|media|static)/
    RewriteRule ^(.*) http://localhost:8000$1 [P]
    <Proxy *>
            Order deny,allow
            Allow from all
            Allow from localhost
    </Proxy>
</VirtualHost>

If you have SELinux enabled you might need to apply the following command:

setsebool -P httpd_can_network_connect 1

If you have issues accessing your statics files from Apache it might be SELinux also:

restorecon -Rv /var/www/my_django_cms/

25 thoughts on “Setting Up Python and Supervisor on CentOS”

  1. Fantastic instructions, thanks a lot. I have Centos 6 and could not find any easier instructions that this. I’ve posted a link on the Laravel forum.

  2. Great instructions! I needed to setup supervisor for zookeeper, this got it done in under 20 minutes. One question though:

    In first section, “Install needed packages”, is the line source /usr/bin/virtualenvwrapper.sh supposed to be inside ~/.bashrc or is it to be executed only once after editing? It’s not clear here.

    This might be the cause for the following remark: I needed to source ~/.bashrc before I could start the service (without reboot) – otherwise it would fail, not finding it’s config file.

  3. Thank you Dirk, I edited the post to address your comment, and note that you can skip the whole virtualenvwrapper if you only need Supervisord.

  4. Thanks for the instruction. What do I need to change in supervisord or gunicorn if I will be using nginx instead of apache?

  5. You are welcome Francis, you don’t need to change anything from Gunicorn & Supervisor side, just configure Nginx to proxy to Gunicorn, something like this:


    upstream app_server {
    server 127.0.0.1:8000 fail_timeout=0;
    }

    server {
    listen 80 ;
    server_name www.example.com;

    # Static content
    location ~ /(media|static)/ {
    root /var/www/;
    expires 30d;
    }

    location / {
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header Host $http_host;
    proxy_redirect off;
    proxy_pass http://app_server;
    }
    }

  6. Hi!
    I have pkg_resources.DistributionNotFound: meld3>=0.6.5 after running echo_supervisord_conf > supervisord.conf. Please, append sudo pip install --upgrade Distribute command into instruction. Thank you.

  7. @Tmtm Thank you for your feeback.
    Many errors can happen in many places, I try to keep minimal setup and people can trouble shoot specific problems they may face.

  8. Hi Rayed,
    Many thanks for this great article. Works well indeed.
    Supervisord is great !
    Rgds Harry

  9. Do you know how I can run a custom config file from your startup script? Usually I can just do supervisord -c /path/to/config/supervisord.conf. Any help is greatly appreciated. Thanks!

  10. @Haris
    Try changing the line:
    daemon $prog_bin --pidfile $PIDFILE
    with:
    daemon $prog_bin --pidfile $PIDFILE -c /path/to/config/supervisord.conf

  11. Thanks @Rayed. That did not work, but I was able to just create a sym link with my supervisord.conf file and default conf file path.

    Another question. I don’t know if this is due to supervisor or your script, but I usually start like 10 supervisor workers and I notice over time some of those workers slow down. Like the next day I will look and it’s only 7 running. And then I have to run service supervisord stop, and then service supervisord start to get all 10 to fire up again. Do you know what could be the problem there?

    Much thanks for all your help!

  12. Thanks Rayed! I did do both of the things you mentioned already but I will look into this issue at supervisor support channels since it doesn’t seem to be related to your script. Thanks again for all your help!

  13. In the /etc/rc.d/init.d/supervisord file, I think the start() function should say [ ! -f $PIDFILE ] instead of [ -f $PIDFILE ]

  14. Thank you @efu for your comment, but “supervisor” should start with the previous line:
    daemon $prog_bin --pidfile $PIDFILE

    The line you mentioned will show success message only if the PIDFILE exists, or it will show failure message, it won’t actually start it:
    [ -f $PIDFILE ] && success $"$prog startup" || failure $"$prog startup"

  15. Dear Rayed.,

    First of all a very nice blog for supervisord setup.,

    I have installed supervisord through yum on RHEL 7 with epel repository and configured rqworker and rqscheduler for django cms project under /etc/supervisord.d/ both the services were working absolutely fine but when i restarted supervisor and rqworker, rqscheduler services to apply the changes done to Django. both rqworker and rqscheduler stopped successfuly but while starting it is throwing FATAL Error exiting too quickly.

    ************************************************************
    [root]# service supervisord status
    Redirecting to /bin/systemctl status supervisord.service
    ● supervisord.service – Process Monitoring and Control Daemon
    Loaded: loaded (/usr/lib/systemd/system/supervisord.service; enabled; vendor preset: disabled)
    Active: active (running) since Thu 2016-04-21 20:15:10 AST; 6s ago
    Process: 6463 ExecStart=/usr/bin/supervisord -c /etc/supervisord.conf (code=exited, status=0/SUCCESS)
    Main PID: 6466 (supervisord)
    CGroup: /system.slice/supervisord.service
    └─6466 /usr/bin/python /usr/bin/supervisord -c /etc/supervisord.conf
    **************************************************************

    above is the output for supervisord daemon but subprocess are not showing as it use to show and below is the error that i am getting.


    [root]# supervisorctl status
    rqworker FATAL Exited too quickly (process log may have details)
    rqscheduler FATAL Exited too quickly (process log may have details)

    Awaiting for your valuable assistance.

    Thanks in advance.

    Best Regards.,
    Absar.

  16. @Absar I haven’t tried the steps with RHEL 7 nor CentOS 7, I am not sure if systemd would change anything or not.
    But I would recommend checking the logs of the app to see what went wrong, may be under:
    /var/log/supervisor/rqworker-stderr–supervisor.log

  17. Hi Rayed,

    Followed your tutorial and everything went well without issues, but when I try to add a new program using supervisorctl add trs, it says
    ERROR: no such process/group: trs
    Yes, I have trs.conf file in /etc/supervisord.d/
    Any ideas what might cause this?
    Thank you very much.

  18. Hi Rayed,

    I have a trouble, that it cant do

    sudo supervisorctl add my_service
    sudo supervisorctl start my_service

    it says *** Unknown syntax: add my_service

    any ideas how to fix this ?
    Thanks before for your tutorial & help

  19. Hello Thank you for such a great tutorial, do you have an idea why is this sending my laravel emails with our without a dispatch multiple times? sometomes 2, sometimes 5… it is not consistent…

    Regards.

  20. @ Mario Sorry no idea. Try enabling logging to see if there are multiple processes running simultaneously without proper locking!

Leave a Reply

Your email address will not be published. Required fields are marked *