Tag Archives: python

Django Themes (or where to put base.html?)

The Wrong Way

I used to create a new directory to hold common templates like “base.html”, and add it TEMPLATES_DIR in the settings.py file:

TEMPLATE_DIRS = (os.path.join(BASE_DIR, 'templates') ,)

But in most cases the “base.html” would need to use CSS, JS, and image files to be functional, so I changed the url routing to access them (from the DEBUG mode only), something like:

$ vi apps/urls.py
:
# Serve Static Files 
from django.conf import settings
from django.conf.urls.static import static
if settings.DEBUG:
    urlpatterns += static(settings.MEDIA_URL, document_root=settings.MEDIA_ROOT)
    urlpatterns += static(settings.STATIC_URL, document_root=settings.STATIC_ROOT)
    urlpatterns += patterns('django.views.static',
        url(r'^(?P<path>(js|css|img)/.*)$', 'serve', {'document_root':  settings.BASE_DIR+'/../www'}),
        )

This setup isn’t ideal for many reasons:

  • I had to modify settings.py and urls.py with complicated settings.
  • Theme design span multiple directories, and it isn’t self contained.
  • Switching the design is complicated, and include many changes.

Django Simple Themes

Nowadays I create a new Django application e.g. “my_theme” to hold my “base.html” template and all needed static files (CSS, JS, Images, etc …).

./manage.py startapp my_theme

Then add it to INSTALLED_APPS:

INSTALLED_APPS = (
    # django core apps ...
    'my_theme',
    # other apps ...
)

The directory structure for my new app looks like this:

my_theme/
    templates/
        base.html
    static/
        my_theme/
            css/
            js/
            img/

and from my “base.html” (or any other template) I could access the static file using the static tag:

{% load staticfiles %}
<img src="{% static "my_theme/img/logo.png" %}" />

I don’t even need to change the “urls.py” file to access the static file, since the development server (i.e. ./manage.py runserver) already knows how to find them.

But for production I have to define:

STATIC_ROOT = os.path.join(BASE_DIR, '../www/static')

and run:

./manage.py collectstatic --noinput

New Theme

By having all theme files inside an application I can start new theme by copying “my_theme” to something like “new_theme” and replace it in the INSTALLED_APPS in the settings.py.

What about uploaded files?

To access uploaded file from development server you need to define both MEDIA_URL and MEDIA_ROOT and change your “urls.py”:

$ vi apps/urls.py
:
from django.conf import settings
from django.conf.urls.static import static
if settings.DEBUG:
    urlpatterns += static(settings.MEDIA_URL, document_root=settings.MEDIA_ROOT)

Source: Serving files uploaded by a user during development

Sample Theme

You can download my sample theme from:
https://github.com/rayed/django_theme

Django returning JSON for AJAX requests

In your views.py you can have a page that return JSON data for AJAX request like this:

import json
from django.http import HttpResponse

def ajax(request):
    data = {}
    data['something'] = 'useful'
    return HttpResponse(json.dumps(data), content_type = "application/json")

This would work fine if you fill data your self, but if you getting a data from a model try the following:

from django.core import serializers
def tasks_json(request):
    tasks = Task.objects.all()
    data = serializers.serialize("json", tasks)
    return HttpResponse(data, content_type='application/json')

Setting Up Python and Supervisor on CentOS

CentOS default repository is very limited, and even if you install EPEL you will get old packages, in my case I needed to install Supervisor to manage my Django application, after trying to do it manually and through EPEL I ended up with the following setup.

Install Needed Package

sudo yum install python-setuptools
sudo easy_install pip
sudo pip install supervisor

Setup Supervisor

We’ve already installed “Supervisor” globally, but we need to create its configuration, luckily it comes with default config:

echo_supervisord_conf > supervisord.conf
sudo cp supervisord.conf /etc/supervisord.conf
sudo mkdir /etc/supervisord.d/
sudo vi /etc/supervisord.conf
:
[include]
files = /etc/supervisord.d/*.conf
:

Next we need to set “Supervisor” to run automatically every time you restart your machine, we need to create /etc/rc.d/init.d/supervisord with the following content:

sudo vi /etc/rc.d/init.d/supervisord
#!/bin/sh
#
# /etc/rc.d/init.d/supervisord
#
# Supervisor is a client/server system that
# allows its users to monitor and control a
# number of processes on UNIX-like operating
# systems.
#
# chkconfig: - 64 36
# description: Supervisor Server
# processname: supervisord

# Source init functions
. /etc/rc.d/init.d/functions

prog="supervisord"

prefix="/usr/"
exec_prefix="${prefix}"
prog_bin="${exec_prefix}/bin/supervisord"
PIDFILE="/var/run/$prog.pid"

start()
{
       echo -n $"Starting $prog: "
       daemon $prog_bin --pidfile $PIDFILE
       [ -f $PIDFILE ] && success $"$prog startup" || failure $"$prog startup"
       echo
}

stop()
{
       echo -n $"Shutting down $prog: "
       [ -f $PIDFILE ] && killproc $prog || success $"$prog shutdown"
       echo
}

case "$1" in

 start)
   start
 ;;

 stop)
   stop
 ;;

 status)
       status $prog
 ;;

 restart)
   stop
   start
 ;;

 *)
   echo "Usage: $0 {start|stop|restart|status}"
 ;;

esac

Then make sure CentOS knows about it:

sudo chmod +x /etc/rc.d/init.d/supervisord
sudo chkconfig --add supervisord
sudo chkconfig supervisord on
sudo service supervisord start

Sample Supervisor App

Here is a sample of Django App to be controlled and monitored by Supervisor, just put it:

sudo vi /etc/supervisord.d/my_django_cms.conf
[program:my_django_cms]
directory=/var/www/dashboard/www
command=/home/rayed/.virtualenvs/dev/bin/gunicorn apps.wsgi:application -b 127.0.0.1:8080 --workers 8 --max-requests 1000
# UNIX Socket version (better with Nginx)
#command=/home/rayed/.virtualenvs/dev/bin/gunicorn apps.wsgi:application -b unix:/tmp/my_django_cms.sock --workers 8  --max-requests 1000
environment=DJANGO_ENV="prod"
user=rayed
autostart=true
autorestart=true
redirect_stderr=True

After that:

 sudo supervisorctl add my_django_cms
 sudo supervisorctl start my_django_cms

Setting Apache to Proxy to Gunicorn

To add virtual host to Apache that forward dynamic content to Gunicorn:

sudo vi /etc/httpd/conf.d/my_django_cms.conf
NameVirtualHost *:80
<VirtualHost *:80>
    ServerName dj.example.com
    ServerAdmin webmaster@example.com
    DocumentRoot /var/www/my_django_cms/www
    RewriteEngine on
    ProxyPreserveHost On
    RewriteCond $1 !^/(favicon\.ico|robots\.txt|media|static)/
    RewriteRule ^(.*) http://localhost:8000$1 [P]
    <Proxy *>
            Order deny,allow
            Allow from all
            Allow from localhost
    </Proxy>
</VirtualHost>

If you have SELinux enabled you might need to apply the following command:

setsebool -P httpd_can_network_connect 1

If you have issues accessing your statics files from Apache it might be SELinux also:

restorecon -Rv /var/www/my_django_cms/

Django Multiple Settings with Single File

Instead of having multiple settings files one for production and one for development and so on, I prefer to use an “if” statement with environment variable:

ENV = os.environ.get('DJANGO_ENV', '')
print "==== Active Env: [%s]  ====" % (ENV)    
if ENV == "dev":
    # DEBUG
    DEBUG = True
    TEMPLATE_DEBUG = DEBUG
else:
    DEBUG = False
    TEMPLATE_DEBUG = False

Then from your “.bashrc” file:

export DJANGO_ENV=dev

Django “render” vs “render_to_response”

Summary: Always use render and not render_to_response

In Django you have more than one way to return a response, but many times I get confused between render and render_to_response, render_to_response seems shorter, so why not use it!

To explain let’s assume simple posts page:

def article_list(request, template_name='article/list.html'):
    posts = Post.objects.all()
    # DON'T USE
    return render_to_response(template_name, {'posts': posts})

In this example you will be able to access “posts” in your template, but unfortunately you will not have access to other important variables from other Middlewares, most importantly: user, csrf_token, and messages. To make “render_to_response” pass all these parameters you must send a “context_instance” like this:

return render_to_response(template_name, {'posts': posts}, context_instance=RequestContext(request))

Not so short after all, compare to “render” version:

# USE THIS :)
return render(request, template_name, {'posts': posts})

In fact “render” is always shorter than “render_to_response”, even without the “context_instance”:

return render(request, template_name, {'posts': posts})
... vs ...
return render_to_response(template_name, {'posts': posts})

Install Python Image Library (PIL) on OSX

PIL or Python Imaging Library is a library that allows you to manipulate images in Python programming language, trying to install “PIL” using “pip” tool won’t work perfectly so here is how to proper installation.

Install Brew

Brew is a package management system that simplifies the installation of software on Mac OS X, you can install it easily with the following command:

ruby -e "$(curl -fsSL https://raw.github.com/mxcl/homebrew/go)"

Install Pil

Actually we will install ”pillow” a fork of ”pil”:

brew install samueljohn/python/pillow

Configure your Python Path

To make PIL available to Python we add its path to PYTHONPATH variable:

vi ~/.bash_profile
PYTHONPATH=/usr/local/lib/python2.7/site-packages:$PYTHONPATH
export PYTHONPATH

Check it

To test if you have PIL installed:

python -c "import PIL.Image"

You shouldn’t get any errors.

My New Project: AgentX Implementation in Python

Update: I changed the module name to ”pyagentx”, thank you Daniel for the suggestion.

During this Eid vacation I spent many hours working on AgentX implementation in Python.

You can find the project in GitHub:

https://github.com/rayed/pyagentx

What is AgentX?

AgentX is protocol to extend SNMP agents, defined in RFC 2741.

But what is SNMP? let’s say you have a Linux machine you want to monitor, you will use Simple Network Management Protocol or SNMP for short, you install an SNMP agent on the machine like Net-SNMP, and from your management station you connect to the SNMP agent and ask it for the data you want report on, for example the current state of network link.

What happen if you install new software, like PostgreSQL DB, and your SNMP agent doesn’t support it, how can you monitor it! The good news SNMP agents (e.g. Net-SNMP) can be extend with multiple options, the bad news most options are very hard.

One of the most flexible options is AgentX protocol, you will need to build an application that run AgentX protocol (AgentX SubAgent), upon startup it will contact the SNMP agent (AgentX master) and register a part of the MIB tree that your app will handle, the SNMP agent (AgentX master) will forward all queries to your app which will return the result back to the master which will forward it to the management station (Cacti, NMS).

Net-SNMP already have an API to build AgentX Sub Agent, and there are Python module that utilise it, but unfortunately it doesn’t look active, and as far as I know Net-SNMP API it self isn’t the easiest thing to work with.

This is why I started working with Pure Python implementation for AgentX protocol, it is already in a working condition and tested with Net-SNMP agent, I’ve some more issues to resolve before I can use it in production.

So please if you are interested in the field give my module a try, and let me know how can improve it, and also suggest better name.