Django Image and File Field Caveats

Everytime I work with Image or File fields in Django I forget some tiny detail that waste 10-20 minutes until I remember what was I missing, I always say I will remeber it next time but I never do! so I made a list of common errors I keep doing while working with Image/File fields:

For complete working project: https://github.com/rayed/dj-imagefield-example

Setting MEDIA_URL and MEDIA_ROOT

Make sure you set proper values for MEDIA_URL and MEDIA_ROOT in your settings.py, e.g. I use the following structure:

my_new_site/
    apps/
       apps/
          settings.py
       gallery/
       blog/
    www/
       media/
       static/
    requirements.txt


I would put the following settings (for both MEDIA and STATIC files):

STATIC_URL = '/static/'
STATIC_ROOT = os.path.join(BASE_DIR, '..','www','static')
MEDIA_URL = '/media/'
MEDIA_ROOT = os.path.join(BASE_DIR, '..','www','media')

Make sure uploaded files are accessable in development server

Serving uploaded files (media) is the job of the web server and not Django, but to ease the development process I usually make Django serve it in debug mode.

This is done by adding the following at the end of your main urls.py:

from django.conf import settings
from django.conf.urls.static import static
if settings.DEBUG:
    urlpatterns += static(settings.MEDIA_URL, document_root=settings.MEDIA_ROOT)

HTML forms changes

Make sure you have enctype=”multipart/form-data” in your form tag, i.e.:

<form method="post" enctype="multipart/form-data">


Instead of:

<form method="post">

Don’t forget request.FILES

I always forget to include “request.FILES” in my ModelForms, and I always get “This field is required” error message!

def author_create(request, template_name='author/form.html'):
    form = ImageForm(request.POST or None, request.FILES or None)
    if form.is_valid():
        form.save()
        return redirect('author:home')
    return render(request, template_name, {'form':form})    

ImageField requires Imaging library

If you want to use Django built in ImageField you have to install Pillow Imaging library! FileField doesn’t needed it though.

View the image

Just a reminder that you can access the image URL in your template like so:

<img src="{{ author.image.url }}"  />

Install psycopg2 (PostgreSQL adapter for Python) on OSX

I was playing with Django with Postgres backend, and I had little difficulty installing “psycopg2” the Python DB adapter for Postgres on my Mac OSX.

I’ve installed Postgres using Postgres.app for OSX which is straight forward and standard Mac app.

But when I tried installing “psycopg2″ using “pip” (the python package manager) I got an error:

$ pip install psycopg2
:
Error: pg_config executable not found.
:

I just searched for “pg_config” in my system:

$ find / -name pg_config  2>/dev/null
/Applications/Postgres.app/Contents/Versions/9.3/bin/pg_config

Then added to my PATH env and pip worked:

$ export PATH=$PATH:/Applications/Postgres.app/Contents/Versions/9.3/bin/
$ pip install psycopg2

Solving Python virtualenv “DistributionNotFound: distribute”

After upgrading my Ubuntu machine from 12.04 to 14.04 I had this error on virtualenv wrapper:

stevedore.extension distribute
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/stevedore/extension.py", line 75, in _load_plugins
    invoke_kwds,
  File "/usr/local/lib/python2.7/dist-packages/stevedore/extension.py", line 87, in _load_one_plugin
    plugin = ep.load()
  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 2087, in load
    if require: self.require(env, installer)
  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 2100, in require
    working_set.resolve(self.dist.requires(self.extras),env,installer)))
  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 628, in resolve
    raise DistributionNotFound(req)
DistributionNotFound: distribute

After some investigation I found out the cause of the error, it seems that I’ve installed virtualenv-wrapper using pip and not Ubuntu apt-get, so when I installed it using apt-get it conflicted with the pip installation.

Solution

  • Remove virtualenvwrapper Ubuntu package: sudo aptitude remove virtualenvwrapper
  • Remove virtualenvwrapper pip package: sudo pip uninstall virtualenvwrapper virtualenv-clone virtualenv stevedore
  • Reinstall virtualenvwrapper Ubuntu package: sudo aptitude install virtualenvwrapper

Django Themes (or where to put base.html?)

The Wrong Way

I used to create a new directory to hold common templates like “base.html”, and add it TEMPLATES_DIR in the settings.py file:

TEMPLATE_DIRS = (os.path.join(BASE_DIR, 'templates') ,)

But in most cases the “base.html” would need to use CSS, JS, and image files to be functional, so I changed the url routing to access them (from the DEBUG mode only), something like:

$ vi apps/urls.py
:
# Serve Static Files 
from django.conf import settings
from django.conf.urls.static import static
if settings.DEBUG:
    urlpatterns += static(settings.MEDIA_URL, document_root=settings.MEDIA_ROOT)
    urlpatterns += static(settings.STATIC_URL, document_root=settings.STATIC_ROOT)
    urlpatterns += patterns('django.views.static',
        url(r'^(?P<path>(js|css|img)/.*)$', 'serve', {'document_root':  settings.BASE_DIR+'/../www'}),
        )

This setup isn’t ideal for many reasons:

  • I had to modify settings.py and urls.py with complicated settings.
  • Theme design span multiple directories, and it isn’t self contained.
  • Switching the design is complicated, and include many changes.

Django Simple Themes

Nowadays I create a new Django application e.g. “my_theme” to hold my “base.html” template and all needed static files (CSS, JS, Images, etc …).

./manage.py startapp my_theme

Then add it to INSTALLED_APPS:

INSTALLED_APPS = (
    # django core apps ...
    'my_theme',
    # other apps ...
)

The directory structure for my new app looks like this:

my_theme/
    templates/
        base.html
    static/
        my_theme/
            css/
            js/
            img/

and from my “base.html” (or any other template) I could access the static file using the static tag:

{% load staticfiles %}
<img src="{% static "my_theme/img/logo.png" %}" />

I don’t even need to change the “urls.py” file to access the static file, since the development server (i.e. ./manage.py runserver) already knows how to find them.

But for production I have to define:

STATIC_ROOT = os.path.join(BASE_DIR, '../www/static')

and run:

./manage.py collectstatic --noinput

New Theme

By having all theme files inside an application I can start new theme by copying “my_theme” to something like “new_theme” and replace it in the INSTALLED_APPS in the settings.py.

What about uploaded files?

To access uploaded file from development server you need to define both MEDIA_URL and MEDIA_ROOT and change your “urls.py”:

$ vi apps/urls.py
:
from django.conf import settings
from django.conf.urls.static import static
if settings.DEBUG:
    urlpatterns += static(settings.MEDIA_URL, document_root=settings.MEDIA_ROOT)

Source: Serving files uploaded by a user during development

Sample Theme

You can download my sample theme from:
https://github.com/rayed/django_theme

Django returning JSON for AJAX requests

In your views.py you can have a page that return JSON data for AJAX request like this:

import json
from django.http import HttpResponse

def ajax(request):
    data = {}
    data['something'] = 'useful'
    return HttpResponse(json.dumps(data), content_type = "application/json")

This would work fine if you fill the data your self, but if you are getting the data from a model try the following:

from django.core import serializers
def tasks_json(request):
    tasks = Task.objects.all()
    data = serializers.serialize("json", tasks)
    return HttpResponse(data, content_type='application/json')

Setting Up Python and Supervisor on CentOS

CentOS default repository is very limited, and even if you install EPEL you will get old packages, in my case I needed to install Supervisor to manage my Django application, after trying to do it manually and through EPEL I ended up with the following setup.

Install Needed Package

sudo yum install python-setuptools
sudo easy_install pip
sudo pip install supervisor

Setup Supervisor

We’ve already installed “Supervisor” globally, but we need to create its configuration, luckily it comes with default config:

echo_supervisord_conf > supervisord.conf
sudo cp supervisord.conf /etc/supervisord.conf
sudo mkdir /etc/supervisord.d/
sudo vi /etc/supervisord.conf
:
[include]
files = /etc/supervisord.d/*.conf
:

Next we need to set “Supervisor” to run automatically every time you restart your machine, we need to create /etc/rc.d/init.d/supervisord with the following content:

sudo vi /etc/rc.d/init.d/supervisord
#!/bin/sh
#
# /etc/rc.d/init.d/supervisord
#
# Supervisor is a client/server system that
# allows its users to monitor and control a
# number of processes on UNIX-like operating
# systems.
#
# chkconfig: - 64 36
# description: Supervisor Server
# processname: supervisord

# Source init functions
. /etc/rc.d/init.d/functions

prog="supervisord"

prefix="/usr/"
exec_prefix="${prefix}"
prog_bin="${exec_prefix}/bin/supervisord"
PIDFILE="/var/run/$prog.pid"

start()
{
       echo -n $"Starting $prog: "
       daemon $prog_bin --pidfile $PIDFILE
       [ -f $PIDFILE ] && success $"$prog startup" || failure $"$prog startup"
       echo
}

stop()
{
       echo -n $"Shutting down $prog: "
       [ -f $PIDFILE ] && killproc $prog || success $"$prog shutdown"
       echo
}

case "$1" in

 start)
   start
 ;;

 stop)
   stop
 ;;

 status)
       status $prog
 ;;

 restart)
   stop
   start
 ;;

 *)
   echo "Usage: $0 {start|stop|restart|status}"
 ;;

esac

Then make sure CentOS knows about it:

sudo chmod +x /etc/rc.d/init.d/supervisord
sudo chkconfig --add supervisord
sudo chkconfig supervisord on
sudo service supervisord start

Sample Supervisor App

Here is a sample of Django App to be controlled and monitored by Supervisor, just put it:

sudo vi /etc/supervisord.d/my_django_cms.conf
[program:my_django_cms]
directory=/var/www/dashboard/www
command=/home/rayed/.virtualenvs/dev/bin/gunicorn apps.wsgi:application -b 127.0.0.1:8080 --workers 8 --max-requests 1000
# UNIX Socket version (better with Nginx)
#command=/home/rayed/.virtualenvs/dev/bin/gunicorn apps.wsgi:application -b unix:/tmp/my_django_cms.sock --workers 8  --max-requests 1000
environment=DJANGO_ENV="prod"
user=rayed
autostart=true
autorestart=true
redirect_stderr=True

After that:

 sudo supervisorctl add my_django_cms
 sudo supervisorctl start my_django_cms

Setting Apache to Proxy to Gunicorn

To add virtual host to Apache that forward dynamic content to Gunicorn:

sudo vi /etc/httpd/conf.d/my_django_cms.conf
NameVirtualHost *:80
<VirtualHost *:80>
    ServerName dj.example.com
    ServerAdmin webmaster@example.com
    DocumentRoot /var/www/my_django_cms/www
    RewriteEngine on
    ProxyPreserveHost On
    RewriteCond $1 !^/(favicon\.ico|robots\.txt|media|static)/
    RewriteRule ^(.*) http://localhost:8000$1 [P]
    <Proxy *>
            Order deny,allow
            Allow from all
            Allow from localhost
    </Proxy>
</VirtualHost>

If you have SELinux enabled you might need to apply the following command:

setsebool -P httpd_can_network_connect 1

If you have issues accessing your statics files from Apache it might be SELinux also:

restorecon -Rv /var/www/my_django_cms/

Django Multiple Settings with Single File

Instead of having multiple settings files one for production and one for development and so on, I prefer to use an “if” statement with environment variable:

ENV = os.environ.get('DJANGO_ENV', '')
print "==== Active Env: [%s]  ====" % (ENV)    
if ENV == "dev":
    # DEBUG
    DEBUG = True
    TEMPLATE_DEBUG = DEBUG
else:
    DEBUG = False
    TEMPLATE_DEBUG = False

Then from your “.bashrc” file:

export DJANGO_ENV=dev