Category Archives: Uncategorized

Django memory leak with gunicorn

tl;dr add “–max-requests” to Gunicorn to easily solve memory leak problems.

If you have a long running job that leaks few bytes of memory it will eventually will consume all of your memory with time.

Of course you need to find out where is the memory leak and fix it, but sometimes you can’t because it on a code that you use and not your own code.

Apache webserver solve this problem by using “MaxRequestsPerChild” directive, which tells Apache worker process to die after serving a specified number of requests (e.g. 1000), which will free all the memory the process acquired during operation.

I had a similar problem with Django under Gunicorn, my Gunicorn workers memory keep growing and growing, to solve it I used Gunicorn option “–max-requests”, which works the same as Apache’s “MaxRequestsPerChild”:

gunicorn apps.wsgi:application -b --workers 8 --max-requests 1000

Django Themes (or where to put base.html?)

The Wrong Way

I used to create a new directory to hold common templates like “base.html”, and add it TEMPLATES_DIR in the file:

TEMPLATE_DIRS = (os.path.join(BASE_DIR, 'templates') ,)

But in most cases the “base.html” would need to use CSS, JS, and image files to be functional, so I changed the url routing to access them (from the DEBUG mode only), something like:

$ vi apps/
# Serve Static Files 
from django.conf import settings
from django.conf.urls.static import static
if settings.DEBUG:
    urlpatterns += static(settings.MEDIA_URL, document_root=settings.MEDIA_ROOT)
    urlpatterns += static(settings.STATIC_URL, document_root=settings.STATIC_ROOT)
    urlpatterns += patterns('django.views.static',
        url(r'^(?P<path>(js|css|img)/.*)$', 'serve', {'document_root':  settings.BASE_DIR+'/../www'}),

This setup isn’t ideal for many reasons:

  • I had to modify and with complicated settings.
  • Theme design span multiple directories, and it isn’t self contained.
  • Switching the design is complicated, and include many changes.

Django Simple Themes

Nowadays I create a new Django application e.g. “my_theme” to hold my “base.html” template and all needed static files (CSS, JS, Images, etc …).

./ startapp my_theme

Then add it to INSTALLED_APPS:

    # django core apps ...
    # other apps ...

The directory structure for my new app looks like this:


and from my “base.html” (or any other template) I could access the static file using the static tag:

{% load staticfiles %}
<img src="{% static "my_theme/img/logo.png" %}" />

I don’t even need to change the “” file to access the static file, since the development server (i.e. ./ runserver) already knows how to find them.

But for production I have to define:

STATIC_ROOT = os.path.join(BASE_DIR, '../www/static')

and run:

./ collectstatic --noinput

New Theme

By having all theme files inside an application I can start new theme by copying “my_theme” to something like “new_theme” and replace it in the INSTALLED_APPS in the

What about uploaded files?

To access uploaded file from development server you need to define both MEDIA_URL and MEDIA_ROOT and change your “”:

$ vi apps/
from django.conf import settings
from django.conf.urls.static import static
if settings.DEBUG:
    urlpatterns += static(settings.MEDIA_URL, document_root=settings.MEDIA_ROOT)

Source: Serving files uploaded by a user during development

Sample Theme

You can download my sample theme from:

Protect your Server with Fail2Ban

Fail2ban is a program that scan your log files for any malicious behavior, and automatically block the offending IP.

The default Fail2ban installation on Ubuntu will protect ssh, but in this article I will show how to protect against WordPress comment spammers too, to slow them down.

Installation & Configuration

# Install fail2ban
$ sudo apt-get install fail2ban

# Copy default config to custom config
$ sudo cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local

# Add your own IPs so they never get blocked
$ sudo vi /etc/fail2ban/jail.local
ignoreip =

# restart it
$ sudo service fail2ban restart

Fail2ban is now configured and running.

You can use the following commands to inspect and trouble shoot its operation:

# fail2ban usually add new rules to your IPTables
$ sudo iptables -L

# You can check the status of specific rules using the command:
$ sudo fail2ban-client status ssh

# and of course check log to see if it is working:
$ sudo tail -f /var/log/fail2ban.log 

Protecting WordPress Comments

By default fail2ban protect many ssh but let’s assume you want to protect WordPress from spam bots trying to post comments on your blog.

First we add a filter to catch the attempts by creating new filter file named “/etc/fail2ban/filter.d/wordpress-comment.conf”:

$ sudo vi /etc/fail2ban/filter.d/wordpress-comment.conf 
# Block IPs trying to post many comments
failregex = ^<HOST> -.*POST /wordpress/wp-comments-post.php

Then we create a new JAIL by adding the following to “jail.local” file:

$ sudo vi /etc/fail2ban/jail.local
enabled = true
port = http,https
filter = wordpress-comment
logpath = /var/log/apache2/*access*.log
bantime = 3600
maxretry = 5

Then restart fail2ban using:

sudo service fail2ban restart

كيف تنشئ موقع انترنت مجانا

هل تملك عنوان بريد الكتروني تحت gmail او hotmail؟ ماذا لو بحثت عن اسمك هل ستظهر صفحتك في تويتر او فيس بوك؟

لماذا لا تقوم بحجز اسم نطاق خاص بك مثل ويكون عنوانك البريدي هو، وبالمجان ايضاً.

نحتاج للخدمات ما يلي:

  1. تسجيل اسم النطاق domain: المركز السعودي لمعلومات الشبكة (SaudiNIC) يقدم نطاق تحت العنوان “.sa” مجانا.
  2. استضافة اسم النطاق: موقع ClouDNS يقدم خدمة استضافة النطاقات مجانا. (نعم تسجيل النطاق شئ واستضافته شئ آخر) اتمنى لو يقدم المركز السعودي هذه الخدمة مجانا ايضاً.
  3. استضافة موقع الويب، والخيارات هنا عديدة ما اعرفها هي:
    1. Github Pages مع اسم نطاق مخصص Setting up a custom domain with GitHub Pages
    2. موقع مع اسم نطاق مخصص Using a custom domain name
  4. استضافة البريد: موقع ClouDNS يقدم خدمة استقبال واعادة ارسال البريد Email Forwarding لثلاث حسابات مجانا.

Accelerating Postgres connections with PgBouncer

PgBouncer is a lightweight connection pooler for PostgreSQL, connection pooling makes Postgres connection much faster, which is important in Web applications.

Here I will explain the steps I used to configure it under Ubuntu 14.04 LTS.

Step 1: We would configure users allowed to connect to PgBouncer:

$ sudo vi /etc/pgbouncer/userlist.txt
"rayed"  "pgbouncer_password_for_rayed"

Step 2: We configure databases PgBouncer will pool for, and how PgBouncer will authenticate user:

$ sudo vi /etc/pgbouncer/pgbouncer.ini
rayed = host=localhost user=rayed password=postgres_password_for_rayed
auth_type = md5
;auth_file = /8.0/main/global/pg_auth
auth_file = /etc/pgbouncer/userlist.txt

The default value for “auth_type” is “trust” which means a system user “rayed” will be allowed to connect to Postgres user “rayed”, I change to “md5″ to force a password checking from the file “/etc/pgbouncer/userlist.txt”.

Step 3: We will allow PgBouncer to start:

$ sudo vi /etc/default/pgbouncer 

The default value is “0” which means don’t start PgBouncer ever, it is a way to make sure you know what you are doing :)

Step 4: Starting pgBouncer:

$ sudo service pgbouncer start

Step 5: Test the connection, by default “psql” connect using port “5432”, and pgBouncer use “6432”, so to test a pgBouncer connection we would use the following command:

$ psql -p 6432 

If you get “Auth failed” error make, make sure the password you entered is the one you typed in step 1, if the command doesn’t ask for password try it with “-W” option, e.g. “psql -p 6432 -W”.

Translation in Django


In your project “” setup the following values:

    os.path.join(BASE_DIR, 'locale'),

Here we defined the location of our translation files, by default Django will look for it under application directories under “locale” directory, but here we define it for the whole project.

The LANGUAGE_CODE line define a fixed translation to Arabia “ar”.

Source Code

For Python source code Django uses “ugettext” function aliased as “_” (underscore) to translate text strings:

from django.utils.translation import ugettext as _
from django.http import HttpResponse
from django.shortcuts import render

def page1(request):
    output = _("Welcome to my site.")
    return HttpResponse(output)

def page2(request, template_name='index.html'):
    return render(request, template_name)


Inside Django templates, Django uses trans template tag with the text to translate, don’t forget to load the tag using “{% load i18n %}”

{% load i18n %}

<h1>{% trans "Hello World" %}</h1>


After preparing the code we the following steps:

  • Collect translation string using “ makemessages” command.
  • Edit the translation file “django.po”
  • Compile the translation to “”.

$ cd project_home

# Make the "locale" directory to store translation data
$ mkdir locale

# scan the project for translation strings
$ makemessages -l ar

# Edit the translation file and add your translation
$ vi locale/ar/LC_MESSAGES/django.po
msgid "Hello World"
msgstr "مرحبا يا عالم"

# compile django.po to
$ compilemessages

That’s it, you should be able to see your applications translated!

Graph you Data and Email It

I have a new website and I want to know the number of new signup every day, so I wrote a small script that will print the number of new signups today.


I ran this script daily using a cron job and add it to user.dat file:

0 0 * * *   /home/rayed/bin/ >> /home/rayed/var/user.dat

After few days the file will look like this:


I could send this file daily and read, but it wouldn’t give a good picture of how new user signup is changing by time, so the next logical step is to convert it to a graph for easier understanding.

I used gnuplot to convert the textual data to a graph, and automatically email it to me.

So I wrote the following small script,




set terminal png \
	medium \
	size 800,400;
plot  'user.dat' with lines;

img_base64=`echo $plot | gnuplot | base64`

sendmail $to_email <<EOF
From: <$from_email>
To: <$to_email>
Subject: Plot and Inline image from CLI
Mime-Version: 1.0
Content-Type: multipart/related; boundary="boundary-example"; type="text/html"

Content-Type: text/html; charset="US-ASCII"

This email sent from Linux CLI:
<IMG SRC="cid:plot_image_1" ALT="Plot">

Content-ID: <plot_image_1>
Content-Type: IMAGE/PNG
Content-Transfer-Encoding: BASE64



When you run it you will receive the following graph on your email:


Of course you can edit the email HTML template, add new graph, or do whatever you like to customise it.

I hope you find it useful.