In this post we will run Django application behind a NGINX webserver, which is a very common practice in production.

An easy way to start an Elasticsearch server with Kibana (Elasticsearch frontend) instance!

In the last post I covered starting a new Django project with Docker, and how to build a custom container image for my project, but what I haven’t covered is how to use a Database and how persist your data since all data stored in the container ephemeral and would only exist during the life span of the container.

Docker is an interesting technology, although still young it gained a lot of buzz and momentum.

The last couple of weeks I started playing with Docker to see how can I use it for Django project development and deployment.

Here I document minimal steps to setup a CentOS 7 machine, it suppose to be a live post, so I’ll keep updating it every time I found a step worth mentioning!

Update: It turns out that this setup is not needed, you should a have single GOPATH directory with all of your projects inside it, and use the vendor feature to handle each project dependencies. More details in a new post.

When working with Go language you must setup the GOPATH environment variable, but soon you will face two problems:

• Each project should have its own Go dependencies and its own Git code repo, so putting your source under GOPATH would be problematic.
• When working with “Atom” with “Go Plus” plugin, it needs to install several Go packages which would pollute your own source.

To solve both problems I added the following to my “.bash_login”:

Test

sudo yum groupinstall "Development Tools"
sudo yum install epel-release.noarch
sudo yum install yum-utils rpmdevtools

mkdir -p ~/rpmbuild/{SOURCES,SPECS}
wget http://example.com/lame-3.99.5.tar.gz
mv lame-3.99.5.tar.gz ~/rpmbuild/SOURCES
rpmdev-newspec lame
mv lame.spec ~/rpmbuild/SPECS
cat < < EOF > ~/rpmbuild/SPECS/lame.spec
# ------------------------------------------------------------
Name:           lame
Version:        3.99.5
Release:        1%{?dist}
Summary:        LAME MP3 Encoder

License:        N/A
Source0:        lame-3.99.5.tar.gz

%description
Educational tool to be used for learning about MP3 encoding. LAME aims to be the basis of a patent free audio compression codec.


%prep
%setup -q


%build
%configure
make %{?_smp_mflags}


%install
rm -rf $RPM_BUILD_ROOT
%make_install


%files
%doc
/usr/bin/lame
%dir /usr/include/lame
/usr/include/lame/*
/usr/lib64/*
/usr/share/doc/lame/html
/usr/share/man/man1/lame.1.gz

%changelog
# ------------------------------------------------------------
EOF
rpmbuild -ba ~/rpmbuild/SPECS/lame.spec
sudo rpm -Uvh ~/rpmbuild/RPMS/x86_64/lame-3.99.5-1.el7.centos.x86_64.rpm 


yumdownloader --source sox
sudo yum-builddep sox-14.4.1-6.el7.src.rpm 
rpm -i sox-14.4.1-6.el7.src.rpm 
vi ~/rpmbuild/SPECS/sox.spec
:  Add "--with-lame" to the configure line
%configure --with-lame
:
rpmbuild -ba ~/rpmbuild/SPECS/sox.spec 
sudo rpm -Uvh ~/rpmbuild/RPMS/x86_64/sox-14.4.1-6.el7.centos.x86_64.rpm
sox | grep mp3

References: https://fedoraproject.org/wiki/How_to_create_an_RPM_package

My presentation at Riyadh Linux User Group about Ansible

I want to allow host-2 to copy a file securely from host-1, so the easiest way is to use “scp” command which use “ssh” as a transport to copy the file.

If you want to do it manually it is straight forward “scp” invocation:

host-2$ scp host-1:data.csv .

But if you want to automate it you have to use “ssh” keys, but this means leaving a private ssh key on host-2 that can access host-1 without any restriction, i.e.

“sar” is a Unix command that collect, report, or save system activity information, it is different from other system status command like “top” or “vmstat” that only show real time status only, “sar” in the other hand collect these data so you can find the system state at any time.

This is a draft on configuring Munin to monitor services on a Linux machine, still dirty but published for my reference, if you have question let me know.

If you have a long running job that leaks few bytes of memory it will eventually will consume all of your memory with time.

Fail2ban is a program that scan your log files for any malicious behavior, and automatically block the offending IP.

PgBouncer is a lightweight connection pooler for PostgreSQL, connection pooling makes Postgres connection much faster, which is important in Web applications.

These are the step I perform after installing new Ubuntu machine.

Setting up a Postgres replaication

During this Eid vacation I spent many hours working on AgentX implementation in Python.

You can find the project in GitHub:

(https://github.com/rayed/pyagentx)

When I started producing backup worthy files (code, documents, projects, etc …), I realised the importance of backups after losing important files which happens to everybody. So I started my journey with backup solutions.

Backup generation 1: My first backup was simple directory copy operation, I copied my important directories to external floppy (then CD), and since it is manual operation I always forget about it and my backups was always old.

Just type:

sudo update-alternatives --config editor

I am trying new way to fix the annoying “cannot change locale (UTF-8)” error, this error usually show when you login from your OSX to a Linux machine:

So you managed to login to your Ubuntu Linux machine with a password by using the magic of ssh keys, and you installed “apticron” to email you whenever their is a system update, you login to the system and issue the update command “sudo aptitude update” and “sudo” asks you about your password! not only it is annoying, it also can’t be automated, imaging you manage 10 servers and you have to type the password to update each machine! or you want to automate using some remote execution application like Fabric or Salt (I am Python fan, so I won’t mention Chef or Puppet here).

The following is a sample Makefile for simple project, here I am building a simple “Bloom Filter” library, and “main” program to use it.

The library will have the following files:

• hash.h and hash.c
• bloom.h and bloom.c

and we should get “libbloom.a” out of it.

The “main” program will use:

• main.c
• bloom.h
• libbloom.a

and we should get “main” executable. So our make file should look like this:

$ cat Makefile
CFLAGS=-Wall -O3
LDFLAGS= -L.
LDLIBS=-lbloom

CFLAGS += `pkg-config --cflags libpcre`
LDFLAGS += `pkg-config --libs libpcre`

OBJS=main.o other.o libbloom.a
BLOOM_OBJS=hash.o bloom.o

all: main libbloom.a

main: $(OBJS)
        $(CC) $(LDFLAGS) -o $@ $^ $(LDLIBS)

libbloom.a: $(BLOOM_OBJS)
        ar rcs $@ $^

.PHONY: clean
clean:
        -rm main libbloom.a *.o 

Makefile use TAB not spaces for indentation.

My notes on installing and configuring PostgreSQL on Ubuntu Linux.

PostgreSQL is used by many large web sites and services, including Skype, Reddit, Instagram, “.org” registerer, check longer list here Prominent Users

Best of all it isn’t owned by anyone, not Oracle nor anyone else!

bash completion extends bash’s standard completion behavior to achieve complex command lines with just a few keystrokes. This project was conceived to produce programmable completion routines for the most common Linux/UNIX commands, reducing the amount of typing sysadmins and programmers need to do on a daily basis.

Just install:
sudo aptitude install bash-completion

Getting IPv6 connectivity to your server isn’t very hard in fact if you have a decent provier you might already have it configured.

Do you have IPv6 already configured?

First try this command

$ ifconfig -a 

You will have result like this:

eth0      Link encap:Ethernet  HWaddr 00:1e:0b:d6:7b:b8  
          inet addr:88.85.245.43  Bcast:88.85.245.47  Mask:255.255.255.248
          inet6 addr: fe80::21e:bff:fed6:7bb8/64 Scope:Link
          :
he-ipv6   Link encap:IPv6-in-IPv4

inet6 addr: 2001:470:1f08:1160::2/64 Scope:Global
inet6 addr: fe80::5855:f52b/64 Scope:Link
:
lo        Link encap:Local Loopback

inet addr:127.0.0.1  Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
:

This guide for my own reference please don’t rely on it, for a better introduction check out Git Reference website.

Basic concept and terminologies, Git have 3 logical places:

• Working Area: normal directory where you edit your project.
• Staging Area: Think of the staging as your project state before you start your coding session, after each coding session you add your changes to staging area using git add filename or git add .
• Snapshot Area: After multiple coding sessions you would finish a logical change, e.g. adding commenting system in web site, next you need to store a snapshot of your project after these changes, you would do that with git commit

Setting the environment:

Lately I migrated all of my server to Ubuntu server, what I really loved about it the easy of updating the system, all I need to is:

$ sudo aptitude update # to update packages meta data
$ sudo aptitude safe-upgrade # to perform the actual upgrade


But sometimes on of the package information meta data files get corrupted, and I will get this error:

$ sudo aptitude update
:
W: A error occurred during the signature verification. The repository is not updated and the previous index files will be used.GPG error: http://security.ubuntu.com hardy-security Release: The following signatures were invalid: BADSIG 40976EAF437D05B5 Ubuntu Archive Automatic Signing Key <ftpmaster@ubuntu.com>