Enabling SSL in Python

After installing Python 2.5 for use with Google's AppEngine I discovered that SSL had not been compiled in. I use Ubuntu 10.10 as the test server. First install the SSL headers:-

$ sudo apt-get install libssl-dev

Then rebuild Python to include SSL:-

$ cd /usr/local/src/Python-2.5.5/
$ sudo rm Modules/Setup
$ sudo joe Modules/Setup.dist

Uncomment lines 206-209 and set line 206 to:-

SSL=/usr

Save and exit, then:-

$ sudo ./configure --prefix=/usr/local/python2.5
$ sudo make clean
$ sudo make
$ sudo make install

Finally, remove the virtualenv already created and create a new one so that the new python executable and libraries are copied in.

Configuring virtualenv and appengine

Google's AppEngine requires a Python 2.5 environment. I created this by first installing Python 2.5 on my system, then installing virtualenv:-

$ cd /usr/local/src
$ sudo tar -zxf virtualenv-1.5.1.tar.gz
$ cd virtualenv-1.5.1
$ sudo python setup.py install

Install AppEngine into your project folder:-

$ cd ~/appengineproject
$ unzip google_appengine_1.4.1.zip
$ sudo ln -s ~/appengineproject/google_appengine/ /usr/local/google_appengine

And finally create a virtualenv for AppEngine

$ virtualenv --python=python2.5 pythonenv

and configure it by adding the following lines to pythonenv/lib/python2.5/site-packages/gae.pth:

~/appengineproject/google_appengine
~/appengineproject/google_appengine/lib/yaml/lib
~/appengineproject/google_appengine/lib/antlr
~/appengineproject/google_appengine/lib/ipaddr
~/appengineproject/google_appengine/lib/webob
~/appengineproject/google_appengine/lib/django
~/appengineproject/google_appengine/lib/fancy_urllib
~/appengineproject/google_appengine/lib/simplejson
~/appengineproject/google_appengine/lib/graphy

python 2.5, zlib 1.2.5 and failing make test

I needed to have Python 2.5 for an App Engine project. My test virtual machine is running Ubuntu 10.10 with Python 2.6.5 so I decided to install Python 2.5 and use virtualenv to manage different development projects. Obviously this wasn't going to be that simple. Firstly I installed Python 2.5 using a convenient staging area. I prefer /usr/local/src:-

$ cd /usr/local/src
$ sudo wget http://www.python.org/ftp/python/2.5.5/Python-2.5.5.tgz
$ sudo tar -xzf Python-2.5.5.tgz
$ cd Python-2.5.5
$ sudo ./configure --prefix=/usr/local/python2.5
$ sudo make
$ sudo make test

And I saw the first problem:-

test_zlib skipped -- No module named zlib

This meant I also had to install zlib:-

$ cd /usr/local/src
$ sudo wget http://zlib.net/zlib-1.2.5.tar.gz
$ sudo tar -zxf zlib-1.2.5.tar.gz
$ cd zlib-1.2.5
$ sudo ./configure
$ sudo make
$ sudo make install

This improved things a bit, but I still had a failure with the zlib test:-

test test_zlib failed -- Traceback (most recent call last):
File "/usr/local/src/Python-2.5.5/Lib/test/test_zlib.py", line 72, in test_baddecompressobj
self.assertRaises(ValueError, zlib.decompressobj, 0)
AssertionError: ValueError not raised

A bit more research showed that changes in zlib after 1.2.3 caused the zlib test to fail. Sadly the fix was not being backported to 2.5, meaning that I had to change the test manually:-

$ cd/usr/local/src/Python-2.5.5/Lib/test
$ sudo joe test_zlib.py

Go to line 72, change 0 to -1 on that line, save and exit.

Now the zlib test passed, and Python 2.5 can be installed:-

$ cd /usr/local/src/Python-2.5.5
$ sudo make install

OS/X 10.6 LC_CTYPE

On OS/X 10.6, Apple have defined the LC_CTYPE environment variable as 'UTF-8'.

$ locale
LANG=
LC_COLLATE="C"
LC_CTYPE="UTF-8"
LC_MESSAGES="C"
LC_MONETARY="C"
LC_NUMERIC="C"
LC_TIME="C"
LC_ALL=

LC_CTYPE is meant to specify the encoding for the locale as well as the character properties, yet UTF-8 defines only the encoding. This can lead to problems with internationalisation in software, like Python's Babel module which throws a ValueError exception:-

ValueError: expected only letters, got 'utf-8'

This can be fixed by changing the LC_CTYPE value to include the character properties by specifying the country/language portion as well:-

export LC_CTYPE="en_CA.utf-8"

You may want to add it to your .profile to make the change permanent:-

echo "LC_CTYPE='en_CA.utf-8'" >> ~/.profile

Ideally Apple are the ones who need to fix this.

Amin Maalouf

"Many are so angry or bewildered that they give up trying to understand what is going on. Many refuse to make a contribution to the emerging universal culture because at some point they decided once and for all that the world around them was incomprehensible, hostile, bloodthirsty, demented, or diabolical. Many are tempted to see themselves as mere victims - victims of America, of the West, of capitalism or liberalism, of the new technologies, of the media, of change, and so on. It cannot be denied that such people really do feel wronged and suffer accordingly, but their reaction seems to me inappropriate. To imprison oneself in a victim mentality can do the injured party even more harm than the aggression itself. And this is as true of societies as of individuals. They huddle themselves away, they close their minds, they ruminate, they give up looking for anything new, they don't explore any more, they don't make any more progress, they are afraid of the future, of the present and of everyone else."
Amin Maalouf, "On Identity" (p. 102)

Directory sharing between Apache and a user

There are times when a user needs to maintain files in a directory managed by Apache. This would usually be a folder created via the web server and is therefore owned by that web server, the www-data or nobody user.

There is more than one way to allow another user access to that directory and the one I prefer is to ensure that the Apache user has a umask of 002 and to add the user to the Apache group.

These are the steps I took on Ubuntu 10.4 LTS:-

sudo echo 'umask 002' >> /etc/apache2/envvars
sudo usermod -a -G www-data username
sudo apache2ctl restart

The location of the envvars file can be found by inspecting /etc/init.d/apache2 (or maybe /etc/init.d/httpd).

These are the steps I took on a Virtual Private Server at Servint (as root):-

echo 002 > /var/cpanel/easy/apache/rawenv/umask
echo 'umask 002' >> /usr/local/apache/bin/envvars
usermod -a -G nobody username
/etc/init.d/httpd stop
/etc/init.d/httpd start

As this is a managed server and the envvars file is recreated by a script when Apache is rebuilt, I had to update both /var/cpanel/easy/apache/rawenv/umask and /usr/local/apache/bin/envvars. The first is to include the umask setting in the automated creation of envvars and the second was to do a manual insert before restarting the web server.