Ryan Kanno: The diary of an Enginerd in Hawaii

Everything you've ever thought, but never had the balls to say.

My LinkedIn Profile
Follow @ryankanno on Twitter
My Feed

Custom Python installation for Django on Dreamhost

Now that my MBA class is finally done for the summer, I can focus on more important things… like upgrading my Python installation on Dreamhost for my Django application. Seeing as how Dreamhost is still behind the Python times, with Python 2.4 hidden in Dreamhost obscurity, I figured I’d blog about updating your Dreamhost Python installation (and subsequent MySQLdb libraries) to Python 2.5.

The very first thing I did was search Google. You know, I really don’t know how people lived pre-Googs. In any case, I found this blog posting describing exactly what I wanted to do. Thanks Ben! Since I’m not a big fan of running one large batch script people create in their blogs, I’ll break it down for the non-*nix fans out there.

Before I begin, I’m assuming that you already have Django running on Dreamhost. If you’re having a “wtf” moment, make sure to stop by Jeff’s blog and read “Setting up Django on Dreamhost“. (This is how I set mine up). To follow my short tutorial, you’ll need shell access to your Dreamhost account.

After ssh’ing into your Dreamhost account, you should be in your home directory (/home/username). According to the Filesystem Hierarchy Standard, the /opt dir “is reserved for the installation of add-on application software packages.” With that said, issue the following commands:

$ mkdir opt
$ mkdir downloads
$ cd downloads
$ wget http://www.python.org/ftp/python/2.5.1/Python-2.5.1.tgz
$ tar xvzf Python-2.5.1.tgz

First, create a directory named opt. Next, create a directory named downloads for all your files. Change into the downloads directory, then download the latest Python from http://www.python.org. Finally, unzip and untar the package into the download directory. Everything will extract into a directory named Python-2.5.1.

$ cd Python-2.5.1
$ ./configure --prefix=$HOME/opt/ --enable-unicode=ucs4
$ make
$ make install

Change into the Python-2.5.1 directory and type in the following configure command. Basically, configure prepares your installation for compilation. The –prefix flag will install machine-independent data files in subdirectories of the specified directory. The default is to install in /usr/local, but it’s overwritten with the opt directory created earlier. Finally, run make and make install which will install your custom Python installation.

$ cd ..
$ rm -rf Python-2.5.1

Finally, delete the Python-2.5.1 directory. Before you can use this Python installation, you have to add the /opt/bin directory to your path. To do this, add /opt/bin to your .bash_profile file in your home directory. To do so, you’ll have to add the following to your .bash_profile.

export PATH=$HOME/opt/bin/:$PATH

Basically, this allows you to type ‘python’ in your shell and reach the custom Python 2.5.1 installation instead of the Dreamhost one. To make sure that our Python installation is working, type the following in your home directory (cd ~):

$ source .bash_profile
$ python --version

After the last command, you should see the following: Python 2.5.1. If that displays, your upgrade was successful! After upgrading your Python installation, you’re not done yet. Since Dreamhost uses an old MySQL-Python installation, we’ll upgrade that as well. Type the following in your home directory:

$ cd downloads
$ wget http://internap.dl.sourceforge.net/sourceforge/mysql-python/MySQL-python-1.2.2.tar.gz
$ tar xvzf MySQL-python-1.2.2.tar.gz
$ cd MySQL-python-1.2.2
$ python setup.py install

First, change into the downloads directory and issue the wget command to download the latest MySQL-Python files from Sourceforge. Once you’ve received the files, unzip and untar the package. All the files will extract into a directory called MySQL-python-1.2.2. Change into this directory and install the files by typing python setup.py install. If you followed the custom Python installation, the files should build and extract into the ~/opt/lib/python2.5/site-packages/ directory as an egg file.

You now have a custom Python installation and a MySQL-Python upgrade!

Voila!

Update: Just so you don’t get caught up in the same mistake that I made, to be sure that your Django fcgi installation is using your custom Python installation, make sure the dispatch.fcgi file reads as such:

#!/home/USERNAME_GOES_HERE/opt/bin/python
import sys

sys.path += ['WHATEVER_PATHS_YOU_NEED']

from fcgi import WSGIServer
from django.core.handlers.wsgi import WSGIHandler
import os

os.environ['DJANGO_SETTINGS_MODULE'] = 'wegoeat.settings'
WSGIServer(WSGIHandler()).run()
Be Sociable, Share!
  1. Hello,

    I read your manual for use Python2.5 in dreamhost and I have can compile it, and execute in a shell. But when I will execute via web, I have the error “FastCGI: incomplete headers”.

    Do you know because launch this error?

  2. Sorry,

    I have found a solution. The error has mine, because I follow the instruction of dreamhost’s wiki: http://wiki.dreamhost.com/Python#Installing_Python_2.5.1_with_the_Unix_account_setup .
    And in these intructions, he compile python with “–enable-shared” option.

    Now I repeat the compilation without this option, and voilà, it’s run. http://proves.xin.cat/

    Thanks for your time.

  3. Wowww, the last post not sended.

    In this, I have tell that my problem have solutioned. The error was that I follow the instructions of Dreamhost’s wiki for compile Python2.5, and in this, it use the option –enable-shared.

    Then, I have compiled another the Python2.5 without this option and voilà, it’s run.

    Thanks for your time.

  4. First, thanks very much for posting this. Quite cool.

    I keep getting this:

    [hades~]$ opt/bin/python2.5
    ‘import site’ failed; use -v for traceback
    Python 2.5.1 (r251:54863, Dec 27 2007, 21:13:17)
    [GCC 3.3.5 (Debian 1:3.3.5-13)] on linux2
    Type “help”, “copyright”, “credits” or “license” for more information.
    >>> import os
    Traceback (most recent call last):
    File “”, line 1, in
    ImportError: No module named os
    >>>

    Dreamhost also seems to have removed the os module from their other python installations. What gives?

    Thanks,
    Len

  5. @Leonard

    This is what I did from my installation.

    [look]$ python
    Python 2.5.1 (r251:54863, Jul 23 2007, 15:42:46)
    [GCC 3.3.5 (Debian 1:3.3.5-13)] on linux2
    Type “help”, “copyright”, “credits” or “license” for more information.
    >>> import os
    >>>

    You might want to check out your PYTHONPATH system variable. This is what I’ve added to my .bash_profile.

    export PATH=$HOME/opt/bin/:$PATH
    export PYTHONPATH=$PYTHONPATH:$HOME/python/lib/site-packages

  6. [...] to custom-install Python2.5 or some other more recent version. Instructions on how to do that are here. After you’ve got python2.4 or later, you can [...]

  7. [...] Python 2.5 running on Dreamhost and migrate my application to that. It turned out to be easy with these instructions. I had tried compiling Python on Dreamhost before, but had always ended up with a binary that did [...]

  8. @douglas – You’re welcome!

  9. It works also for py2.6!!!!!!!!!!!!!!!

  10. Hello,

    Thanks for this article,
    I’m not dreamhost user, but your tutorial works in my web hosting.

    but i still have little trouble,
    if i had change something in settings.py, it’s not affect my django website.
    my running django web still used old settings.
    do you know what to do in this situation ?

    Thanks, sorry for my poor english

  11. @Sigit -

    I suspect what’s happening is that you’re deploying Django+Python via mod_python on your shared hosting. If you take a look at these instructions on the Django site: http://docs.djangoproject.com/en/dev/howto/deployment/modpython/, there’s a line there that says the following: ‘When deploying Django sites on mod_python, you’ll need to restart Apache each time you make changes to your Python code.’ I think all you need to do is tell Apache to restart itself anytime you issue code changes and things should be awesometown! :)

  12. [...] Upgrade the version of python on your linux accounts to python 2.6 by installing a local version of python on your account … instructions on how to do this are at http://blog.localkinegrinds.com/2007/08/20/custom-python-installation-for-django-on-dreamhost/ [...]

  13. [...] I recompiled Python to create a 64bit version. While I was at it, I upgraded from v2.5.2 to v2.6.5. This great page by Ryan Kanno outlined the steps. I just changed the version number. I also deleted my old installation of Python so there [...]

  14. [...] not much supportive ("If you are positive that you need to install Python, reconsider"), and a few unofficial ones. Here are the steps I used – try them at our own expense, since I can't give any guarantees [...]

  15. Hi, I put together this script to compile Python and some dependencies in the home folder of a Dreamhost shared server.
    Feel free to use it, modify it and contribute to it on Bitbucket.
    http://bitbucket.org/tmslnz/python-dreamhost-batch/src/

    Make sure you read though it. You can disable packages by commenting them out at the start of the script.
    It is safe enough to use as it backs up your .bashrc file and installation folders.

  16. [...] was a blog post out there where someone outlined how to do a custom Python environment. It’s pretty simple, [...]

  17. I couldn’t get the mysql-python tar file from the link posted (granted, it is 3 years old, i think) so i tried getting this version instead:
    http://sourceforge.net/projects/mysql-python/files/mysql-python/1.2.3/MySQL-python-1.2.3.tar.gz/download

    It doesn’t seem like it’s the right package b/c when I go to run python setup.py install, i get this error:
    Traceback (most recent call last):
    File “setup.py”, line 5, in
    from setuptools import setup, Extension
    ImportError: No module named setuptools

    1.2.3 assumes the module “setuptools” is installed already, but that is not the case when setting this up for the first time.

    there is also the 1.2.2 package here:
    http://sourceforge.net/projects/mysql-python/files/mysql-python/1.2.2/MySQL-python-1.2.2.tar.gz/download

    which is the one you would want and installs setuptools for you.

  18. @Dale – Looks like you need to install setuptools. You can find it here -> http://pypi.python.org/pypi/setuptools :)

  19. @ryan, thanks for getting back, i mentioned if you grab v1.2.2, it will install setup tools for you.

  20. @Dale – Sorry about that. Totally missed it in the comments. :)

  21. [...] Custom Python Install by LocalKineGrinds This was written by Jeff. Posted on Sunday, September 28, 2008, at 8:06 am. Filed under Django, Python. Bookmark the permalink. Follow comments here with the RSS feed. Post a comment or leave a trackback. [...]

  22. Thanks a bunch for this write-up, it really eliminated all the questions I had doing this, and I’m running shiny new Python 2.7.2 with Django 1.3 as we speak. Excellent work!

  23. Hi, for continuity’s sake please note that the previously linked script (http://bitbucket.org/tmslnz/python-dreamhost-batch/src/) now lives at GitHub as a Git repository. New address: https://github.com/tmslnz/Dreamhost-Custom-Env

Please leave a reply »

Powered by Wordpress. Stalk me.