Mitch Garnaat from CloudRight has pointed out that you can actually set the policy of the S3 file in the set_contents_from_file call instead of making another roundtrip request into S3 (and saving you some coin). Thanks Mitch!

Btw, I’m using App Engine Patch 1.0 and Boto 1.6a.

Sorry I haven’t updated my blog in a few weeks months, but I’ve been a little busy. With that said, along with Erlang, I’ve been playing around with Google App Engine, App Engine Patch (for Django support), and the Boto library (for Amazon S3 support). After not having touched Python code in a few months, I wanted to document some of my lessons learned to help over developers who may be in a similar boat.

Lessons learned

• If you’re upgrading the App Engine Patch, make sure you don’t have the App Engine library installed in a hidden directory
• Uploading bulk data changed ever so slightly
• If you’re not running off of Boto’s trunk, you’ll need to patch your Boto installation to work with App Engine.

Make sure the App Engine library isn’t installed in a hidden directory

Apparently, Google’s SDK 1.1.9 doesn’t like to rely on files that won’t be uploaded with your application – and hidden directories are no longer uploaded. I was running into the dreaded purple-nurple screen of death. Thank goodness for this AppEngine Google Group post, but I’m still not even sure when this popped up considering Google’s articles still refer to this setup.

Bulk upload

Compared to the previous SDK I was playing around with, bulk uploading changed significantly. I recall having to patch Goog’s bulkupload.py file to get unicode support. However, their new remote api tool has definitely fixed this issue, so +1 for Googs. People are reporting that uploading unicode is still broken, but it’s not. Or at least it wasn’t for me. Second, if you’re like me and don’t read documentation, you’ll find out (the hard way) that the method signature to HandleEntity changed. Instead of accepting a datastore.Entity, it’s now expecting a db.Model object.

Note: When actually running the remote api tool, you’ll also want to make sure your PYTHONPATH includes your current project. (Another one-liner in the documentation. )

Integrating Boto + App Engine

I wasn’t running off of Boto’s trunk and I was getting an obscure type conversion error. Being too lazy to check out the source, I jumped to their issue tracker and found a patch (halfway down the page) by one of the App Engine Patch lead devs. Apply the patch and you’ll be on your way to uploading images/data from App Engine into Amazon S3! If you’re looking for example code, I’ve included a small snippet of what I tested.

1
2
3
4
5
6
7
8
9
10
11

    @staticmethod
    def upload_to_s3(original_filename, photo):
        """ Upload a photo file, storing its original name as metadata in an S3 bucket """
        connection = Connection(settings.AWS_ACCESS_KEY_ID, settings.AWS_SECRET_ACCESS_KEY)
        bucket = connection.get_bucket(settings.AWS_IMAGE_BUCKET_NAME)
        photo_uuid = str(uuid.uuid4())
        new_key = Key(bucket)
        new_key.key = photo_uuid
        new_key.set_metadata('original_filename', original_filename)
        new_key.set_contents_from_file(photo, policy='public-read')
        return photo_uuid

Note: I only tested the code above with small images ~300-500K in size and it seemed to work perfectly fine (with no load! ). As always, feel free to use, steal, take, and/or copy anything on this blog. Hopefully somewhere, someone on the Interwebs will find these tips handy!

Enjoy!

I guess Googs finally caught on as their 1.1.2 installer works on Win2K! FTW!

Update

After finally getting time to play around with the Google App Engine Django helpers, here’s a few more steps to integrate nicely with the helper suite.

• Move the appengine installation from C:\AppEngine\ to where the Windows installer would have installed it to: C:\Program Files\Google\google_appengine (make sure to clean up your .pyc files)
• Add the following to your PYTHONPATH system variable: %APPENGINE%\;%APPENGINE%\lib;%APPENGINE%\lib\yaml\lib;%APPENGINE%\lib\webob;

After following the instructions, you should be good to go with Django + AppEngine! FTW! Whee.

I sense some pure, unadultered haterade. (j/k)

Since Python is one of those insert_any_synonym_for_fun languages that just works, here’s how to get the Google App Engine SDK working in Win2K.

• Download the Linux/Other platform package and unzip to somewhere neat.
• Add a System Environment variable called ‘APP_ENGINE_HOME’ that points to your App Engine installation. (Notice, I installed mine into C:\AppEngine)

  

• Add the System Environment variable to your System Path so the Windows shell can execute the included Python files.
• Make sure you have .py files associated with the python.exe executable located in your Python installation. (Check file types under folder options)
• Follow the tutorials: here and here, or learn with others – just to name a few.
• Oh, and before I forget, if you develop an application and realize that you can’t kill the development appserver (dev_appserver.py) by pressing Ctrl-C, I found a solution here. Basically, press Ctrl-C, hit the server with your browser one more time and voila, the development application server dies. Thanks Frank!

As an added bonus…

Last but not least…

Big ups to Mr. Fitz for solving all my Google App Engine issues and thanks to Mr. Harper for causing them.

Voila! (Enjoy)

Three things spurred me to release django-yui-layout-templates.

1. I’ve always wanted to see what GoogleCode offered in relation to SourceForge / RubyForge.
2. I’m so caught up in corporate America staring at Java / Ruby code all day, not only haven’t I blogged about anything Django related in quite a while, but it’s nice to get some commentary from the community, i.e. “your code sucks”. (Brings me back to reality)
3. I found myself using the same templates on a variety of projects and figured that I could do my part and help eliminate unncessary cruft/duplication.

So without further adieu, check out the project here. I know, I know – nothing revolutionary here, but I figure since Django is picking up some steam, these templates might help a Djangonaut get a head start on their next million dollar idea.

Voila! Enjoy!

Background

As a Django “proof-of-concept”, I’m working on a local restaurant review site for my home state of Hawai`i. (I actually just released it yesterday). For each restaurant, I want to be able to calculate the average of all reviews and display this listing in a paginated view. (Yes, I do realize there’s no average rating, but that has to do with there being no users. ;P).

The Problem

Having a serious “wtf was I thinking moment”, I initially wrote a Restaurant model function that returned the average (review) rating for each restaurant instance. Little did I realize that when I actually displayed the restaurant’s average reviews, I would be making an additional SQL avg() call for every restaurant. Though I’m paging “n” records at a time, this function added an additional “n” SQL calls for every view that contained a restaurant listing, just to name a few.

In pseudo-code, my initial naive function resembled the following: (I’m sure we’re all guilty of writing something of the sort… ok, fine, I know I was. ;P)

1
2
3
4
5
6

     def get_average_review(self):
         query = 'QUERY TO GET AVERAGE (SELECT AVG(rating)...); (I have the query below)'
         # Get cursor from connection
        cursor = connection.cursor()
        cursor.execute(query)
        return cursor.fetchall()

Duh.

Here’s a picture of the number of queries it took:

The “extra()” solution

After profiling my application and realizing what a bone-headed mistake I made, I began researching the extra() Queryset modifier. Yes, I realize that these extra lookups aren’t the most portable and often violate the DRY principle, but it’ll probably suffice for most of all my personal projects.

Since I’m already retrieving a list of Restaurants and filtering them via letter, island, and what not, I figured I could add an average rating subquery. The entire call looks as such:

1
2
3
4
5
6
7

     restaurants = Restaurant.objects.filter(name__istartswith = letter).extra(
             select={'avg_rating': 'SELECT AVG(overall_rating) FROM restaurants_restaurant as res, reviews_review, django_content_type \
                                          WHERE restaurants_restaurant.id = res.id \
                                          AND res.id = reviews_review.object_id \
                                          AND reviews_review.content_type_id = django_content_type.id \
                                          AND django_content_type.model = \'restaurant\''},
                       )

As you can see, I’m exploiting the fact that restaurants_restaurant will be available from the Restaurant.objects.filter() call. (I know, I know… bad for portability).

But voila!

Now, in my templates, when I iterate over the restaurants, I can get issue the following:

1
2
3
4
5
6
7
8
9
10
11

{% for restaurant in restaurant_list %}
<tr>
    <td><a href="{{restaurant.get_absolute_url}}">{{ restaurant.name }}</a></td>
    <td>{% if restaurant.avg_rating %}
	   {% load show_stars %} 
           <span class="average-rating">
	   {% show_stars <strong>restaurant.avg_rating</strong> of 5 round to quarter %}
           </span>
           {% endif %}</td>
</tr>
{% endfor %}

Notice how I used my show_stars template tag that I blogged about a few weeks ago to display the average restaurant rating. (Cheap shameless plug, but damn effective! ) I’d link to a page in action, but since I just opened up my site to a few select users, I’ll update this post when I actually have any reviews.

Oh, and before I forget, thanks to my co-worker Stephen for assisting me with my SQL issues!

Here’s a picture of the final result:

Note:

As an added bonus, I also realized a few other ‘spots’ where the .extra() Queryset modifier would come in handy. Since I’m also using the wonderful django-voting application from Jonathan Buchanan, I came across this post about accessing a dictionary via a template in the Django-users Google Group.

Basically, I had come across the same issue as the poster. Since I allow users to vote on reviews (similar to Amazon, Yelp, etc.), I wanted to retrieve the score of each Review instance to display on a paginated listing of all Reviews. Using the same extra() modifier, I was able to inject the total number of votes and the score when I retrieved all Reviews as such:

Btw, I just injected most of the code from Jonathan’s template tag.

1
2
3
4
5
6
7
8
9
10
11

.extra(select={'total_votes': 'SELECT COUNT(vote) FROM votes as v, reviews_review as rev, django_content_type \
                                        WHERE reviews_review.id = rev.id \
                                        AND v.object_id = reviews_review.id \
                                        AND v.content_type_id = django_content_type.id \
                                        AND django_content_type.model = \'review\'', 
 
                                        'score': 'SELECT SUM(vote) FROM votes as v, reviews_review as rev, django_content_type \
                                        WHERE reviews_review.id = rev.id \
                                        AND v.object_id = reviews_review.id \
                                        AND v.content_type_id = django_content_type.id \
                                        AND django_content_type.model = \'review\''},)

Pretty neat right?

Now, when iterating through the reviews, I can use the following:

1
2
3
4
5
6
7
8
9
10
11
12
13

{% for review in object_list %}
	<tr>
		<td><a href="{{review.content_object.get_absolute_url}}">{{ review.content_object.name }}</a></td>
		<td><a href="{% url profile-detail username=review.user.username %}">{{ review.user.username }}</a></td>
		<td><nobr>{% load show_stars %}
			<span class="rating">{% show_stars review.overall_rating of 5 round to half %}</span>
			</nobr>
		</td>
		<td><span style="font-weight:bold; color:#092e20;">{{ review.get_recommendation_display }}</span></td>
		<td><span style="font-size:.875em;">{{ review.submit_date|timesince }} ago</span></td>
		<strong><td>Total of {{ review.score|default:0 }} from {{ review.total_votes }} {{  review.total_votes|pluralize:"person,people" }}.</td></strong>
	</tr>
{% endfor %}

Hope y’all learned something like I did! Oh, and before I forget my standard disclaimer, “since this is on my blog, feel free to take/use/steal/distribute/copy/modify any code you see fit, but if you find any bugs, have any comments, or think the code can be cleaner, I’d love to hear from you.”

Enjoy!

According to Max, it’s against Facebook’s TOS to cache the data. I’ve totally must’ve skipped over it in the TOS… ok, fine… I admit, I didn’t even read it. Thanks Max! And to think this response was less than an hour after my post! Yay, community. In any case, this problem is still relevant to many a graph-structured sites. Comment on!

Since I don’t know of many techies in Hawaii, I figured I would post a blog in hopes of attracting a solution from the akamai Django community.

The Background

I’m currently writing a Facebook application in Django (which I’ll be blogging about later).

The Problem

I want to be able to programatically tell if you’ve updated your friends in Facebook. Basically, at the end of the day, Facebook provides an API call that allows you to retrieve a list of uid’s that represent your ‘friends’. I’m planning to cache this data within my own system. I was wondering how Pythonic people would solve the following:

Is there a data type or algorithm (or combination thereof) I can use to compare the returned uids with my uid data cache? This comparison algorithm will tell me if you’ve added or subtracted new friends in Facebook. As a bonus, I’d prefer this comparison optimized (for Python).

Possible Solutions

So I’ve sat down and made a list of possible solutions:

• Creating an in memory hash from my cached data and seeing which Facebook API returned uids collide and which don’t. (EASY)
• Using SET comparison in MySQL. (EASY)
• Possibly encoding the uids in a way to optimize the comparison of two sets; though I’d have to research this. (EW)
• …? (Your input goes here)

Btw, I’d never thought I’d say this, but without CLR, I feel so lost. ;P

Like I wrote in my last post, a few years out of grad school and a bunch of web applications later, and I’m really, really rusty. In any case, feel free to leave any comments/ideas and I’ll be sure to credit you for your solution!

After reading Leah’s blog (of Pownce fame) about “rounding to the nearest half”, I was immediately struck by two sobering realities:

1. I would need that exact same algorithm for my current site.
2. A few years out of grad school, and I couldn’t even begin to fathom a solution. Thank you, Internet.

In any case, I figured I’d wrap my solution into a template tag that people could use/steal/copy. Here’s a generated test page that illustrates what my template tag would display.

You can even customize the display a bit:

I know, I know… I can do a lot better. In any case, here’s the code snippet that I placed in a file called show_stars.py:

import math
 
from django.template import Library, Node, TemplateSyntaxError, VariableDoesNotExist, resolve_variable
from django.conf import settings
 
register = Library()
 
IMG_TEMPLATE = '<img src="%s" alt="%s"/>'
 
PATH_TO_WHOLE_STAR = IMG_TEMPLATE % (settings.MEDIA_URL + 'img/stars/star.png', "Whole Star")
PATH_TO_THREE_QUARTER_STAR = IMG_TEMPLATE % (settings.MEDIA_URL + 'img/stars/three-quarter.png', "3/4 Star")
PATH_TO_HALF_STAR = IMG_TEMPLATE % (settings.MEDIA_URL + 'img/stars/half.png', "1/2 Star")
PATH_TO_QUARTER_STAR = IMG_TEMPLATE % (settings.MEDIA_URL + 'img/stars/quarter.png', "1/4 Star")
PATH_TO_BLANK_STAR = IMG_TEMPLATE % (settings.MEDIA_URL + 'img/stars/blank.png', "Empty Star")
 
class ShowStarsNode(Node):
    """ Default rounding is to the whole unit """
    def __init__(self, context_var, total_stars, round_to):
        self.context_var = context_var
        self.total_stars = int(total_stars)
        self.round_to = round_to.lower()
 
    def render(self, context):
        try:
            stars = resolve_variable(self.context_var, context)
        except VariableDoesNotExist:
            return ''
 
        if self.round_to == "half":
            stars = round(stars*2)/2
        elif self.round_to == "quarter":
            stars = round(stars*4)/4
        else:
            stars = round(stars)
 
        fraction, integer = math.modf(stars)
        integer = int(integer)
        output = []
 
        for whole_star in range(integer):
            output.append(PATH_TO_WHOLE_STAR)
        if self.round_to == 'half' and fraction == .5:
            output.append(PATH_TO_HALF_STAR)
        elif self.round_to == 'quarter':
            if fraction == .25:
                output.append(PATH_TO_QUARTER_STAR)
            elif fraction == .5:
                output.append(PATH_TO_HALF_STAR)
            elif fraction == .75:
                output.append(PATH_TO_THREE_QUARTER_STAR)
 
        if fraction:
            integer += 1
 
        blanks = int(self.total_stars - integer)
 
        for blank_star in range(blanks):
            output.append(PATH_TO_BLANK_STAR)
 
        return "".join(output)
 
""" show_stars context_var of 5 round to half """
def do_show_stars(parser, token):
    args = token.contents.split()
    if len(args) != 7:
        raise TemplateSyntaxError('%s tag requires exactly six arguments' % args[0])
    if args[2] != 'of':
        raise TemplateSyntaxError("second argument to '%s' tag must be 'of'" % args[0])
    if args[4] != 'round':
        raise TemplateSyntaxError("fourth argument to '%s' tag must be 'round'" % args[0])
    if args[5] != 'to':
        raise TemplateSyntaxError("fourth argument to '%s' tag must be 'to'" % args[0])
    return ShowStarsNode(args[1], args[3], args[6])   
 
register.tag('show_stars', do_show_stars)

I’ve also wrapped up the star graphics I used. A big ups to the Silk Icons creator. For those of you that don’t care for .zip files, here’s all the star images I used:

To use the tag, it’ll look something like this in your templates:

{% load show_stars %}
     {% show_stars context_var of 5 round to half %}
{% endif %}

Note: You can change the following:

1. context_var is whatever var contains the value you want rounded (it was ‘value’ in the screenshots)
2. 5 can be any integer; this is the maximum number of stars to display. I should probably check to make sure you don’t put a number smaller than context_var, but I’ll leave that up to you.
3. half can be any of the following values: whole, half, or quarter
4. Make sure to edit the paths to the star files in the templatetag file

And since this is on my blog, feel totally free to take/use/steal/distribute/copy/modify any code you see fit. All I ask is if you find any bugs, have any comments, or can think of ways to make my ugly code cleaner – I’d love to hear from you.

Enjoy!

Here’s how the PaginatorTag looks compared to the Digg-style tags.

Like the PaginatorTag, this tag is a very basic inclusion tag that builds on the variables already set on the context when paginating with the generic object_list view. There are a few additional context variables created:

• page_numbers – a list of page numbers to display
• in_leading_range – boolean if the page is within the leading range
• in_trailing_range – boolean if the page is within the trailing range
• pages_outside_leading_range – a list of page numbers outside the leading range
• pages_outside_trailing_range – a list of page numbers outside the trailing range

If you don’t understand what these are, don’t worry – I don’t remember either. I could’ve just appended what needed to be displayed in page_numbers, but instead, I broke it out into what needed to be displayed before and after the actual pages so one could customize the code a little more. In any case, without further adieu, here’s the code snippet:

The first thing I did was create a file called digg_paginator.py.

from django import template
 
register = template.Library()
 
LEADING_PAGE_RANGE_DISPLAYED = TRAILING_PAGE_RANGE_DISPLAYED = 10
LEADING_PAGE_RANGE = TRAILING_PAGE_RANGE = 8
NUM_PAGES_OUTSIDE_RANGE = 2 
ADJACENT_PAGES = 4
 
def digg_paginator(context):
    if (context["is_paginated"]):
        " Initialize variables "
        in_leading_range = in_trailing_range = False
        pages_outside_leading_range = pages_outside_trailing_range = range(0)
 
        if (context["pages"] <= LEADING_PAGE_RANGE_DISPLAYED):
            in_leading_range = in_trailing_range = True
            page_numbers = [n for n in range(1, context["pages"] + 1) if n > 0 and n <= context["pages"]]           
        elif (context["page"] <= LEADING_PAGE_RANGE):
            in_leading_range = True
            page_numbers = [n for n in range(1, LEADING_PAGE_RANGE_DISPLAYED + 1) if n > 0 and n <= context["pages"]]
            pages_outside_leading_range = [n + context["pages"] for n in range(0, -NUM_PAGES_OUTSIDE_RANGE, -1)]
        elif (context["page"] > context["pages"] - TRAILING_PAGE_RANGE):
            in_trailing_range = True
            page_numbers = [n for n in range(context["pages"] - TRAILING_PAGE_RANGE_DISPLAYED + 1, context["pages"] + 1) if n > 0 and n <= context["pages"]]
            pages_outside_trailing_range = [n + 1 for n in range(0, NUM_PAGES_OUTSIDE_RANGE)]
        else: 
            page_numbers = [n for n in range(context["page"] - ADJACENT_PAGES, context["page"] + ADJACENT_PAGES + 1) if n > 0 and n <= context["pages"]]
            pages_outside_leading_range = [n + context["pages"] for n in range(0, -NUM_PAGES_OUTSIDE_RANGE, -1)]
            pages_outside_trailing_range = [n + 1 for n in range(0, NUM_PAGES_OUTSIDE_RANGE)]
        return {
            "base_url": context["base_url"],
            "is_paginated": context["is_paginated"],
            "previous": context["previous"],
            "has_previous": context["has_previous"],
            "next": context["next"],
            "has_next": context["has_next"],
            "results_per_page": context["results_per_page"],
            "page": context["page"],
            "pages": context["pages"],
            "page_numbers": page_numbers,
            "in_leading_range" : in_leading_range,
            "in_trailing_range" : in_trailing_range,
            "pages_outside_leading_range": pages_outside_leading_range,
            "pages_outside_trailing_range": pages_outside_trailing_range
        }
 
register.inclusion_tag("digg_paginator.html", takes_context=True)(digg_paginator)

To give a brief explanation of the file:

(LEADING_PAGE_RANGE and TRAILING_PAGE_RANGE is the number of pages before the views switch from the top view to the bottom view).

Also, there is a variable passed in from the view function called base_url which is basically the portion of the url before /page/. Don’t worry, you’ll see what I mean after the template file. Basically, I needed this url because I reconstruct the pagination links to:

http://yoursite.com/model/page/page_number/records/num_records

Next, I created a file called digg_paginator.html.

{% spaceless %}
 
{% if is_paginated %}
<div class="paginator">
{% if has_previous %}<span class="prev"><a href="{{base_url}}/page/{{ previous }}/records/{{results_per_page}}" title="Previous Page">&laquo; Previous</a></span>{% else %}<span class="prev-na">&laquo; Previous</span>{% endif %}
 
{% if not in_leading_range %}
	{% for num in pages_outside_trailing_range %}
		<span class="page"><a href="{{base_url}}/page/{{ num }}/records/{{results_per_page}}" >{{ num }}</a></span>
	{% endfor %}
	...
{% endif %}
 
{% for num in page_numbers %}
  {% ifequal num page %}
    <span class="curr" title="Current Page">{{ num }}</span>
  {% else %}
  	<span class="page"><a href="{{base_url}}/page/{{ num }}/records/{{results_per_page}}" title="Page {{ num }}">{{ num }}</a></span>
  {% endifequal %}
{% endfor %}
 
{% if not in_trailing_range %}
	...
	{% for num in pages_outside_leading_range reversed %}
		<span class="page"><a href="{{base_url}}/page/{{ num }}/records/{{results_per_page}}" >{{ num }}</a></span>
	{% endfor %}
{% endif %}
 
{% if has_next %}<span class="next"><a href="{{base_url}}/page/{{ next }}/records/{{results_per_page}}" title="Next Page">Next &raquo;</a></span>{% else %}<span class="next-na">Next &raquo;</span>{% endif %}
</div> 
{% endif %}
 
{% endspaceless %}

Notice what the next, previous, and page urls look like. You’ll have to make sure your urls.py file reflects this. Here’s a snippet of what my urls.py looks like:

url(r'^browse/all/restaurants/page/(?P<page>[\d]+)/records/(?P<records>[\d]+)/$', restaurant_views.browse_all),

Finally, just so you can have the same Django-like color-scheme, here’s the relevant css.

/** PAGINATOR **/
.paginator { padding:.5em .75em; float:left; font:normal .8em arial; }
 
.paginator .prev-na,
.paginator .next-na {
	padding:.3em;
	font:bold .875em arial;
}
 
.paginator .prev-na,
.paginator .next-na {
	border:1px solid #ccc;
	background-color:#f9f9f9;
	color:#aaa;
	font-weight:normal;
}
 
.paginator .prev a, .paginator .prev a:visited,
.paginator .next a, .paginator .next a:visited {
	border:1px solid #c2ee62;
	background-color:#edfdd0;
	color:#234f32;
	padding:.3em;
	font:bold .875em arial;
}
 
.paginator .prev, .paginator .prev-na { margin-right:.5em; }
.paginator .next, .paginator .next-na { margin-left:.5em; }
 
.paginator .page a, .paginator .page a:visited, .paginator .curr {
	padding:.25em;
	font:normal .875em verdana;
	border:1px solid #C2EE62;
	background-color:#EDFDD0;
	margin:0em .25em;	
	color:#006000;
}
 
.paginator .curr { 
	background-color:#234f32;
	color:#fff;
	border:1px solid #234f32;
	font-weight:bold;
	font-size:1em;
}
 
.paginator .page a:hover,
.paginator .curr a:hover,
.paginator .prev a:hover,
.paginator .next a:hover {
	color:#fff;
	background-color:#234f32;
	border:1px solid #234f32;
}

Btw, I do realize that I didn’t do any checking of the variables in the digg_paginator.py. I’m assuming that the developer won’t put in strange values. (I know, I know – bad assumption). If you wanna see what I’m talking about, try these values:

LEADING_PAGE_RANGE_DISPLAYED = TRAILING_PAGE_RANGE_DISPLAYED = 10
LEADING_PAGE_RANGE = TRAILING_PAGE_RANGE = 4
NUM_PAGES_OUTSIDE_RANGE = 6
ADJACENT_PAGES = 6

Oh, one last thing. To use the tag, it’ll look something like this in your templates:

{% load utils.paginator.digg_paginator %}
{% digg_paginator %}

Oh, and since this is on my blog, feel free to take/use/steal/distribute/copy/modify any code you see fit, but since I threw this together in a few hours, if you find any bugs, have any comments, or think the code can be cleaner, I’d love to hear from you.

Enjoy!

The very first thing I did was search Google. You know, I really don’t know how people lived pre-Googs. In any case, I found this blog posting describing exactly what I wanted to do. Thanks Ben! Since I’m not a big fan of running one large batch script people create in their blogs, I’ll break it down for the non-*nix fans out there.

Before I begin, I’m assuming that you already have Django running on Dreamhost. If you’re having a “wtf” moment, make sure to stop by Jeff’s blog and read “Setting up Django on Dreamhost“. (This is how I set mine up). To follow my short tutorial, you’ll need shell access to your Dreamhost account.

After ssh’ing into your Dreamhost account, you should be in your home directory (/home/username). According to the Filesystem Hierarchy Standard, the /opt dir “is reserved for the installation of add-on application software packages.” With that said, issue the following commands:

$ mkdir opt
$ mkdir downloads
$ cd downloads
$ wget http://www.python.org/ftp/python/2.5.1/Python-2.5.1.tgz
$ tar xvzf Python-2.5.1.tgz

First, create a directory named opt. Next, create a directory named downloads for all your files. Change into the downloads directory, then download the latest Python from http://www.python.org. Finally, unzip and untar the package into the download directory. Everything will extract into a directory named Python-2.5.1.

$ cd Python-2.5.1
$ ./configure --prefix=$HOME/opt/ --enable-unicode=ucs4
$ make
$ make install

Change into the Python-2.5.1 directory and type in the following configure command. Basically, configure prepares your installation for compilation. The –prefix flag will install machine-independent data files in subdirectories of the specified directory. The default is to install in /usr/local, but it’s overwritten with the opt directory created earlier. Finally, run make and make install which will install your custom Python installation.

$ cd ..
$ rm -rf Python-2.5.1

Finally, delete the Python-2.5.1 directory. Before you can use this Python installation, you have to add the /opt/bin directory to your path. To do this, add /opt/bin to your .bash_profile file in your home directory. To do so, you’ll have to add the following to your .bash_profile.

export PATH=$HOME/opt/bin/:$PATH

Basically, this allows you to type ‘python’ in your shell and reach the custom Python 2.5.1 installation instead of the Dreamhost one. To make sure that our Python installation is working, type the following in your home directory (cd ~):

$ source .bash_profile
$ python --version

After the last command, you should see the following: Python 2.5.1. If that displays, your upgrade was successful! After upgrading your Python installation, you’re not done yet. Since Dreamhost uses an old MySQL-Python installation, we’ll upgrade that as well. Type the following in your home directory:

$ cd downloads
$ wget http://internap.dl.sourceforge.net/sourceforge/mysql-python/MySQL-python-1.2.2.tar.gz
$ tar xvzf MySQL-python-1.2.2.tar.gz
$ cd MySQL-python-1.2.2
$ python setup.py install

First, change into the downloads directory and issue the wget command to download the latest MySQL-Python files from Sourceforge. Once you’ve received the files, unzip and untar the package. All the files will extract into a directory called MySQL-python-1.2.2. Change into this directory and install the files by typing python setup.py install. If you followed the custom Python installation, the files should build and extract into the ~/opt/lib/python2.5/site-packages/ directory as an egg file.

You now have a custom Python installation and a MySQL-Python upgrade!

Voila!

Update: Just so you don’t get caught up in the same mistake that I made, to be sure that your Django fcgi installation is using your custom Python installation, make sure the dispatch.fcgi file reads as such:

#!/home/USERNAME_GOES_HERE/opt/bin/python
import sys

sys.path += ['WHATEVER_PATHS_YOU_NEED']

from fcgi import WSGIServer
from django.core.handlers.wsgi import WSGIHandler
import os

os.environ['DJANGO_SETTINGS_MODULE'] = 'wegoeat.settings'
WSGIServer(WSGIHandler()).run()

The very first thing I did was unleash my first Django web application on Dreamhost. Thanks to an excellent tutorial from Jeff Croft, a detailed explanation about FastCGI contained within the Django documentation, and a few helpful pointers on the Dreamhost wiki, I was able to get my application deployed in a matter of a few hours.

You can check it out here!

However, after going through Jeff’s excellent tutorial, I still wasn’t completely satisfied with my Django deployment on Dreamhost. Something was missing. There wasn’t a seamless way to continue development on my home machine, deploy to a test environment, and still keep my live site intact. After all, I’m a true believer in the open source dictum of ‘release early, release often‘, and without a way to test my application on a live server, I wasn’t happy with my configuration management.

Ideally, I envisioned having a live web application (i.e. http://www.wegoeat.com/) and another url that I could deploy my beta releases to (i.e. http://beta.wegoeat.com/). From a configuration management standpoint, I would tag major release builds and to maintain that release over its life (via bug fixes, minor enhancements), I would create a branch of the tag. Thus, the live site would be updated from the branches directory, while the beta url would update from the trunk in my Subversion repository. So to summarize the ‘extra’ steps I did to ensure a smoother deployment cycle, I’ve conjured up the following action list.

1. The very first thing I did was follow Jeff’s tutorial – instead of creating a single directory in my django_projects directory, I created two. One was named ‘project_live’ and the other ‘project_beta’.
2. Next, I checked out the appropriate source files from the appropriate locations in my Subversion repository. The ‘project_live’ directory came from my branches directory and represents my ‘live’ site. The ‘project_beta’ directory came from the trunk and represents my ‘beta’ site. Obviously, the settings.py file for the Django applications as well as the configuration files for FastCGI were different according to the directories. Since my settings will probably be very different then your settings, I’ll leave this as an exercise to the reader.
3. Note, as far as Dreamhost goes, I created two domain entries, one @ http://www.wegoeat.com that will host my live site, and another @ http://beta.wegoeat.com that will be my beta site.
4. I followed my own tutorial and created a post-commit hook to update the appropriate Dreamhost directories when I committed to the repository.

And voila! We’re done.

Now, I can develop on my home machine where I’ve checked out the trunk of my Subversion repository. Whenever I commit, the post-commit hook updates the project_beta directory on my Dreamhost account, and all the while, my live site is still functioning.

Stay tuned for my next blog where I discuss how to get Custom PHP + MediaWiki + EAccelerator playing nicely together on Dreamhost!