Ryan Kanno: The diary of an Enginerd in Hawaii

Since I actually used this method to reduce the number of Update:”explicit” SQL calls made in WeGoEat, I figured I’d write a little blog explaining the context in which it was used, and maybe, just maybe, it’ll help shed some light on how others can take advantage of this neat little function.

Background

As a Django “proof-of-concept”, I’m working on a local restaurant review site for my home state of Hawai`i. (I actually just released it yesterday). For each restaurant, I want to be able to calculate the average of all reviews and display this listing in a paginated view. (Yes, I do realize there’s no average rating, but that has to do with there being no users. ;P).

The Problem

Having a serious “wtf was I thinking moment”, I initially wrote a Restaurant model function that returned the average (review) rating for each restaurant instance. Little did I realize that when I actually displayed the restaurant’s average reviews, I would be making an additional SQL avg() call for every restaurant. Though I’m paging “n” records at a time, this function added an additional “n” SQL calls for every view that contained a restaurant listing, just to name a few.

In pseudo-code, my initial naive function resembled the following: (I’m sure we’re all guilty of writing something of the sort… ok, fine, I know I was. ;P)

1
2
3
4
5
6

     def get_average_review(self):
         query = 'QUERY TO GET AVERAGE (SELECT AVG(rating)...); (I have the query below)'
         # Get cursor from connection
        cursor = connection.cursor()
        cursor.execute(query)
        return cursor.fetchall()

Duh.

Here’s a picture of the number of queries it took:

The “extra()” solution

After profiling my application and realizing what a bone-headed mistake I made, I began researching the extra() Queryset modifier. Yes, I realize that these extra lookups aren’t the most portable and often violate the DRY principle, but it’ll probably suffice for most of all my personal projects.

Since I’m already retrieving a list of Restaurants and filtering them via letter, island, and what not, I figured I could add an average rating subquery. The entire call looks as such:

1
2
3
4
5
6
7

     restaurants = Restaurant.objects.filter(name__istartswith = letter).extra(
             select={'<strong>avg_rating</strong>': 'SELECT AVG(overall_rating) FROM restaurants_restaurant as res, reviews_review, django_content_type \
                                          WHERE restaurants_restaurant.id = res.id \
                                          AND res.id = reviews_review.object_id \
                                          AND reviews_review.content_type_id = django_content_type.id \
                                          AND django_content_type.model = \'restaurant\''},
                       )

As you can see, I’m exploiting the fact that restaurants_restaurant will be available from the Restaurant.objects.filter() call. (I know, I know… bad for portability).

But voila!

Now, in my templates, when I iterate over the restaurants, I can get issue the following:

1
2
3
4
5
6
7
8
9
10
11

{% for restaurant in restaurant_list %}
<tr>
    <td><a href="{{restaurant.get_absolute_url}}">{{ restaurant.name }}</a></td>
    <td>{% if restaurant.avg_rating %}
	   {% load show_stars %} 
           <span class="average-rating">
	   {% show_stars <strong>restaurant.avg_rating</strong> of 5 round to quarter %}
           &lt;/span&gt;
           {% endif %}&lt;/td&gt;
&lt;/tr&gt;
{% endfor %}

Notice how I used my show_stars template tag that I blogged about a few weeks ago to display the average restaurant rating. (Cheap shameless plug, but damn effective! :P) I’d link to a page in action, but since I just opened up my site to a few select users, I’ll update this post when I actually have any reviews.

Oh, and before I forget, thanks to my co-worker Stephen for assisting me with my SQL issues!

Here’s a picture of the final result:

Note:

As an added bonus, I also realized a few other ’spots’ where the .extra() Queryset modifier would come in handy. Since I’m also using the wonderful django-voting application from Jonathan Buchanan, I came across this post about accessing a dictionary via a template in the Django-users Google Group.

Basically, I had come across the same issue as the poster. Since I allow users to vote on reviews (similar to Amazon, Yelp, etc.), I wanted to retrieve the score of each Review instance to display on a paginated listing of all Reviews. Using the same extra() modifier, I was able to inject the total number of votes and the score when I retrieved all Reviews as such:

Btw, I just injected most of the code from Jonathan’s template tag.

1
2
3
4
5
6
7
8
9
10
11

.extra(select={'total_votes': 'SELECT COUNT(vote) FROM votes as v, reviews_review as rev, django_content_type \
                                        WHERE reviews_review.id = rev.id \
                                        AND v.object_id = reviews_review.id \
                                        AND v.content_type_id = django_content_type.id \
                                        AND django_content_type.model = \'review\'', 
 
                                        'score': 'SELECT SUM(vote) FROM votes as v, reviews_review as rev, django_content_type \
                                        WHERE reviews_review.id = rev.id \
                                        AND v.object_id = reviews_review.id \
                                        AND v.content_type_id = django_content_type.id \
                                        AND django_content_type.model = \'review\''},)

Pretty neat right?

Now, when iterating through the reviews, I can use the following:

1
2
3
4
5
6
7
8
9
10
11
12
13

{% for review in object_list %}
	<tr>
		<td><a href="{{review.content_object.get_absolute_url}}">{{ review.content_object.name }}</a></td>
		<td><a href="{% url profile-detail username=review.user.username %}">{{ review.user.username }}</a></td>
		<td><nobr>{% load show_stars %}
			<span class="rating">{% show_stars review.overall_rating of 5 round to half %}</span>
			</nobr>
		</td>
		<td>"<span style="font-weight:bold; color:#092e20;">{{ review.get_recommendation_display }}</span>"</td>
		<td><span style="font-size:.875em;">{{ review.submit_date|timesince }} ago</span></td>
		<strong>&lt;td&gt;Total of {{ review.score|default:0 }} from {{ review.total_votes }} {{  review.total_votes|pluralize:"person,people" }}.&lt;/td&gt;</strong>
	&lt;/tr&gt;
{% endfor %}

Hope y’all learned something like I did! Oh, and before I forget my standard disclaimer, “since this is on my blog, feel free to take/use/steal/distribute/copy/modify any code you see fit, but if you find any bugs, have any comments, or think the code can be cleaner, I’d love to hear from you.”

Enjoy!

Since I haven’t put out a technical article in a while, this blog will explain how I’ve setup Dreamhost + Django + Subversion to play nicely together in a seamless development environment via a shared hosting provider. Hopefully - someone, somewhere can find this information useful and insightful in their own development environment.

The very first thing I did was unleash my first Django web application on Dreamhost. Thanks to an excellent tutorial from Jeff Croft, a detailed explanation about FastCGI contained within the Django documentation, and a few helpful pointers on the Dreamhost wiki, I was able to get my application deployed in a matter of a few hours.

You can check it out here!

However, after going through Jeff’s excellent tutorial, I still wasn’t completely satisfied with my Django deployment on Dreamhost. Something was missing. There wasn’t a seamless way to continue development on my home machine, deploy to a test environment, and still keep my live site intact. After all, I’m a true believer in the open source dictum of ‘release early, release often‘, and without a way to test my application on a live server, I wasn’t happy with my configuration management.

Ideally, I envisioned having a live web application (i.e. http://www.wegoeat.com/) and another url that I could deploy my beta releases to (i.e. http://beta.wegoeat.com/). From a configuration management standpoint, I would tag major release builds and to maintain that release over its life (via bug fixes, minor enhancements), I would create a branch of the tag. Thus, the live site would be updated from the branches directory, while the beta url would update from the trunk in my Subversion repository. So to summarize the ‘extra’ steps I did to ensure a smoother deployment cycle, I’ve conjured up the following action list.

1. The very first thing I did was follow Jeff’s tutorial - instead of creating a single directory in my django_projects directory, I created two. One was named ‘project_live’ and the other ‘project_beta’.
2. Next, I checked out the appropriate source files from the appropriate locations in my Subversion repository. The ‘project_live’ directory came from my branches directory and represents my ‘live’ site. The ‘project_beta’ directory came from the trunk and represents my ‘beta’ site. Obviously, the settings.py file for the Django applications as well as the configuration files for FastCGI were different according to the directories. Since my settings will probably be very different then your settings, I’ll leave this as an exercise to the reader.
3. Note, as far as Dreamhost goes, I created two domain entries, one @ http://www.wegoeat.com that will host my live site, and another @ http://beta.wegoeat.com that will be my beta site.
4. I followed my own tutorial and created a post-commit hook to update the appropriate Dreamhost directories when I committed to the repository.

And voila! We’re done.

Now, I can develop on my home machine where I’ve checked out the trunk of my Subversion repository. Whenever I commit, the post-commit hook updates the project_beta directory on my Dreamhost account, and all the while, my live site is still functioning.

Stay tuned for my next blog where I discuss how to get Custom PHP + MediaWiki + EAccelerator playing nicely together on Dreamhost!