Comments on: Using the extra() QuerySet modifier in Django for WeGoEat http://blog.localkinegrinds.com/2007/10/24/using-the-extra-queryset-modifier-in-django-for-wegoeat/ Everything you've ever thought, but never had the balls to say. Fri, 19 Mar 2010 00:53:14 -1000 http://wordpress.org/?v=2.9.2 hourly 1 By: ryankanno http://blog.localkinegrinds.com/2007/10/24/using-the-extra-queryset-modifier-in-django-for-wegoeat/comment-page-1/#comment-8785 ryankanno Fri, 26 Oct 2007 01:16:37 +0000 http://blog.localkinegrinds.com/2007/10/24/using-the-extra-queryset-modifier-in-django-for-wegoeat/#comment-8785 @Matt - Actually, last night I worked on denormalizing the tables and using a post.save signal, whenever a Review is added, the Restaurant instance associated saves itself which in turn updates a few columns, one being called "average_review" (just as Ross/Jeff suggested). Thanks for the comment though - I love all the help the community provides; especially for us techies in remote places. :) @Matt –

Actually, last night I worked on denormalizing the tables and using a post.save signal, whenever a Review is added, the Restaurant instance associated saves itself which in turn updates a few columns, one being called “average_review” (just as Ross/Jeff suggested). Thanks for the comment though – I love all the help the community provides; especially for us techies in remote places. :)

]]> By: Matt http://blog.localkinegrinds.com/2007/10/24/using-the-extra-queryset-modifier-in-django-for-wegoeat/comment-page-1/#comment-8783 Matt Fri, 26 Oct 2007 00:14:34 +0000 http://blog.localkinegrinds.com/2007/10/24/using-the-extra-queryset-modifier-in-django-for-wegoeat/#comment-8783 Not sure that you want to be doing an AVG and/or COUNT on every request - might be better (at a certain point) to start using summary tables. Thanks for the post, though! Not sure that you want to be doing an AVG and/or COUNT on every request – might be better (at a certain point) to start using summary tables. Thanks for the post, though!

]]> By: Trent http://blog.localkinegrinds.com/2007/10/24/using-the-extra-queryset-modifier-in-django-for-wegoeat/comment-page-1/#comment-8743 Trent Thu, 25 Oct 2007 13:09:14 +0000 http://blog.localkinegrinds.com/2007/10/24/using-the-extra-queryset-modifier-in-django-for-wegoeat/#comment-8743 Your website is very niceI just wish I could pick it apart and find all the little faults in it that you never thought about. Like you do to my things. But I can't Your website is very niceI just wish I could pick it apart and find all the little faults in it that you never thought about. Like you do to my things. But I can’t

]]> By: Empty http://blog.localkinegrinds.com/2007/10/24/using-the-extra-queryset-modifier-in-django-for-wegoeat/comment-page-1/#comment-8717 Empty Thu, 25 Oct 2007 04:19:01 +0000 http://blog.localkinegrinds.com/2007/10/24/using-the-extra-queryset-modifier-in-django-for-wegoeat/#comment-8717 Nice post. Extra is really useful but sometimes takes some work to get it to do what you want. I agree with Ross, I would likely denomalize this, especially if I expected to generate any significant traffic. Nice post. Extra is really useful but sometimes takes some work to get it to do what you want. I agree with Ross, I would likely denomalize this, especially if I expected to generate any significant traffic.

]]> By: Ross Poulton http://blog.localkinegrinds.com/2007/10/24/using-the-extra-queryset-modifier-in-django-for-wegoeat/comment-page-1/#comment-8716 Ross Poulton Thu, 25 Oct 2007 04:01:57 +0000 http://blog.localkinegrinds.com/2007/10/24/using-the-extra-queryset-modifier-in-django-for-wegoeat/#comment-8716 :) I can't take all the credit for it, the idea first came into my head when I saw it in the <a href='http://www2.jeffcroft.com/blog/2007/may/28/lost-theoriescom-source-code-update/' rel="nofollow">Lost Theories source code that Jeff Croft made available</a>. It works very well, and it's the way I've handle sorting by # comments and ratings on <a href='http://www.djangosites.org/latest/' rel="nofollow">Django Sites</a>. :)

I can’t take all the credit for it, the idea first came into my head when I saw it in the Lost Theories source code that Jeff Croft made available.

It works very well, and it’s the way I’ve handle sorting by # comments and ratings on Django Sites.

]]> By: ryankanno http://blog.localkinegrinds.com/2007/10/24/using-the-extra-queryset-modifier-in-django-for-wegoeat/comment-page-1/#comment-8712 ryankanno Thu, 25 Oct 2007 00:43:03 +0000 http://blog.localkinegrinds.com/2007/10/24/using-the-extra-queryset-modifier-in-django-for-wegoeat/#comment-8712 @Ross - By far the best suggestion yet! :) I hadn't even begun to think of denormalizing the database at all! <strike>+1</strike> +100 for Ross. :P I'll be working on the signal tonight. Gosh, community rocks! @Ross -

By far the best suggestion yet! :) I hadn’t even begun to think of denormalizing the database at all! +1 +100 for Ross. :P I’ll be working on the signal tonight. Gosh, community rocks!

]]> By: Ross Poulton http://blog.localkinegrinds.com/2007/10/24/using-the-extra-queryset-modifier-in-django-for-wegoeat/comment-page-1/#comment-8711 Ross Poulton Thu, 25 Oct 2007 00:08:04 +0000 http://blog.localkinegrinds.com/2007/10/24/using-the-extra-queryset-modifier-in-django-for-wegoeat/#comment-8711 That's a neat use of the extra() function; however I believe this may be a case for database de-normalisation. The most efficient way of doing this is probably to add an 'avg_rating' field to your Restaurant model, and use a signal so that whenever a review is saved, you re-calculate the average rating for that Restaurant and save it directly on the model. There is a huge upside to this de-normalised method: You can use sort_by() or limit() to only show restaurants that have a rating above 4, or sort them by rating. Lastly, it should produce more portable code, as subselects may or may not work the same way on different database servers. That’s a neat use of the extra() function; however I believe this may be a case for database de-normalisation. The most efficient way of doing this is probably to add an ‘avg_rating’ field to your Restaurant model, and use a signal so that whenever a review is saved, you re-calculate the average rating for that Restaurant and save it directly on the model.

There is a huge upside to this de-normalised method: You can use sort_by() or limit() to only show restaurants that have a rating above 4, or sort them by rating.

Lastly, it should produce more portable code, as subselects may or may not work the same way on different database servers.

]]> By: ryankanno http://blog.localkinegrinds.com/2007/10/24/using-the-extra-queryset-modifier-in-django-for-wegoeat/comment-page-1/#comment-8710 ryankanno Wed, 24 Oct 2007 23:31:23 +0000 http://blog.localkinegrinds.com/2007/10/24/using-the-extra-queryset-modifier-in-django-for-wegoeat/#comment-8710 I definitely agree that the SQL makes subselects for every row. Since I'm no database guru, I'll profile it under SQL analyzer to see how bad/optimized it is. Yet, I don't think Django is the problem here at all. If anything, it's me as the developer, and if you were trying to be nice about it... Thanks! :) If I knew the optimized SQL to write, I would. ;) Maybe the optimal thing to do is to perform the query to retrieve the objects, then perform the average call, then merge the data QuerySet's together... :P I definitely agree that the SQL makes subselects for every row.

Since I’m no database guru, I’ll profile it under SQL analyzer to see how bad/optimized it is. Yet, I don’t think Django is the problem here at all. If anything, it’s me as the developer, and if you were trying to be nice about it… Thanks! :) If I knew the optimized SQL to write, I would. ;)

Maybe the optimal thing to do is to perform the query to retrieve the objects, then perform the average call, then merge the data QuerySet’s together… :P

]]> By: Julian http://blog.localkinegrinds.com/2007/10/24/using-the-extra-queryset-modifier-in-django-for-wegoeat/comment-page-1/#comment-8709 Julian Wed, 24 Oct 2007 22:30:36 +0000 http://blog.localkinegrinds.com/2007/10/24/using-the-extra-queryset-modifier-in-django-for-wegoeat/#comment-8709 It is definitely an improvement. But showing it as a good solution and blog about it is not. This SQL still makes subselects, for every single row. The real problem is Django in this case, we should focus on that, instead of praising workarounds which circumvent real SQL solutions. It is definitely an improvement. But showing it as a good solution and blog about it is not. This SQL still makes subselects, for every single row. The real problem is Django in this case, we should focus on that, instead of praising workarounds which circumvent real SQL solutions.

]]> By: ryankanno http://blog.localkinegrinds.com/2007/10/24/using-the-extra-queryset-modifier-in-django-for-wegoeat/comment-page-1/#comment-8706 ryankanno Wed, 24 Oct 2007 20:22:33 +0000 http://blog.localkinegrinds.com/2007/10/24/using-the-extra-queryset-modifier-in-django-for-wegoeat/#comment-8706 @Benjamin: I think that at the end of the day, this is an improvement (at least performance wise) - but I can see how it is arguably so. I agree that the SQL is being executed with the subquery, but I think you have to look at where the query is being optimized. In the subquery extra() clause, the database is handling the optimization for you; ie we're letting the database do what it's good at. As to how optimized the MySQL query analyzer is for its B-tree is up for debate (I'm no database expert). I think letting the database handle this versus making "N" number of MySQL AVG() calls (while retrieving connections, etc) is a definite improvement performance-wise. However, as I said earlier, I think it can be debated since this type of SQL injection obviously leads to maintenance hell in large projects. :) But then again, since I'm no database guru, this could all be total bs. :P As for your join and AVG() function call, maybe someone can pitch in and set <strike>us</strike> me straight. :) <strong>Update:</strong> @Benjamin: Read my previous comment as I think your comment was where the templatetag is going. @Benjamin:

I think that at the end of the day, this is an improvement (at least performance wise) – but I can see how it is arguably so. I agree that the SQL is being executed with the subquery, but I think you have to look at where the query is being optimized.

In the subquery extra() clause, the database is handling the optimization for you; ie we’re letting the database do what it’s good at. As to how optimized the MySQL query analyzer is for its B-tree is up for debate (I’m no database expert). I think letting the database handle this versus making “N” number of MySQL AVG() calls (while retrieving connections, etc) is a definite improvement performance-wise.

However, as I said earlier, I think it can be debated since this type of SQL injection obviously leads to maintenance hell in large projects. :) But then again, since I’m no database guru, this could all be total bs. :P As for your join and AVG() function call, maybe someone can pitch in and set us me straight. :)

Update: @Benjamin: Read my previous comment as I think your comment was where the templatetag is going.

]]>