Peterbe.com

When doing local Django development with runserver you end up doing some changes, then refreshing in Firefox/Chrome/Safari again and again. Doing this means that all your static resources are probably served via Django. Presumably via django.views.static.serve, right? What's wrong with that? Not much, but we can do better.

So, you serve it via Nginx and let Nginx take care of all static resources. You'll still use Django's own runserver so no need for mod_wsgi, gunicorn or uWSGI. This requires that you have Nginx installed and running on your local development environment. First you need to decide on a fake domain name. For example mylittlepony. Edit your /etc/hosts file by adding this line:

127.0.1.1       mylittlepony

Then create the file /etc/nginx/sites-available/mylittlepony and type something like this in it:

server {
   root /home/peterbe/projects/mylittlepony/static;
   server_name mylittlepony;
   gzip            off;

   location = /favicon.ico  {
       rewrite "/favicon.ico" /img/favicon.ico;
   }
   proxy_set_header Host $host;
   location / {
     if (-f $request_filename) {
         add_header X-Static hit;
         access_log   off;
     }

     if (!-f $request_filename) {
         proxy_pass http://127.0.0.1:8000;
         add_header X-Static miss;
     }
   }
}

Then when you've done that enable it:

# cd /etc/nginx/sites-enabled
# ln -s ../sites-available/mylittlepony
# /etc/init.d/nginx reload

Now test the site with curl or something:

$ curl -I http://mylittlepony/ 
HTTP/1.1 200 OK
Server: nginx/0.7.65
Date: Fri, 08 Oct 2010 14:35:04 GMT
Content-Type: text/html; charset=utf-8
Connection: keep-alive
Expires: Fri, 08 Oct 2010 14:35:04 GMT
Vary: Cookie
Last-Modified: Fri, 08 Oct 2010 14:35:04 GMT
ETag: "fecf14808e52fe8652373f6e49e1ac06"
Cache-Control: max-age=0
Set-Cookie: csrftoken=f6eb9e767ca058ecde24bb51c8db9448; Max-Age=31449600; Path=/
Set-Cookie: sessionid=c48af92360ad29410d199081e6067f54; expires=Fri,  
  22-Oct-2010 14:35:04 GMT; Max-Age=1209600; Path=/
X-Static: miss

Now test to get a static resource:

$ curl -I http://mylittlepony/css/jquery-ui-1.8.4.css
HTTP/1.1 200 OK
Server: nginx/0.7.65
Date: Fri, 08 Oct 2010 14:36:31 GMT
Content-Type: text/css
Content-Length: 24806
Last-Modified: Tue, 31 Aug 2010 18:11:38 GMT
Connection: keep-alive
X-Static: hit
Accept-Ranges: bytes

Awesome! If you're curious how much faster Nginx is than Django's serve view, here's a hasty benchmark from my laptop:

# ab -n 10000 -c 10 http://localhost:8000/css/jquery-ui-1.8.4.css
...
Requests per second:    446.34 [#/sec] (mean)
Time per request:       22.405 [ms] (mean)
Time per request:       2.240 [ms] (mean, across all concurrent requests)
Transfer rate:          10893.44 [Kbytes/sec] received
...

And the same directly from Nginx:

# ab -n 10000 -c 10 http://mylittlepony/css/jquery-ui-1.8.4.css
...
Requests per second:    15709.54 [#/sec] (mean)
Time per request:       0.637 [ms] (mean)
Time per request:       0.064 [ms] (mean, across all concurrent requests)
Transfer rate:          384039.88 [Kbytes/sec] received

Obviously you're not going to be able to hit your website that hard but trust me, when you work like this a lot and do a lot of refreshing over and over (working on some Javascript code for example) you can really feel the difference. The static resources load faster because and Django just have to do one single thing which is to create the HTML page. Your stdout on the runserver is only going to log actual views used. Like this:

[08/Oct/2010 15:43:14] "GET / HTTP/1.0" 200 8639
[08/Oct/2010 15:43:27] "GET /crm/clients/ HTTP/1.0" 200 28830
[08/Oct/2010 15:43:30] "GET /crm/clients/A1215/ HTTP/1.0" 200 12804

I hope this helps other people shave milliseconds off their development time.

And by "dangerous" I mean super slow to the point of making your browser shake of over exhaustion.

I have a big fat table where on the left hand side of each row there's a little toggle to open up an initially hidden sub-table. And there are toggles for those sub-tables to open up further sub-tables. It might sound complicated but it works great. The code for each toggle looks something like this:

$('a.toggle-order-on').click(function() {
   var tbody_parent = $(this).parents('tbody');

   $('tr.printdisplay:hidden', tbody_parent).show();
   $('tr.outdoor-marketing:hidden', tbody_parent).show();
   $('tr.digital:hidden', tbody_parent).show();

   // expand the little table too
   $('a.toggle-printdisplay-on', tbody_parent).click();

   var td_parent = $(this).parents('td');
   $(this).hide();
   $('a.toggle-order-off:hidden', td_parent).show();

   return false;
});

Note the heavy use of the super useful :hidden selector which is basically a reversing wrapper on the :visisble selector

What I then needed was a way to open up every single row in the whole table with one click. Here was the code I wrote:

$('a.toggle-order-on:visible').trigger('click');

See? Makes sense does it?

Problem with this was that when the table was big sometimes clicking this would make my otherwise fast browser (Chrome or Firefox) stutter and sometimes stall or at worst the alert pop-up about "a script is slowing this page down" would appear.

So I started the Firebug Profiler and clicked a couple of times and collected some numbers. On average it took 3-4.5 seconds!! and about 20,000-35,000 calls to complete the full expansion. Yikes! About 90% of the time spent by jQuery was on the visisble() function.

Solution: Instead of using the click trigger I simply just called the .show() effect on all things manually without using any :visible or :hidden operators. Here's the new code:

$('tr.printdisplay').show();
$('tr.outdoor-marketing').show();
$('tr.digital').show();
$('tr.printdisplaydetail').show();

$('a.toggle-order-off').show();
$('a.toggle-order-on').hide();

$('a.toggle-printdisplay-off').show();
$('a.toggle-printdisplay-on').hide();

This time, with the profile again, I it took on average 0.2-0.3 seconds and required about 2000-4000 calls. HUGE difference.

So, remember that next time. Don't just re-use working code en mass if it's using a much of :visible or :hidden selectors somewhere in there.

wkhtmltopdf is by far the best tool available to make PDFs. Yes. I have tried ReportLab and PISA. ReportLab might be more powerful but what you gain in fine-grained control you lose in hours and hours of productivity.

Anyway, I've learned something about font-size shrinkage and using wkhtmltopdf. Basically, if use percentage to change a font size (Arial in this case) you get a PDF where the letters are unevenly spaced between. It took me a while to figure out what the hell was going on until I changed the font-size from 90% to exactly 11px.

('font-size:90%'; the spots of red are my highlights of the ugly spacings)

('font-size:11px'; not perfect but much better)

So, at first I thought this was the first time wkhtmltopdf has disappointed me but I guess I'll just have to remember not to use percentages and continue to favor wkhtmltopdf as my choice of weapon in the PDF production world.

I can't believe I haven't seen this before. wikiHow is a great site. Just look at this article about how to spice up waffles for example.

Interestingly though, gone are the days of surfing. People don't take what's offered, they take what the search for. However, with wikiHow I'm eager to pop back in, throw a few articles on my Instapaper and then read them later on my Kindle.

Now I'm going to read about How to Be a Slacker Mom. Very relevant.

I'm brand spanking new to the node.js web application development. The framework I'm currently using is express which seems OK. So I've got an app that consists of 1 static HTML file, a lot of Javscript/CSS/image resources and some express GET and POST views that return small snippets of HTML. All data will be loaded with AJAX to avoid having to use any HTML templating on first load. What's cool about this is that it's soo fast! Everything except the JSON data can be loaded from an Nginx server.

At the moment I've got a light static HTML page that loads about 240Kb of Javascript and CSS (jQuery UI is big) and a couple of bytes of JSON data pulled from Node. As a little anal perfectionism I put an Nginx server in front so that Node doesn't have to serve any of the static files. To get that you have to have a Nginx site enabled that looks like this:

server {
   root /home/peterbe/task-calendar/static;
   location / {
     if (-f $request_filename) {
         add_header X-Static hit;
         access_log   off;
     }
     if (!-f $request_filename) {
         proxy_pass http://127.0.0.1:8000; # where Node is running
         add_header X-Static miss;
     }
   }
}

I think much of the fun of working with this app is that it's a delight to see it load in the browser without any sluggishness or delay. Lovely!

Dawned on me that the Django auth user model that ships with Django is like the string built-in of a high level programming language. With the string built-in it's oh so tempting to add custom functionality to it like a fancy captialization method or some other function that automatically strips whitespace or what not. Yes, I'm looking at you Prototype for example.

By NOT doing that, and leaving it as it is, you automatically manage to Keep It Simple Stupid and your application code makes sense to the next developer who joins your project.

I'm not a smart programmer but I'm a smart developer in that I'm good at keeping things pure and simple. It means I can't show off any fancy generators, monads or metaclasses but it does mean that fellow coders who follow my steps can more quickly hit the ground running.

My colleagues and I now have more than ten Django projects that rely on, without overriding, the django.contrib.auth.models.User class and there has been many times where I've been tempted to use it as a base class or something instead but in retrospect I'm wholeheartedly happy I didn't. The benefit isn't technical; it's a matter of teamwork and holistic productivity.