Peterbe.com

This is an update on Date formatting in python or in PostgreSQL where the test wasn't done very well. The solution using Python for the formatting created a new DateTime object each time for each formatting because the time_stamp extracted from the database was a string. That would be beneficial to the Python formatting alternative but that's not the whole point. I suspect that the way I did the experiment last time (code is lost by the way) was wrong and didn't focus on the correct benchmark.

In this, my second attempt, I've done a more correct test and tried it on 500 selects. 500 formatted in SQL and 500 formatted in Python. The results are even more outstanding for PostgreSQL than last time.

Here are the results:

times1 (python formatted)
0.113439083099
times2 (sql formatted)
0.00697612762451

That means that doing the date formatting in SQL is 16 times faster!!

Bare in mind that this is optimization and you always have to be careful when doing optimization. For example, the SQL database shouldn't get involved in the presentation and if you need to use a different locale you might to change your application in two places which is risky.

Yesterday I blogged about a simple algorithm for removing a string from a list of string case insensitively. I was happy to see that several people joined in with suggestions just a few hours after I published it. I have now thought about it a bit more and to honour those who commented I've done a little benchmark to find out which one is the best.

What both my own solution and some peoples suggestions forgot was to raise a ValueError if it fails just like this would: list("abc").remove("d")

So, I've tidied up the suggestions and where need be I've added the exception throw. This is not the first time list comprehension in python impresses me. The winner in terms of performance is Andrews list comprehension suggestion. Here are the timeing results:

f1 0.704859256744
f2 1.5358710289
f3 1.37636256218
f4 0.468783378601
f5 0.475452899933
f6 0.666154623032

Often when working with lists of strings in python you might want to deal with the strings in a case insensitive manner. Today I had to fix an issue where I couldn't use somelist.remove(somestring) because the somestring variable might be in there but of a different (case)spelling.

Here was the original code:: def ss(s): return s.lower().strip() if ss(name) in names: foo(name + " was already in 'names'") names.remove(name)

The problem there is that you get an ValueError if the name variable is "peter" and the names variable is ["Peter"]. Here is my solution. Let me know what you think:

def ss(s):
   return s.lower().strip()

def ss_remove(list_, element):
   correct_element = None
   element = ss(element)
   for item in list_:
       if ss(item) == element:
           list_.remove(item)
           break

L = list('ABC')
L.remove('B')
#L.remove('c') # will fail
ss_remove(L, 'c') # will work
print L # prints ['A']

Might there be a better way?

UPDATE Check out Case insensitive list remove call (part II)

I've been playing with PIL's ImageDraw to create images from text. This isn't anything new but I thought I'd combine it with some Web 2.0 technology. The page is marked up like before in valid and accessible XHTML, then a javascript kicks in to automatically replace the plain text headers with image generated ones.

The benefit of this is that the image replacement stuff happens AFTER the page has been loaded for snappier response times. The page looks better with image headlines because you're not font-limited there (see apple.com for example). And most importantly: you want images for headlines but you also want to be found on Google.

Go to the demo page to see it in action.

After reading Matt Harrison's notes about Python at Google I noticed something which I couldn't add up.

"Python programmers at Google must follow a strict style guideline (based on PEP8 with 2 spaced indenting). When engineers are first granted commit access to their SCM system, they must pass a style test."

"based on PEP8" but rejecting such an important part as indentation really is.

From PEP 8 Style Guide for Python Code

"Use 4 spaces per indentation level."

I learnt something useful today which I can't explain. When you use the tempfile module in the Python standard library and the mkstemp() function you get two things back: an integer and a string.:

>>> import tempfile
>>> x, y = tempfile.mkstemp()
>>> x, type(x)
(3, <type 'int'>)
>>> y, type(y)
('/tmp/tmpK19sIx', <type 'str'>)
>>> help(tempfile.mkstemp)

I don't know what to do with the integer so I just ignore it. I thought I would get the result of open("/tmp/tmpK19sIx","w+b").

There's been a lot of debate in the python blog community about which web framework is best for everyone else. There's also been a hot debate about templating languages much fueled by Guido's rejection of XMLish requirements. In the few blogs I've followed there seems to be a couple of finalists that deserve extra scrutiny: Zope3, Django and TurboGears.

I'm drooling over all of them. Zope3 feels like a wise granddad with vigour, Django as a bubblegum-chewing teenager with lots of energy and TurboGears as something in between. (if Twisted was to get an analogy it'd be Einstein or someone like that) I want to use all three or at least Zope3 and one of Django or TurboGears depending on the project. Stuff that needs thinking and will have to last and scale: Zope3. Quick and simple websites about a single topic: Django/TurboGears.

If you're into Zope python product stuff, read on, otherwise don't bother.

Thanks to Brian Lloyd (Zope corp) and Florent Guillaume (Nuxeo) I now know how to set security declarations on a class outside the class. It doesn't work like a normal python class (new or old style) which was a bit of a surprise. This is how you do it in a "normal" python class:

class _Z:
  def __init__(self):
      self.z = "Z"
  def declareProtected(self, *a,**k):
      print "++declare something+"

def foo():
  print "I'm being called"
  return _Z()

class A:
  security=foo()
  def __init__(self):
      pass
A.security.declareProtected("foo")

Had a quick play with Yahoo!'s Inbound Links API today. You use their web services API to check which other URLs a URL is linked to from. This can come in handy if you want to know which other sites make a link to your article. Googleblog is using this (obviously not by using the Yahoo! API); look at this blog post for example and scroll to the end of the text.

The inspiration came from Fredrik Lundh's term extraction example that I'm actually now use in a production site. So I basically took Fredriks code and modified it for Inbound Links.

I need to create a Zope index for a ZCatalog that is KeywordIndex. A KeywordIndex is a list (array if you like) that is used to describe some data. For example, if the data is "Peter is a Swedish Londoner", the the keywords are ("peter", "swedish", "londoner"). What about if the data you want to create an index of is a filename like "NameLog.txt" or "holiday-00412-juli-05.jpg". I've now quickly written a little something that seems to do a decent job. It splits the filenames (these are filenames only and no paths) by caMel structure, dot (.), underscore (_), dash (-) and digits.

If you want to play with my little script, have a look at filenamesplitter.py If you open that script you'll see that it tests a whole bunch of filenames (taken from the Demo issuetracker) and if you want to see what this the result is, here it is: