Peterbe.com

This took me a while to grok and perhaps by mentioning it here, it'll prevent other people from making the same mistake as I did and perhaps preventing myself from doing the same mistake again.

In the ZCatalog, when you set up indexes you can give them a name and an index attribute. If you omit the index attribute, it'll try to hook into the objects by the name of the index. For example, if you set the index to be title with no indexed attribute it'll fetch the title attribute of the objects it catalogs. But if you set the indexed attribute to be something like idx_getTitle you can do something like this in your class:

def idx_getTitle(self):
   """ return title as we want it to be available in the ZCatalog """
   return re.sub('<*.?>','', self.title)

The same can not be done with indexes of type DateIndex.

Together with my colleague Jan we today tried to fix a problem with Zope ZEO server that wouldn't start. Very strange we thought since another ZEO server on the same machine was working fine. We compared zope.conf files and found nothing obvious. Here was the error you get with 'bin/runzeo':

root@da-ovz-vm99182:/var/lib/zope-2.8.9/aragdb-zeo1# ./bin/runzeo 
Error: error opening file //var/lib/zope-2.8.9/aragdb-zeo1/etc/zeo.conf: 
[Errno ftp error] no host given

What?? ftp error??

Long story short, the error was this line in 'runzeo':

CONFIG_FILE="/${INSTANCE_HOME}/etc/zeo.conf"

which should have been this:

CONFIG_FILE="${INSTANCE_HOME}/etc/zeo.conf"

Not so easy to spot because the CONFIG_FILE line is something you rarely look closely at.

I saw this today on the Zope mailing list:

TypeError: open() takes at least 1946389116 arguments (4 given)

For anybody who's ever used Python you'll find this funny.

To set the Expires header in my web application I used to use the Zope DateTime function rfc822() which doesn't return the date in GMT format. Here's how I used it:

>>> from DateTime import DateTime
>>> hours = 5
>>> then = DateTime() + hours/24.0
>>> then.rfc822()
'Thu, 16 Aug 2007 20:43:59 +0100'

Then I found out (from using YSlow) that it's better to use the GMT format (RFC 1123), and here's how to do that in Zope:

>>> from App.Common import rfc1123_date
>>> from time import time
>>> rfc1123_date(time() + 3600*hours)
'Thu, 16 Aug 2007 19:45:12 GMT'

(notice that even though my locale is here in London, because of the summer time an hour is added)

There are several ways of serving static files from Zope. The simplest way is to just do something like this:

size = os.stat(file_path)[stat.ST_SIZE]
REQUEST.RESPONSE.setHeader('Content-Type','image/jpeg')
REQUEST.RESPONSE.setHeader('Content-length',int(size))
return open(file_path, 'rb').read()

The disadvantage with this is if the file is say >1Mb, the whole 1Mb needs to be loaded into RAM before it can be sent. This is surely garbage collected afterwards but if you serve many of these at the same time, there's obviously the risk that the RAM gets too full.

The alternative is to use filestream_iterator from the ZPublisher which is an iterator that sends out chunks of the file at a time. Note that to use the filestream_iterator you must set the Content-Length header first. Here's how to use it:

from ZPublisher.Iterators import filestream_iterator
...
size = os.stat(file_path)[stat.ST_SIZE]
REQUEST.RESPONSE.setHeader('Content-Type','image/jpeg')
REQUEST.RESPONSE.setHeader('Content-length',int(size))
return filestream_iterator(file_path, 'rb')

This week we improved MOBi with a neat little feature that makes it possible to decide what the SMS response should be to inbound SMS on your own server.

To test this you need to be UK resident and willing to part with the 25p it costs to receive the response. Send peterbe is NN (where NN is a number, how old you think I am) to 83211 and await a result. I've set up the sub word "is *" to forward all inbound SMS to an address on www.peterbe.com and here on this server I run the following Python Script:

##parameters=sender, message
##
guess = message.split()[-1]
try:
    age = int(guess)
except ValueError:
    return "That's not a number :-("

if age > 27:
    return "That's not it! It's less than that. Try again"
elif age < 27:
    return "That's not it! It's more than that. Try again"
return "Wow! You know my age!!"

Today I bumped into a nasty challenge when trying to copy a Zope image from ZODB to the filesystem. The reason why it wasn't easy was that Zope splits the file up into chunks if it's upload size is more than 65Kb (2^16 bytes) which I guess it does for being able to "stream" the file back into the browser when viewing the image instead of sending one large blob.

So, this didn't work:

>>> imgobj.meta_type
Image
>>> open(temp_file_path,'wb').write(imgobj.data)

...if the data is big because sometimes imgobj.data is a string (the binary data) and sometimes is a ImplicitAcquirerWrapper that you have to iterate over.

I've wanted to Gzip the static content (CSS & Javascript) on my sites for a long time but never found a good enough solution. mod_gzip is out of the question because as far as I've understood it it does a compression on the fly, every time you request the file.

Other solutions have disappointed me because enabling gzip compression has been for all content. I don't want my HTML files gzipped because they're rendered on the fly based on business logic plus by compressing the HTML files. Long story short my yet-to-be-released app now serves the following files from Zope but are only compressed and whitespace slimmed once per server restart:

FILE               ORIG SIZE  NEW SIZE   REDUCTION
screen.css             15224      2738   556%
print.css               2633       885   298%
jquery-latest.js*      57712     18130   318%
jquery-latest.pack.js  20461     10513   195%
common.js               3803      1131   336%
complete-expense.js    18184      2847   639%
Total                 118017     36244   326%

* only used in debug mode

I often use the Zope management interface but not to do much ZMI like stuff. Just simple things like creating new base objects, changing properties or testing a change to a template. For some reason I seem to lack the ability to spell the word manage_main. I can't remember how many times I've accidently typed manage-main , mange_main, manag_main, manage)_main etc.

There's got to be an end to that! Hence the MUnderscorePatch monkey patch. Now, all I have to do is to type in m_ at the end of the URL and it becomes the same as manage_main. Ie. I go to http://localhost:8080/ then I append m_ so that it becomes http://localhost:8080/m_ and it redirects automatically to http://localhost:8080/manage_main

Silly but actually quite useful for me. Want to download it too?

Here are a few lessons learnt when sending HTML emails in Zope with unicode data. I'm still a beginner with this stuff and still probably have a lot to learn. What I've achived now works but might break with different encodings or on different systems so don't assume that these patterns will work in your setup.

To the send the email I'm using the default MailHost that comes with Zope 2.8.5. I call it's send() passing the subject line a second time because the subject line is already in the body string.

The most valuable piece of magic to learn and remember from this is that when you construct the multi-part message you have to attach the plain text before the html text. Thank you Lukasz Lakomy for that tip!