Peterbe.com

I need a mini calculator in my web app so that people can enter basic mathematical expressions instead of having to work it out themselfs and then enter the result in the input box. I want them to be able to enter "3*2" or "110/3" without having to do the math first. I want this to work like a pocket calculator such that 110/3 returns a 36.6666666667 and not 36 like pure Python arithmetic would. Here's the solution which works but works like Python:

def safe_eval(expr, symbols={}):
   return eval(expr, dict(__builtins__=None), symbols)

def calc(expr):
   return safe_eval(expr, vars(math))

assert calc('3*2')==6
assert calc('12.12 + 3.75 - 10*0.5')==10.87
assert calc('110/3')==36

I recently wrote a Grok application that implements a REST API for Atom Publishing so that I can connect a website I have via my new Nokia phone has LifeBlog which uses the Atom API to talk to the server.

Anyway, the authentication on Atom is WSSE (good introduction article) which basically works like this:

PasswordDigest = Base64 \ (SHA1 (Nonce + CreationTimestamp + Password))

This is one of the pieces in a request header called Authorization which can look something like this:

Authorization: WSSE profile="UsernameToken"
X-WSSE: UsernameToken Username="bob", PasswordDigest="quR/EWLAV4xLf9Zqyw4pDmfV9OY=", 
Nonce="d36e316282959a9ed4c89851497a717f", Created="2003-12-15T14:43:07Z"

What I did was I wrote a simple Python script to mimic what the Nokia does but from a script. The script creates a password digest using these python modules: sha, binascii and base64 and then fires off a POST request. Here's thing, if you generate this header with base64.encodestring(ascii_string) you get something like this:

quR/EWLAV4xLf9Zqyw4pDmfV9OY=\n

Notice the extra newline character at the end of the base64 encoded string. This is perfectly valid and is decoded easily with base64.decodestring(base64_string) by the Grok app. Everything was working fine when I tried posting to http://localhost:8080/++rest++atompub/snapatom and my application successfully authenticated the dummy user. I was happy.

Then I set this up properly on atom.someotherdomain.com which was managed by Apache who internally rewrote the URL to a Grok on localhost:8080. The problem now was that the Authentication header value was broken into two lines because of the newline character and then the whole request was rejected by Apache because some header values came without a : semi-colon.

The solution was to not use base64.encodestring() and base64.decodestring() but to instead use base64.urlsafe_b64encode() and base64.urlsafe_b64decode(). Let me show you:

>>> import base64
>>> x = 'Peter'
>>> base64.encodestring(x)
'UGV0ZXI=\n'
>>> base64.urlsafe_b64encode(x)
'UGV0ZXI='
>>> base64.decodestring(base64.urlsafe_b64encode(x))
'Peter'

If you're still reading, then hopefully you won't make the same mistake as I did and wasting time on trying to debug Apache. The lesson learned from this is to use the URL safe base64 header values and not the usual ones.

Geopy is a great little Python library for working with geocoding and distances using various online services such as Google's geocoder API.

Today I spent nearly half an hour trying to debug what was going on with my web application since I was getting this strange error:

AttributeError: 'VincentyDistance' object has no attribute '_kilometers'

I've been struggling hard this morning to get Jeditable to work in IE (6 and 7). Whilst everything was working perfectly fine in Firefox, in IE the clickable editable text would just disappear and never return. The solution was to use the latest jQuery 1.2.1. I was using version 1.1.4 which was why it didn't work.

Jeditable is a brilliant plugin with really good configuration options (hint read the source code's documentation comment) and I'll now send an email to Mika about this pitfall and suggest that he includes it in his documentation.

Unlike previous incarnations of Spellcorrector not it does not by default load the two huge language files for English and Swedish. Alternatively/additionally you can load your own language file. The difference between loading a language file and training on your own words is that trained words are always assumed to be correct.

Another major change with this release is that a pickle file is created once the language file or own training file has been parsed once. This works like a cache, if the original text file changes, the pickle file is recreated. The outcome of this is that the first time you create a Spellcorrector instance it takes a few seconds if the language files is large but on the second time it takes virtually no time at all.

I've recently improved the IssueTrackerProduct so that when you start to write in the little textarea it expands and grows vertically as the text gets larger and larger. Other sites like Highrise do this too for note taking.

Long story short, here's the demo and here's the solution:

function _getNoLines(element) {
  var hardlines = element.value.split('\n');
  var total = hardlines.length;
  for (var i=0, len=hardlines.length; i<len; i++) {
     total += Math.max(Math.round(hardlines[i].length / element.cols), 1) - 1;
  }
  return total;
}

$(function() {

  // First, for all the textareas that have lots of lines of text 
  // in them, we want to double their number of rows
  $('textarea.autoexpanding').each(function() {
     while (_getNoLines(this) > parseInt(this.rows))
       this.rows = '' + Math.round((parseInt(this.rows) * 1.5));
  });

  // When a user enters new lines, if they have entered more
  // lines than the textarea has rows, then double the textareas rows
  $('textarea.autoexpanding').bind('keyup', function() {
     if (_getNoLines(this) > parseInt(this.rows))
       this.rows = '' + Math.round((parseInt(this.rows) * 1.5));
  });

}

From the doc string:

A very spartan attempt of a script that converts HTML to
plaintext.

The original use for this little script was when I send HTML emails out I also
wanted to send a plaintext version of the HTML email as multipart. Instead of 
having two methods for generating the text I decided to focus on the HTML part
first and foremost (considering that a large majority of people don't have a 
problem with HTML emails) and make the fallback (plaintext) created on the fly.

This little script takes a chunk of HTML and strips out everything except the
<body> (or an elemeny ID) and inside that chunk it makes certain conversions 
such as replacing all hyperlinks with footnotes where the URL is shown at the
bottom of the text instead. <strong>words</strong> are converted to *words* 
and it does a fair attempt of getting the linebreaks right.

As a last resort, it strips away all other tags left that couldn't be gracefully
replaced with a plaintext equivalent.
Thanks for Fredrik Lundh's unescape() function things like:
   'Terms &amp;amp; Conditions' is converted to
   'Termss &amp; Conditions'

It's far from perfect but a good start. It works for me for now.

Version at the time of writing this: 0.1.

I wouldn't be surprised if I've reinvented the wheel here but I did plenty of searches and couldn't really find anything like this.

Let's run this for a while until I stumble across some bugs or other inconsistencies which I haven't quite done yet. The one thing I'm really unhappy about is the way I extract the body from the BeautifulSoup parse object. I really couldn't find another better way in the few minutes I had to spare on this.

Feel free to comment on things you think are pressing bugs.

You can download the script here html2plaintext.py version 0.1

UPDATE

I should take a second look at Aaron Swartz's html2text.py script the next time I work on this. His script seems a lot more mature and Aaron is brilliant Python developer.

Like many other Python fellow geeks on Planet Python I too took the Which Programming Language Are You? quiz. Apparently I'm Prolog.

I've never used Prolog and I barely know how it works or what it's syntax looks like. Well, I guess I'll just erase all my current projects and recode them in Prolog from now on. Unpractical but necessary.

I think a lot of Python people have seen Peter Novig's beautiful article about How to Write a Spelling Corrector. So have I and couldn't wait to write my own little version of it to fit my needs.

The changes I added were:

• Python 2.4 compatible
• Uses a pickleable dict instead of a collection
• Compiled a huge list of Swedish words
• Skipped edit distances 2 of words longer than 10 characters
• Added a function suggestions()
• All Unicode instead
• A class instead of a function
• Ability to train on your own words and to save that training persistently

I knew had I heard it before but I must have completely missed it anyway and forgotten to test my new Javascript widget in IE. None of the stylesheet worked in IE and it didn't make any sense. Here's how I did it first:

var closer = document.createElement('a');
a.setAttribute('style', 'float:left; font-weight:bold');
a.onclick = function() { ...

That worked in Firefox of course but not in IE. The reason is that apparently IE doesn't support this. This brilliant page says that IE is "incomplete" on setAttribute(). Microsoft sucked again! Let's now focus on the workaround I put in place.

First I created a function to would take "font-weight:bold;..." as input and convert that to "element.style.fontWeight='bold'" etc:

function rzCC(s){
  // thanks http://www.ruzee.com/blog/2006/07/\
  // retrieving-css-styles-via-javascript/
  for(var exp=/-([a-z])/; 
      exp.test(s); 
      s=s.replace(exp,RegExp.$1.toUpperCase()));
  return s;
}

function _setStyle(element, declaration) {
  if (declaration.charAt(declaration.length-1)==';')
    declaration = declaration.slice(0, -1);
  var k, v;
  var splitted = declaration.split(';');
  for (var i=0, len=splitted.length; i<len; i++) {
     k = rzCC(splitted[i].split(':')[0]);
     v = splitted[i].split(':')[1];
     eval("element.style."+k+"='"+v+"'");

  }
}

I hate having to use eval() but I couldn't think of another way of doing it. Anybody?

Anyhow, now using it is done like this:

var closer = document.createElement('a');
//a.setAttribute('style', 'float:left; font-weight:bold');
_setStyle(a, 'float:left; font-weight:bold');
a.onclick = function() { ...

and it works in IE!