Peterbe.com

In 2001 I started my first and perhaps most successful Open Source project I've ever made: IssueTrackerProduct. After nearly a decade of maintaining it I have now officially abandoned it.

It all started when I needed a way to track feedback on my personal website. That's why it was originally called "SiteTrackerProduct". I needed something where I could collect bug reports and any other pieces of feedback and then process it in some structured fashion. It was therefore very important that it would be possible to run the application open for anonymous access. People should be able to submit bugs and issues without having to create an account. You see, kids, back in that day it was actually very common that sites would force users to register and create accounts even just because the content owner wanted it. These days, it's common knowledge that to get people to open up and share anything for others to benefit you make it absolutely trivial to jump straight in without having to see a registration page that looks like a tax return form.

Now, since I long ago abandoned the Zope2 application server technology stack and I no longer use IssueTrackerProduct for anything real it's no longer feasible to maintain this project. In the last five years or so we were actually using it actively to track all projects at Fry-IT where I used to work. I have to say, even though we did grow out of it, it was actually successful. It handled the load (after some much needed patches towards optimization) and it was easy for people to actually use since unlike many other bug trackers, it focused on the non-technical end user first and foremost. As much as possible was done to make it trivial to type in your bug or issue and it automatically took care of all notifications and access rights.

Being a personal Open Source project, over the years, it became a melting pot for experimenting and perfecting various new ideas. Many of them we take for granted today but back then it was quite novel if I may say so. This includes:

• ability to auto-save unfinished form inputs (added before Gmail had it)
• automatic updates of the content without reload made it possible to see other people participating as you're typing
• ability to reply directly to email notifications without having to open a web browser
• an advanced via-the-web programmable interface for adding and modifying custom fields (e.g. "Customer reference code")
• full-text search combined with ability to search on specific fields by key
• file attachments that are images automatically appear as little thumbnails
• file attachments that have text become searchable (e.g Word documents)
• advanced filtering where you can easily decide to search inclusive or exclusive on certain fields
• persistent filtering automatically saved and share'able between different users
• programmable search filters that are coded in Python which made it possible to create very specific reports
• ability to export and import bugs from and to Excel for offline processing

Writing all of this, I can not resist to get a bit nostalgic. I did sink A LOT of time into this project. Today when I look back at the code and almost feel sick seeing all the mistakes that I made. Much of the ugliness of the code can be attributed partially to the fact that I often used and abused the code to add new features. Also, because we often needed some features (since it was used to manage all of our projects) "yesterday" and then it was hard to justify doing things "properly". For example, the main .py file is over 14,000 lines of code!

I did called it "perhaps most successful Open Source project I've ever made" in the first sentence. The reason for that is that over the years many many people have downloaded it and installed and let it be used by thousands of users. That's something to be proud of.

Anyway! It's time to move on. So long and thank you for all the fish!

The code is still available at github.com/peterbe/IssueTrackerProduct

I just got back from the Helpdesk & IT Support Show in Olympia (Kensington, London). My main impression is: there are many, big players in this industry.

My pet project, the IssueTrackerProduct is very basic in comparison to some of these companies products. Although it's often used in help desk situations the kind of help desk solutions I've seen today are way different. For many of them, it's all about integrating various systems such as asset management, call logging, configuration management, knowledge management, etc. It seems that the actual help desk apps seems to have to be low priority compared to getting all pieces to fit together.

The IssueTrackerProduct was on tuesday this week featured on Ajaxian.com which is a really good looking site that focuses on nice usage of AJAX in websites.

They could have chosen a better screenshot I guess but other than that I'm quite happy to be included.

A week ago I ran some ad hoc benchmarks on various suspect functions in the IssueTrackerProduct and came to a clear and simple conclusion: searching is the bottleneck and within the search it's the searching for file attachments that take all of the time.

If you're interested and open minded, here's the results of that benchmark This sparked some thoughts. First I wrote a filename splitter which isn't rocket science but I'm proud to say that it's use is brilliant. Before, the find-by-file function in the IssueTrackerProduct used a plain old find() test like this:

filename.find('foo') > -1

This is very fast but not very intelligent. For example it'll with match on foobar.txt and plainfooter.gif. So, what I did instead was to create a KeywordIndex and index all the splitted filenames in that index.

I think there's a lot of potential in screencasting. As far as I've understood, screencasting is when you make a movie recoding in some manner of what happens on the computer screen. To test this I downloaded Camtasia Studio 3 and as a demo I created a new Issue Tracker instance on www.issuetrackerproduct.com. The next time I do this I'll make sure I plan what I want to do instead of just making it up after I've started. On this windows computer that I tried this it lagged so incredibly much that it was to move the mouse because it didn't move smoothly. Perhaps there are some further options to free up some resources to make it run better.

If you want to see the result (as a Flash movie) follow this link (1.2Mb)

I've made a quick graph showing how the releases of the IssueTrackerProduct increases steadily in size with every new version. Since the first release, 10 months ago, the release package has more than doubled in size. Much of that is due to the new templates that have been added and some icons.

Is this a positive trend? Yes of course! A lot of the new code isn't just additional fancy features. Sometimes it takes a couple of extra bytes just dealing with stuff under the hood that has no impact whatsoever on the interface. The actual numbers aren't really a measure that can be used, but what is really important to notice is the solid and steady growth. This year has been very busy for me with work but I've always tried to squeeze in a bit of open source work too in the mornings, evenings and lunch breaks. I wish I could focus more on the IssueTrackerProduct but so far we haven't come up with a direct way of getting rich on it so it still remains just a "hobby" for me an my company.

A rather impressive yet useless dependency graph of IssueTracker.py I created this graph simply by downloading py2depgraph.py and depgraph2dot.py from this website

The result is a big ass picture with little boxes that describe each little module that is connected to IssueTracker.py in some way. This image becomes rather useless to me because it digs down into python libs that the code never really goes near. To make this genuinely useful one would have to intercept the .dot file and remove references to libs that aren't interesting.

To me, AJAX (Asyncrounous Javascript And XML) patterns are only interesting if they work as a bonus rather than a must. I've written before about autosaving web forms whereby a form with a big textarea is autosaved on the server every 8 seconds. That feature took existing form functionality and used it in Javascript instead of user actions.

Now I've done it again (actually it was a couple of days ago but I've had time to write about it until now). When you on the IssueTrackerProduct, list issues you'll see a little button that makes it possible to enable "filter options". If you press that button it sets off a GET request to the server to re-request the same page but this time with ShowFilterOptions=true as a parameter. Here's some simplified code:

<div id="filteroptions">
  <form action="ListIssues">
    <input type="hidden" name="ShowFilterOptions" value="1" />
    <input type="submit" value="Show filter options" />
  </form>
</div>

How can we load the filter options widget on the List Issue page without having to refresh the whole page?

I'm on the lookout for a redesign of the IssueTrackerProduct. Not necessarily a complete redesign but its css and general looks need a serious upgrade. Sure it's built with web standards which means that all that is needed is to change some CSS, but in reality it's harder than that. I think I want to keep the general layout except that when you view a particular issue and I no longer want to center the issue and the followups and I want to have the options such as Subscription and Tell-a-friend in a right-hand menu.

The problem is that of inspiration. All good design I see on Zen Garden is for content that is very different from the issuetracker. For example, the issuetrackerproduct design must bare to do both the issueview and the List Issues which requires the whole screen. It just seems that all good design is made for web blogs which is very far from the issuetracker. I want to see design that is using up 100% of the screensize yet manages to not make the textwidth unreadably wide. Some of these narrow designs can be very nice looking but that's not what I'm after now.