Some of you may have noticed the Twitter feed on the right. This is a brand new, albeit late in early adopter terms, addition to my online life. For the longest time, I really wondered what the whole point of Twitter was. People just went on and on about the phenomenon and I never quite got it. Why on earth would anyone be interested in being notified about what I am up to, every moment of my life?

Well, I finally figured out how Twitter applies to me. I will tweet every time I have something to report on when I am programming or dealing with technical issues, pretty much the same theme as this blog. None of the personal, “I’m driving to work”, kinds of tweets, only stuff that may prove useful for someone out there. The feed on the right will help keep everything I report, posts and tweets alike, in one place.

The reason I blog, and also tweet, is not to address a captive audience on a common topic, but just to post searchable information to the web for anyone who is encountering similar situations. It gives me the satisfaction that I am contributing to the community in some way. Majority of the hits on this blog are from Google and other search engines convincing me that reality aligns with my intentions. The effort is well worth it.

It is much easier to tweet than to blog since the message size is so much smaller. Being the kind of writer who obsesses over every sentence twenty times before posting, it takes me a long time to get posts out. It also discourages me from posting often enough. Twitter will provide the avenue to report frequently, since I am always up to something but seldom have the time to type out detailed posts.

While long consolidated posts on this blog will continue to appear, there will be several smaller issues reported in tweets, as and when things surface. As a result, a lot more of my experiences will get logged unlike on the blog where only big things make it, several days/weeks later. It will also give you a glimpse into what goes on in between all these blog posts.

In my previous post, I mentioned how I migrated back to Vim for my development environment. Vim is fantastic but the default settings may not suit your needs. Given that Vim does not provide a GUI based configuration mechanism, getting it to fit your needs will take a bit of tweaking .VIM files and scavenging Google for tips. This was part of the reason why I stuck to Eclipse for so long.

Anyway, I have finally customized Vim enough to use it full time. Below is a list of my customizations; perhaps it will be of use to you.

No Cream with that

Cream is a really popular Vim customization. It demonstrates the flexibility of Vim with its extensive enhancements, convenient packaging and GUI based settings management. If only Vim packaged everything the way Cream does, customization would be so much easier.

I did try Cream prior to attempting my own customization. However, Cream does away with one very essential feature of Vim: Insert Mode. This is done with the spirit to make Vim easier for beginners. Cream does provide a trap door to access this mode but it is very limited and frustrating. After struggling with this for a few days, I got fed up with Cream and decided to start with Vim as a base and customize it to my needs instead of adjusting to Cream’s vision.

Installing Vim

I ended up using the base Vim installer packaged and distributed by Cream. You can find the latest release here. You can also use AppSnap to install Vim for you. It will be much faster if nothing else.

Vim will install by default to C:\Program Files\Vim. There will be a _vimrc in that directory with all the binaries in a sub-directory. All settings in this _vimrc will get loaded by Vim on startup so all the following customizations will go there.

GUI customizations

The GUI allows for customization of various aspects with a simple setting. I have opted for the menu bar and tab bar and disabled all other enhancements. The following lines need to be added to _vimrc in order to achieve this:

set guioptions=em
set showtabline=2

Setting ‘e’ in guioptions ensures that the tab bar is displayed by the GUI. Setting showtabline to 2 further ensures that the tab bar is always displayed, even if there is only one file open. Setting ‘m’ in guioptions ensures that the menu bar is displayed.

Refer to the Vim documentation for guioptions to see all the available options.

Tab navigation like Firefox

I am very comfortable with the tab navigation shortcuts in Firefox so it only made sense to do the same within Vim. This can be done with the following settings:

:nmap <C-S-tab> :tabprevious<cr>
:nmap <C-tab> :tabnext<cr>
:imap <C-S-tab> <ESC>:tabprevious<cr>i
:imap <C-tab> <ESC>:tabnext<cr>i
:nmap <C-t> :tabnew<cr>
:imap <C-t> <ESC>:tabnew<cr>i
:map <C-w> :tabclose<cr>

This maps Ctrl-Tab and Ctrl-Shift-Tab to navigate between tabs, Ctrl-T to create a new tab and Ctrl-W to close the current tab.

Custom Font

Sometime in the past, I looked for a decent programming font and found ProggyClean which, in my opinion, is very readable and pleasant. I’ve used ProggyClean with Eclipse for several years so Vim had to do the same. You can download and install the TTF fonts from this link. There are many variations of ProggyClean but I just stick to the base version.

The next thing that needs to be done is to get Vim to use this font by default. This can be done with the following line in _vimrc:

set gfn=ProggyCleanTT:h12:cANSI

ProggyClean is designed for and looks best at size 12. By the way, the default font in Vim is FixedSys so if you prefer that then you don’t need to change anything. The above syntax can also be used to set any other font of your preference as the default.

Color Scheme

When I was playing with Cream, I came across its Terminal color scheme. Since I really liked this scheme, I incorporated it into my customization. You can get it from here. Save it to C:\Program Files\Vim\vim71\colors\terminal.vim and add the following line in _vimrc:

colorscheme terminal

Vim (and Cream) ship with several other color schemes so if you prefer another one, you can set it as default as above instead.

Cream Statusline

In the above screenshot, you can see a status bar at the bottom. I copied that from Cream. Majority of the code was in one file but it used several common functions from other parts of Cream. I copied all those dependencies into a single file and got it to work. You can get the file from the .ZIP file further down in this post.

You could also re-create it based on the latest versions by copying cream-statusline.vim to the Vim plugin directory. Every time you startup Vim, it will complain with unresolved dependencies. Look for those functions in the Cream source and copy them one by one. Eventually, you will have a functioning status bar.

Maximize on startup

I like my windows to be maximized, especially my text editor, so I needed Vim to maximize on startup. This can be done easily with the following Vim script. Just download the .ZIP, extract the contents to the C:\Program Files\Vim\vim71\plugin directory and Vim will start maximizing on startup. It couldn’t get any easier.

Python Tabs

I code in Python extensively so I had to make sure that Vim interpreted tabs correctly. The settings below ensure that tabs are replaced by spaces and map to four spaces.

set tabstop=4
set shiftwidth=4
set expandtab
set softtabstop=4
set autoindent

SuperTab
No one can deny the convenience of tab completion. Vim has completion built in (Ctrl-P) but SuperTab is much prettier and cooler. This Vim script can be downloaded from here. Again, extract the .ZIP to the Vim plugin directory and the tab key will automatically invoke SuperTab.

Function taglist

It is not a feature I used very much in Eclipse, but a taglist helps to quickly get to a specific function in a source file. Vim Taglist provides this feature by leveraging Ctags. Just download the .ZIP for Vim Taglist and extract to the Vim plugin directory. Next, download Ctags for Windows and extract ctags.exe to C:\Program Files\Vim\vim71.

Next, add the following lines to _vimrc:

let Tlist_Sort_Type = “name”
:nmap <F3> :TlistToggle<cr>

The next time you are editing code, hit F3 and a taglist will show up. Clicking any item in the list will move the cursor to that item in the source. Press F3 again and the taglist will disappear. You can change the shortcut by changing the F3 mapping above.

File Explorer

Another script I have incorporated is a simple file explorer – VTreeExplorer. It is not quite as pretty as Eclipse but is very functional. I do not use this very much and prefer using Windows Explorer. However, it is worth having since it is quickly accessible within Vim. You can download it here. Extract the .ZIP to the plugin directory and add the following lines to _vimrc.

let treeExplVertical=1
let treeExplWinSize=35
let treeExplDirSort=1
let treeExplNoList=1
:nmap <F2> :VSTreeExplore<cr>

I have mapped F2 to open the explorer which can be changed per your convenience.

Miscellaneous customizations

Some other customizations I have are as follows:

set nobackup
set number
set nuw=6

The nobackup option prevents Vim from creating backup files. Otherwise, everytime a file changes, Vim will create a .BAK file with the original contents. I find this annoying since I have to manually delete these files.

The number option enables line numbers, an important piece of information for a programmer. The nuw option sets the line number width to 6 which makes it a little neater.

File Associations

The Cream installer does not add the ‘Edit with Vim’ right-click option, something I used all the time earlier. In addition, I wanted to open all files associated with Vim in a tab in an existing Vim session. All this can be achieved with the following registry changes. Just copy them to a .REG file and double-click the file.

Windows Registry Editor Version 5.00

[HKEY_CLASSES_ROOT\*\shell\Edit with Vim]
@=”"

[HKEY_CLASSES_ROOT\*\shell\Edit with Vim\command]
@=”\”C:\\Program Files\\vim\\vim71\\gvim.exe\” -p –remote-tab-silent \”%1\” \”%*\”"

[HKEY_CLASSES_ROOT\Applications\gvim.exe\shell\open\command]
@=”\”C:\\Program Files\\vim\\vim71\\gvim.exe\” -p –remote-tab-silent \”%1\” \”%*\”"

Summary

The post above goes through a bunch of very common Vim customizations. You can get all of them by downloading this .ZIP file and extracting it to your Vim directory. You can then double click the vim.reg file and set up what is described in the file associations section above. In my case, these simple changes made Vim a lot more usable and convenient. The real power of Vim is in knowing the advanced functionality that makes people rage wars.

Recently, I tossed out Eclipse for a simpler Vim + TortoiseSVN development environment. I’ve used Eclipse for several years and love the integration. However, the 30 second startup time has always discouraged me from starting it up too often. Not to mention the 100+ MB installation size, excessive memory usage and tedious setup and update process. Eclipse rocks and I miss the environment every day, but I need something faster and more nimble that can adapt to my 10 minute coding sessions.

I evaluated a whole bunch of potentially lighter tools but nothing fit me like I wanted. I’ve always been a Vim fan so that ended up being the only way to go. After spending a day customizing it to my needs (another pain-ridden process that had discouraged me in the first place) I have a setup that I can take anywhere with me. I’ll write about my Vim configuration in the near future. Now that I had my editor, I needed an SVN client and TortoiseSVN was a no brainer in that department.

TortoiseSVN has every feature that I need including integration with Araxis Merge, something I missed with Eclipse. But for whatever reason, I was unable to get it to display the “Diff” option when I right clicked on a folder. Now, I like to check my diffs a whole bunch of times before I deliver anything so multiply that with several file changes across multiple directories and it soon gets too tedious. Right clicking every single file, one at a time to see the changes I had made was just not acceptable.

Why TortoiseSVN was not displaying the diff option made no sense but I found an alternative. Just make the following registry changes and you are good to go. You will get a “TortoiseDiff” right-click option and that will bring up the directory diff. The disadvantage is that every folder will display this menu option but at least the option is available.

Windows Registry Editor Version 5.00

[HKEY_CLASSES_ROOT\Folder\shell\TortoiseDiff]
@=”"

[HKEY_CLASSES_ROOT\Folder\shell\TortoiseDiff\command]
@=”\”C:\\Program Files\\TortoiseSVN\\bin\\TortoiseProc.exe\” /command:diff /path:\”%1\”"

Select the code above and save it to a text file called tortoisediff.reg. Make sure the path to TortoiseProc.exe is correct. Then double click the file and it will update your registry. As usual, make these changes at your own risk.

For more TortoiseSVN command line options, check out this page. Turns out you can automate a lot of stuff pretty easily.

Some of you may have noticed that this blog, and my wife’s blog, are now hosted on WordPress.com. I moved since I want to stop depending on my server at home for my web hosting needs. The migration was not too difficult since I was using WordPress for the blogs anyway, but was not as smooth as I hoped.

Where did my pages go?

Since I was using an older version of WordPress that didn’t support exporting blog data, I had two options:-

• Install a plugin that would export the blog content
• Upgrade to the newer version that supported exporting natively

Since it was easier to install the plugin, I tried that first. The export worked fine but when I imported the data into WordPress.com, my pages didn’t show up. They were imported alright, but for some reason, they weren’t visible, either on the web site or in the admin area.

Thinking there was something wrong with the plugin, I tried the second option. I upgraded to version 2.2.3 and created a second export file. Importing that to WordPress.com didn’t help either. The pages were detected in the import but were still invisible.

After some Googling, I gave up and manually re-created every page by copy/pasting the content from my old blog. I don’t have too many pages on my blog so it wasn’t too bad. However, all the original page comments were lost since it wasn’t possible to link them to the new pages.

There’s a bug somewhere, either in the export or import step. Regardless, it was very unfortunate to lose some really nice comments.

What about media?

Blog Images

My blog doesn’t have too many images, just a couple here and there in some posts and application pages. My wife’s blog, though, is a food blog, so there’s tons of images. Since I have a Flickr account, I just posted all the images there and linked to them. It was a little time consuming to download each image, upload to Flickr and then update each affected post, but at least it was easy to take care of.

Music

My music pages link to a whole bunch of songs that I’ve composed. All the source files (Impulse Tracker) were already on Scene.org, the awesome non-profit that has supported the electronic art scene for several years. The MP3 files, though, were still hosted on my server all these years since I didn’t feel like taking up the space on Scene.org. So I finally had to go ahead and upload the MP3 files as well. They add up to 600MB so it’s not too bad. A big thanks to the folks that support Scene.org for their generosity.

What I had to remove though were links to audio CD images for the albums I released which add up to 4.5GB. I don’t think there is a sufficient demand for them anyway. People seem satisfied with MP3 files.

Applications

Almost all my applications were already hosted on Google Code, in terms of source code and binaries. Mshare was not, so I created a project and migrated it there. I also removed all links to my server so now, none of my application pages link to my server anymore.

Pros and Cons

Moving to WordPress.com has been a mixed blessing. There’s a long list of benefits:-

• The site is much faster now
• Feature upgrades will magically appear, I have to do nothing
• I don’t need to worry about security, WordPress IT guys have to
• It is completely free, no investments (like electricity when self-hosting)
• Large community so odds of random people visiting is higher

The things that bother me are the following:-

• I lost my page comments
• They charge $10/year for CNAME redirection, I’m using URL forwarding which isn’t as good
• Because of URL forwarding, I have no idea who my referrers are anymore, everybody seems to come from http://blog.genotrance.com
• They could delete my blog for any random reason
• They don’t allow me to use Google Analytics, my history there is now useless
• I can’t advertise on my blog if I ever decide to

Conclusion

Usually, people go the other way. They start a blog on some popular blog site and when it is successful, they look for independence. For now, I want freedom from self-hosting so I think this is a good move. When one of the cons I mentioned become unacceptable, I’ll probably buy some web space and move there. Until then, this is a great bargain and the most sensible thing to do for small time bloggers like me.

We live in a world of wasted effort. I get my daily dose of email spam despite some pretty decent filtering. And ever since I started this blog a few months back, I gotten to experience the joys of comment spam first hand.

Like any new blog, I started out without comment moderation but after a few weeks, the amount of spam became unbearable and I couldn’t keep up with deleting them. While moderating comments is much better than an open site, it is still tedious and boring to clean up the moderation queue. Few people post legitimate comments which hide in the haystack of spam. For the longest time, I put up with it since I didn’t have the time to research the issue.

Lately, having gotten fed up with the spam and the efforts involved in moderation, I decided to give Akismet a try. I had avoided Akismet for the longest time since I wasn’t keen on registering on WordPress.com. One more login to deal with…

Well, I’m glad I activated Akismet. It’s working very well and has been 100% accurate so far. I can be a lot more confident when I go through the moderation queue. And I no longer need to see the spam in my inbox since Akismet does such a good job.

I still have to look for legitimate comments in the Akismet queue but the odds of a false positive seem low so far. Long story short, turn on Akismet and have at least one line of defence against spam.

I recently shared my solution for encoding home videos I record using a DVD camcorder. While the solution helped me understand the encoding process, it was not exactly user friendly in terms of the number of steps involved. Being the automation freak that I am, I have written a Python script that takes care of the entire process in a single step.

The Main Issues

The main issues with the original solution were as follows:-

Copying the VOB files to your hard disk

Copying an entire DVD to your hard disk is no joke. It takes up a lot of space and is time consuming and boring. This had to be done since VOBMerge didn’t like to operate on VOB files directly on the DVD.

Using DVD Decrypter for corrupt VOBs

Corrupt VOBs are the biggest pain. First of all, you have to download DVD Decrypter to solve this problem. Second of all, it is a trial and error method where you had to calculate which megabyte was failing and uncheck the corresponding cell and try again, only to find another broken cell.

In addition, it would take out major chunks of data from your VOB per cell so if you had a tiny scratch, you potentially lost a few megs of data instead of a few 100K. If you had multiple corrupt VOBs, good luck maintaining high spirits.

Using VOBMerge to merge related VOBS

We have already been forced to copy the VOBs to the hard drive for VOBMerge to work. Now, VOBMerge will happily double the data on our hard drive through the merging process.

Running two batch files on each merged VOB

The merged VOB now has to be processed by two different batch files, one to create a downloadable version of the video and the other for archival purposes. These steps could take a lot of time so you could expect to lose valuable CPU time while you decided to take a walk.

Repeat for each unique video

Okay, so we got done with the first video. Now we have to repeat the merging and encoding all over again for every other video. Let me also mention the time spent deleting and copying VOB files around in between encodes in case you don’t have enough space to accommodate the entire DVD, merged VOBs and the final encoded output all at once.

The Solution

Instead of dealing with so many external utilities, I just whipped together a Python script that does all of the above in one step. All you need to do is decide how the seperate VOBs need to be merged, what the output videos need to be named and an acceptable buffer size in case some VOBs are corrupt.

Below are the command line options of vobenc.py.

Usage:
  -j joblist         : List of jobs to execute
  -f jobfile         : File containing list of jobs to execute
  -s steplist        : List of step ids from config.ini to execute on each job
                       [default: all steps]
                       E.g. -s 1,2,3
  -p pathtovobfiles  : Location of the VOB files
                       [default: current directory]
  -d destinationpath : Location where converted files should be stored
                       [default: current directory]
  -c configfile      : Name of configuration file to load steps from
                       [default: config.ini]
  -b buffersize      : Number of bytes to copy at a time
                       [default: 102400 = 100K]

Job List Syntax:
  -j filename1=id1,id2:filename2=id3,id4

  filename           : Name of the output file
  id1,id2,...        : VTS_x_y.VOB files where the id is x

  E.g. -j outfile1=1,2,3:outfile2=4,5:outfile3=6

Job File Syntax:
  -f jobfilename

  E.g.
    outfile1=1,2,3
    outfile2=4,5
    outfile3=6

The above goes into various concepts which are further explained below:-

Jobs

Jobs allow us to specify all the output videos that need to be encoded as well as associate the output videos with the input VOB files. They can be specified on the command line using -j or in a file using -f. One or two jobs look fine on the command line but if you have more, it might be easier to create a job file and pass it onto vobenc.py.

Jobs automate the task of merging VOB files as well as running through every step for each output video.

Steps

In my original solution, there were essentially three steps that each video went through. The first and second steps were the first and second pass encoding of the video for high quality archival. The third pass was to encode the downloadable version. These steps line up with the concept of steps in vobenc.py.

Below is the corresponding config.ini step file for what was proposed in the original solution. A downloadable version is further below.

[step1]
name = High Quality Pass 1
command = mencoder - -o nul -oac mp3lame -lameopts br=128:cbr:aq=0 -ovc xvid -xvidencopts bitrate=1150:vhq=4:pass=1 -msglevel all=5

[step2]
name = High Quality Pass 2
command = mencoder - -o #OUTFILE#-hi.avi -oac mp3lame -lameopts br=128:cbr:aq=0 -ovc xvid -xvidencopts bitrate=1150:vhq=4:pass=2 -msglevel all=5

[step3]
name = Low Quality
command = mencoder - -o #OUTFILE#-lo.mpg -oac lavc -srate 24000 -ovc lavc -lavcopts acodec=mp3:abitrate=24:vcodec=wmv2:vbitrate=96 -vf scale=264:180 -ofps 15 -msglevel all=1

Using the above format, we could have any number of steps. Also note the #OUTFILE# directive that allows us to pass on the output file specified in the job to the encoding utility.

The way config.ini is laid out, we could pass the VOB input to any utility that would handle the VOB data from STDIN. In the above example, we are using mencoder. We could just as well use ffmpeg or any other command line encoder we prefer.

Steps allow us to automate the various steps that each individual video needs to go through in order to produce the output files.

Buffer Size

Vobenc.py uses a default buffer size of 100K. This means that it attempts to read 100K of data from the DVD at a time. On a 1.4GB DVD containing 74 minutes of video, 100K is approximately 1/3rd of a second of information and is a fair compromise.

When a buffer read fails on the DVD, the entire buffer is skipped and the next buffer is read. This allows us to skip corrupt sections of the VOB without having to manually decide what to do.

If the buffer is too large, this will result in more data loss than required. On the other hand, a very small buffer size will affect read speed since smaller chunks will be skipped on corrupt VOBs leading to more IO errors and timeouts. As a result, the buffer size is changeable by the user depending on the state of the DVD. A DVD which is relatively error free could use a larger buffer size, speeding up the read whereas a DVD with a fair amount of damage could use a smaller buffer size to squeeze out as much information as is possible.

Note, however, that very large buffer sizes are not very useful since the main bottleneck is the actual encoding process. The encoder can only handle so many megs of data at a time.

Vobenc.py displays the amount of data loss per VOB at the end of the encoding process.

Download

Vobenc.py is being distributed as a Python script. You will need the Python interpreter installed on your system in order to run it.

• Vobenc 1.0: vobenc.py config.ini

Summary

I recently used vobenc.py to encode an entire 1.4GB DVD. It took an entire night to get through the encoding but the entire task was seamless and automated. All I had to do was create a job file and that took all of five minutes.

I’ll be maintaining vobenc.py on this blog post for now. Feel free to provide your feedback.

I just found out that AppSnap was reviewed on Lifehacker. My web server apparently took a beating and was down. Considering I was travelling for most of today, I just got to fixing it. Hopefully, it fares better now that I’ve tweaked some settings.

I’m a regular reader and fan of Lifehacker and am very happy to see AppSnap on their front page. Thanks to Gina Trapani and the Lifehacker gang for their show of support. The extra eyes and feedback will surely help make AppSnap more useful to people out there.

I upgraded to Firefox 2.0 recently and was impressed like most other folks. Things were fine until I started noticing excessive hard disk activity as well as sluggish performance and intermittant hangs.

Task Manager showed Firefox using ridiculous amounts of memory, more than 300MB with several open tabs in multiple windows. Considering my wife and I use separate windows for our work, quitting and restarting every day is not convenient since Firefox only restores the session of the window that was closed last.

A few searches on Google led to the following page on the Mozillazine knowledge base; basically, a list of add-ons known to cause all sorts of trouble for Firefox. Comparing my list of installed add-ons with the page led to the following short list of potential culprits:-

• Official Google Toolbar : causes memory leaks and excessive disk activity
• IE Tab : leaks memory

I was only using Google Toolbar to auto-fill my forms so it wasn’t hard to uninstall it. As for IE Tab, I installed the recommended alternative IE View Lite which is a sufficient solution. Having IE embedded into Firefox is a nice to have but not essential.

From that point on, Firefox memory usage has stayed under 100MB for the most part. After several days of usage spanning multiple tabs, windows and websites, it gets to 140MB which is pretty reasonable compared to the original 300MB.

Here’s what my add-on list looks like now:

There’s more tips for reducing memory on this Mozillazine page if you are interested in further improvements.

Recently, I ran into a problem with an image gallery web application I am writing in PHP. Throughout development, which was done on my Windows laptop using XAMPP, everything worked perfectly fine, both on Firefox as well as Internet Explorer.

As soon as I deployed my application on my Linux server, IE suddenly stopped accepting my session cookies. The weird thing was that Firefox continued to work perfectly fine.

Symptoms

My application makes multiple AJAX requests to render each page. Usually, a single session would get created and each subsequent request would reuse that session. Instead, each request was creating a new session. My application didn’t work at all since session variables were no longer accessible. On the server side, the PHP session directory was getting flooded with session files.

I didn’t see any cookies for my application in IE’s cache directory. For every request that IE was making, PHP was creating a new session and IE was rejecting the session cookie returned. Hence, there was no session cookie to send back to the server for the next request and PHP was assigning a new session for each request.

Considering everything worked fine on Firefox but not on IE, one would think it was some bug in my application. But it worked fine in IE when running on the Windows laptop. So there was probably nothing wrong with my application.

Possible Solutions

Google turned up a couple relevant results after a lot of searching. Here’s what I found:-

P3P issue

As per this website, IE 6 had a new feature that would reject sessions in certain circumstances unless a specific header was sent clarifying the intentions of the web appliction. This seemed probable so I gave it a try.

I added the following to the top of my application so that every call would return this HTTP header:-

It didn’t work so it was something else.

Session IDs transported by URL

This website claimed that the latest IE update increased the security levels and that applications had no choice but to hard code the session IDs in the URLs. This can be done by enabling the following option in php.ini:-

Of course, this meant that every URL needed to have the session ID added to it. It didn’t feel like the right option since Microsoft wouldn’t break web applications so badly. Considering how many URLs each application has and how many applications are out there, it would be prohibitive to have to change them all to include a session ID.

Timezone issue

A third website suggested that IE was calculating session cookie timeouts incorrectly such that they seemed to expire in the past. As a result, these already expired cookies were rejected immediately. For example, if the server was in Hawaii and the client in Australia and the server requested a session timeout of one hour, the timeout would have already occurred as far as the client in Australia was concerned.

Firefox didn’t have this issue since it converts both the server as well as the client time to UTC and then calculates the timeout. As interesting as this was, this didn’t seem as the problem since both my laptop client as well as my server were in the same timezone.

Final Solution

The timezone issue did give me a hint to check the time on my client and server. My client is a Windows laptop which had the correct time and timezone thanks to being synchronized with the NTP protocol. My server on the other hand was out of sync.

The timezone was correct, but the UTC time was set to my local time instead. As a result, the server was actually six hours in the past as far as the client was concerned. A timeout of one hour would have expired in the past for an IE instance running on my client. No wonder IE was rejecting my sessions.

I ran ntpdate to fix my time and then reset my timezone using tzselect.

I then refreshed IE which immediately started accepting the sessions. All aspects of my application started working correctly. So much for so little.

Moral of the story, use NTP to ensure that your machines have their time set correctly.

Recently, I released ClearAxis, a simple application that allows you to configure the Araxis Merge utility as the default diff tool within Rational ClearCase. The ClearCase diff tool is rather primitive and could really use some replacing.

ClearAxis is written in Python and is packaged using Py2Exe so that any end user could run it without having to install Python. More importantly, ClearCase can only invoke executables from it’s map file and running python.exe with command line arguments does not work. As a result, ClearAxis had to be a standalone executable.

While developing ClearAxis, I ran into a problem with Py2Exe that I had encountered earlier with Clump as well. If you run the generated .exe from the directory it is located in, everything works fine. If you specify the full path to the .exe on the command line, that works too. However, when ClearAxis was executed through the ClearCase map file, you’d get the following error in the .exe.log file:

Traceback (most recent call last):
File “clearaxis.py”, line 1, in ?
zipimport.ZipImportError: can’t decompress data; zlib not available

With Clump, this error occurred when I added the Clump directory to the system path and tried running clump.exe from any directory. Working around this for Clump wasn’t a huge problem. I created a batch file in the Windows directory which resolved the full path the to .exe. So I didn’t need to add Clump to the system path, nor did I need to figure out this Zlib problem.

With ClearAxis, this workaround wasn’t viable. I searched the web for several hours, looking for solutions to this issue before I gave up and began scavenging the Py2Exe code instead. There’s multiple solutions for this issue using combinations of Py2Exe setup flags. The choice is between the size of the resulting application versus the number of files in the application directory. It also depends on how many executables are to be compiled by Py2Exe for your application.

Py2Exe Default

Just as reference, here’s the default settings in Py2Exe as a baseline. It is a good option if you run your application by resolving it’s full path like in a shortcut or batch file.

zipfile = “shared.lib”
compressed = 0

If the .exe path is not fully resolved, the Zlib error occurs.

Method 1

In this method, the Python libraries and required DLLs are compressed and bundled into the .exe itself. Since Zlib is bundled into the .exe, we can now decompress the libraries and the Zlib error goes away.

zipfile = None
compressed = 1
bundle_files = 2

This approach works well for an application with a single executable. For an application with multiple executables, this method increases the total size considerably since none of the common Python libraries and DLLs are shared. Instead, they are redundantly packaged into each executable.

Method 2

In this method, only the Python libraries are bundled into the executable. We can’t turn on compression since the Zlib DLL is not bundled. If we turn on compression, the Zlib error comes back since the libraries still need to be decompressed and Zlib can not be found.

zipfile = None
compressed = 0

Like Method 1, this approach also increases the total size if multiple executables need to be compiled, though not as much since the DLLs are still shared.

Method 3

In this method, all the Python libraries and the required DLLs are stored on the file system without any compression or packaging. As a result, no decompression needs to be performed and our Zlib error goes away.

zipfile = “shared.lib”
compressed = 0
skip_archive = 1

The issue with this approach is that we end up with more than 200 files spread across more than 15 directories in the resulting application. Not too pretty.

Comparison

The above methods provide different results as far as file size and directory structure are concerned. Here’s what it looks like for Clump which has a single executable. As you can see, Method 1 gives the best bang for the buck.

As for AppSnap which has two executables, one for the console and one for the GUI, the results are more interesting. Do you choose size or a clean directory structure?

Note: All sizes above are calculated after compressing all executables and DLLs with UPX and recompressing shared.lib with 7Zip at the maximum level. Check out the Py2Exe wiki under the 7Zip and UPX section for more information.

Conclusion

If you have one executable in your application, Method 1 seems to be the best option. For two to three executables, Method 2 seems to be a decent middle ground. For more than four executables, Method 3 is the most sensible since the redundancy is no longer justified. And all this only if you expect to run your application from a directory other than it’s own on the command line. Otherwise, the Py2Exe default gives us very acceptable results.

Which method to choose greatly depends on your application’s requirements. ClearAxis uses Method 2 for now but probably will move to Method 1 in the next release to avail the improved total size. As for AppSnap, I don’t care about executing it from a directory other than it’s own so it is using the default method for now.

Take a pick. At least there’s a solution.