Writing Functests:

"This is where functest really shines. When you run functest against any directory or file the collector actually compiles a dependency chain which includes any valid python modules in all the parent directories..."

Hm. So functest would let me dispense with this nonsense boilerplate in my test modules?
import sys
import os
thisdir = os.path.abspath(os.path.dirname(file))
def relpath(p):
    return os.path.abspath(os.path.join(thisdir, p))

I wonder if nose does/will provide similar facilities? It's pretty nice in other respects...

Twill, CherryPy 3 and testing with internal servers

Twill can test CherryPy web servers "in process", without having to run them as servers. I've used twill this way with CherryPy 2, but when I recently tried to do the same thing with a CherryPy 3 server everything broke.

I had based my code on an old CherryPy 2 tutorial by Titus Brown, and was trying to grep through the CherryPy 3 code tree to figure out how to patch it up. Bad idea :)

Yesterday Google finally turned up a solution, from a talk Titus gave at PyCon 2007.

The upshot: don't use cherrypy._cpwsgi.CPWSGIApp. In CherryPy 3, your application itself is already a WSGIApp. What's more, I think cherrypy._cpwsgi.CPWSGIApp may be broken. For sure, it and twill have incompatible ideas about the correct type of the 2nd constructor argument.

Anyway, thanks to Titus, here's an example of how to connect twill to a CherryPy 3 application for in-process testing:

import twill
import cherrypy
# ...
wsgi_app = cherrypy.tree.mount(myApp, "/", myConfig)
twill.add_wsgi_intercept('localhost', 8080, lambda: wsgi_app)

[Update 2008/03/24] Ditched the cherrypy.server.quickstart() invocation, per fumanchu's comment. Reported as Ticket #801.

waf build errors

I'm evaluating waf to see if it's easier to use than SCons (especially for Qt projects). So far the results are encouraging. In particular, waf reports many kinds of problems using straightforward (color-coded!) error messages, rather than Python tracebacks.

target 'build_info' does not exist

Sometimes waf does spew an unintelligible Python traceback reminiscent of SCons. For me, these typically end with a KeyError:

File "/path/to/.waf-1.3.2-blah/wafadmin/", line 290, in must_run
prev_sig = tree.m_sig_cache[key][0]
KeyError: -214303645

This error always translates to "you are trying to build a file which is part of your source tree."

My SConscripts work in situ, creating object files etc. in the same directories where the source code resides. waf prefers to put build products in a separate build/ subdirectory. When waf is instructed to generate an output file in its build tree, and when that file already exists in the source tree, waf fails with the KeyError traceback.

I'd bet this problem is most common in software projects which started life using build systems like Make or SCons -- tools which default to building in situ.

For example, I'm trying to use waf in a subversion workspace where I've been doing SCons builds for years. To make matters worse, one of the build products is a header file, containing revision number and date for the current build. This looks like a source file, so it's likely that I never checked whether it was properly removed by scons -u -c.

In retrospect, I should have started with a clean check-out...

Intermittent Pickle Problems in Jython

The previous post mentioned "integrating" Jython and CPython by transmitting a stream of pickles between the two. I encountered one intermittent problem with this approach, and I'm unsure of its cause. (Hm, and I should probably post this to a Jython mailing list...)


In Jython I'd pickled the str() of a, into which I'd just written the SD representation of a CDK molecule. Jython could create the pickle alright. But when I tried to unpickle it in CPython, sometimes, for some molecules, I got a traceback:

File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/", line 970,  in load_string
    raise ValueError, "insecure string pickle"
ValueError: insecure string pickle

The error occurred consistently in my application code, always on the same input structure. But I couldn't derive a simple test script to demonstrate the problem.


Examination of the problematic pickle data showed that a Python unicode string literal marker had somehow been inserted, and the type code for the item was somehow S (for string) rather than V (for unicode):

Su'ZINC00000181\n  CDK...
 ^ What the... ?


Google turned up a usable workaround: encode the offending string as utf-8 before trying to pickle it.

import codecs

enc = codecs.getencoder('utf8')
sdf = enc(sdf)

Ted Leung: The Sun is going to shine on Python

Congratulations to Ted Leung (and to Frank Wierzbicki):
The Sun is going to shine on Python.

Their work at Sun will extend well beyond Jython, but Ted's post reminded me of my recent experiences. Jython 2.2 sure made it pleasant and easy[1] to get what I needed out of CDK, and to integrate the results into a CPython app via pickle and subprocess.

I did miss some language features such as yield which have appeared in CPython since 2.2. Maybe at some point Ted and/or Frank will have time to re-sync Jython with CPython?

[1] I had one intermittent problem with this approach. See the next post.

Netgear WAN MTU Size

(Not sure why I haven't written this up yet. It sure was a problem for a long time...)

I used to work "offsite" -- outside the home -- several days a week. When offsite my laptop could perform subversion operations against a repository on my home network, with good performance. But when the notebook was "inside" the home network, those same operations would take forever.

Eventually I stumbled on the fix, clearly explained in the help sidebar of the WAN Setup page of my Netgear WGR614 router: change the MTU to 1492. (Shades of Christopher Columbus!)

Nobody reads documentation :\

The normal MTU (Maximum Transmit Unit) value for most Ethernet networks is 1500 Bytes, 1492 Bytes for PPPoE connections, or 1436 for PPTP connections.

Site-specific browsers and web-app testing


Site-specific browsers should simplify testing of some types of web applications, e.g. those which support multiple registered users. On platforms such as Mac OS X, which typically run only one instance of each application, SSBs should allow you to launch multiple, distinct applications which point to the same website.

Is anyone doing this? Is it better (and does it always suffice) to simply run multiple copies of a twill script, and to dispense with browser-based tests?

I haven't tried Fluid yet, since my main work machine is still running Tiger.

Alas, I guess basic Prism apps won't do the job:


Technorati Tags:
, , ,