"compared to Java code, XML is agile and flexible. Compared to Python code, XML is a boat anchor, a ball and chain."
    - Phillip J. Eby

If you are new to Python and coming from Java (or C#, or other similar statically typed OO language), these classic articles from PJE and Ryan Tomayko are necessary reading:

• Python Is Not Java (PJE)
• Java is not Python, either (PJE)
• Python Interfaces are not Java Interfaces (PJE)
• The Static Method Thing (Tomayko)
• Getters/Setters/Fuxors (Tomayko)

Finally did the version 1.0 release! visit www.pylot.org to download.

Pylot is still lacking some features I want to add for it to become a serious performance/load testing tool, but the current release delivers very usable functionality.

Current Features:

• multi-threaded load generator
• HTTP and HTTPS (SSL) support
• response verification with regular expressions
• execution/monitoring console
• real-time stats
• results reports with graphs
• GUI mode
• shell/console mode
• cross-platform

Aside from the GUI, there is also a new shell/console interface mode with real-time output for quickly profiling performance your application/service under test from the command line. In this mode, Pylot can run cross-platform. (tested on Windows XP, Vista, Cygwin, Ubuntu, MacOS)

Note: Extra special thanks to Vasil Vangelovski for implementing the original console output and C++ extension

Screenshots of the GUI and new shell/console UI output:

Gnuplot is a free cross-platform program for scientific plotting, but is also useful for generating all sorts of other graphs.

There are lots of tutorials showing how to use Gnuplot in interactive mode.  However, Gnuplot can also be scripted from the command line for generating static images of graphs/plots.  I usually need to graph performance data, consisting of X,Y data points.  Gnuplot allows me to do this easily by passing a config file to the Gnuplot program in non-interactive mode.

Here is a sample data file (plot.data) containing x, y coordinates:

0 2
3 11
5 1
9 10
14 11
20 18
30 9

Here is a sample Gnuplot config file (plot.cfg)

set term png 
set output "plot.png"
set size 0.8,0.5
set yrange [0:]
set xlabel "Elapsed Time"
set ylabel "Throughput (requests/sec)"
plot "plot.data" using 1:2 title "" w lines

From the command line, you can run it like:

Windows:

> wgnuplot.exe plot.cfg

Unix/Linux:

$ gnuplot plot.cfg

This will create an image named "plot.png" containing the line graph:

Here is sample output of rendering a larger data set with an impulse graph and a line graph:

Now that you can run it from the CLI, it is easy to integrate with other scripts or programs.

I love this sarcastic rant by Ryan Tomayko from March 2006.  In the article, he rubutts some of James Gosling's comments regarding Java vs. dynamic languages.  Ryan pokes some fun at comments about dynamic languages and why they should be taken seriously; rather than thrown into the "scripting language ghetto".

My favorite part:

"Dealing with questions on dynamic languages:

First, call anything not statically compiled a “scripting language”. Attempt to insinuate that all languages without an explicit compilation step are not to be taken seriously and that they are all equivalently shitty. Best results are achieved when you provide no further explanation of the pros and cons of static and dynamic compilation and/or typing and instead allow the reader to simply assume that there are a wealth of benefits and very few, if any, disadvantages to static compilation. While the benefits of dynamic languages–first realized millions of years ago in LISP and Smalltalk–are well understood in academia, IT managers and Sun certified developers are perfectly accepting of our static = professional / dynamic = amateurish labeling scheme.

This technique is also known to result in dynamic language advocates going absolute bat-shit crazy and making complete fools of themselves. There have been no fewer than three head explosions recorded as a result of this technique.

Also, avoid the term “dynamic language” at all cost. It’s important that the reader not be exposed to the concepts separating scripting languages like bash, MS-DOS batch, and perl-in-the-eighties from general purpose dynamic languages like Ruby, Python, Smalltalk, and Perl present day."

When I release and publish code, I typically make it Free Software by copyrighting it and applying a license (usually GNU GPL).  In cases of smaller code bases or scripts I write, I don't bother to license them.  They are generally used as small tools or code snippets.

One choice is to release it as Public Domain.  Being in the public domain is not a license.  It means the material is not copyrighted and no license is needed. However, I want to retain copyrights but allow others the freedom to do anything.  This is based on trust of the author, not a license or an any official law speak.

To make the point clear that I don't care, I adapted Woody Guthrie's Copyright notice to apply to my software.

My new Copyright Notice for unlicensed code:

Copyright (c) 2008, Corey Goldberg (corey@goldb.org)

"This code is Copyrighted, and anybody caught usin' it without my permission, will be mighty good friends of mine, cause I don't give a dern.  Run it.  Study it.  Modify it.  Improve it.  Distribute it.  Yodel it.  I wrote it, that's all I wanted to do."

Feel free to use this notice on any software you publish, cause I don't give a dern.

Since I started this blog, my subscriber count keeps slowly increasing.  I'm still only at 231 subscribers, but the blog is also syndicated via Planet Python and testingReflections.

... I need more subscribers.  Go ahead and subscribe now!

I just quietly did a release of Pylot (my open source web performance tool).  You can grab it here.

The Charles River Skatepark is going to be built right down the street from my apartment.  I will definitely be a regular.

How insane is this setup?!

I just ordered an Asus Eee PC, the tiny portable 7" laptop.  I got the Black 4G Surf model.  I am psyched to get an ultraportable that runs Debian (Xandros) Linux!

I will blog about it if I do anything cool with it.  It comes with Python installed ;)

via Valleywag:

Quote from Bill Gates about Google:

"The Google tools that have tried to do productivity type things, they really don't have the richness, the responsiveness. Most of these Google products, to be frank, the day they announce them is their best day. I remember there was one called G Talk. I can barely remember the name but it was, you know, going to change the world."

Pretty funny comment since I use GTalk all day long and am slowly seeing my contacts fill up with other GTalk users.  I have never used MSN Messenger and never plan to.

This script will output bytes received and transmitted for a local Windows machine since the last reboot:

import re
from subprocess import Popen, PIPE

p = Popen('net statistics workstation', stdout=PIPE)
for line in p.stdout:
    m = re.search('Bytes received\W+(.*)', line)
    if m: print 'Bytes received: %s' % (m.group(1))
    m = re.search('Bytes transmitted\W+(.*)', line)
    if m: print 'Bytes transmitted: %s' % (m.group(1))

This script will output the last reboot date/time for a local Windows machine:

import re
from subprocess import Popen, PIPE

p = Popen('net statistics workstation', stdout=PIPE)
m = re.search('(\d+/\d+/\d{4}.*[A|P]M)', p.stdout.read())
if m: print 'Last Reboot: %s' % (m.group(1)) 

Output:

>> Last Reboot: 3/1/2008 1:51:41 PM

* updated the original script thanks to Ian's comment below

Here is how to zero-pad single digit days or months in a date string:

date = '3/2/2008'
padded_date = time.strftime('%m/%d/%Y', time.strptime(date,'%m/%d/%Y'))
print padded_date

>> 03/02/2008

A palindrome is a sequence that reads the same in either direction.

Here is function I wrote to check if a phrase is a palindrome:

import re

def is_palindrome(txt):
    txt = re.sub('\W+', '', txt).lower()
    return txt == txt[::-1]

phrase = "Go hang a salami, I'm a lasagna hog"
print is_palindrome(phrase)

>> True

I just did a Geolocation mashup to visualize where my blog readers are coming from on a particular day. Below is a visualization of my US/Canada visitors from February 14, 2008.

All of the data is based off of IP addresses from my referrer logs.

To see how to do this, check out: http://www.goldb.org/geo_maps

"Foo" is my favorite word. I use it all over the place. When programming, it is the name of my quick junk files... it is a temporary variable name while i figure out code. We programmers love our foo.

For those that want the origins and etymology, there is a great RFC about it:

I write a lot of command line tools and scripts in Python. Sometimes I need to kick them off remotely. A simple way to do this is to launch a tiny web server that listens for a specific request to start the script.

I add a "WebRequestHandler" class to my script and call it from my main method. There is a "do_something()" method in the class. You call your code from this method.

All you have to do is launch your script and it will sit there and wait for requests. If the request is bad, it spits back a 404 error. If the request path matches what we are looking for (in this case "/foo"), the code is launched.

Now you have an easy way to call your script remotely. Just open a browser and type in the URL: http://your_server/foo, or call it with a tool like 'wget' or 'curl'.

import BaseHTTPServer

class WebRequestHandler(BaseHTTPServer.BaseHTTPRequestHandler):
    def do_GET(self):
        if self.path == '/foo':
            self.send_response(200)
            self.do_something()
        else: 
            self.send_error(404)
            
    def do_something(self):
        print 'hello world'
        
server = BaseHTTPServer.HTTPServer(('',80), WebRequestHandler)
server.serve_forever()

(this was adapted from a code sample in "Python In A Nutshell" by Alex Martelli)

In many programming languages you can't terminate a thread directly.  Python is no different.  Rather than termintaing a thread from the code that spawned it, you just a pass a flag to the thread that tells it to terminate itself.  Typically a thread will run in a loop, periodically checking this flag so it knows if it should continue or not.  To terminate the thread from the outside, you just set its flag to die.

I was using this idiom in Python by setting a boolean flag in my spawned thread.

So a simplified thread class would look something like this:

class MyThread(threading.Thread):
    def __init__(self, num):
        threading.Thread.__init__(self)
        self.running = True
        self.num = num
        
    def stop(self):
        self.running = False
        
    def run(self):
        while self.running:
            print 'hello from thread %d' % self.num
            time.sleep(1)

I just read an old post in comp.lang.python that pointed to a recipe in the Python Cookbook that suggests using threading.Event() rather than a simple boolean flag.

So the thread class would look something like this:

class MyThread(threading.Thread):
    def __init__(self, num):
        threading.Thread.__init__(self)
        self.stop_event = threading.Event()
        self.num = num
        
    def stop(self):
        self.stop_event.set()
        
    def run(self):
        while not self.stop_event.isSet():
            print 'hello from thread %d' % self.num
            time.sleep(1)

They work exactly the same.

I am just wondering what other flexibility threading.Event() gives you, and if there is anything bad about using simple boolean checks to kill threads. I guess I will have to look it up and play around a bit.

Here is my bricked iPod. It won't even turn on.  It is time to retire it. However, it has a perfectly good 30 gig hard drive inside.  Gotta be good for something, right?

First I crack it open and get a look at the insides.

The Toshiba 2.5" IDE hard drive sits right near the top.

Next, I bought a 2.5" USB external IDE hard drive enclosure.

I installed the hard drive into the new enclosure, and it's all done. Now I have a nice external storage device.

I just installed the latest Alpha of Python 3000.

So far so good...

Programmatic Testing:  A style of software testing where tests are executed without human interaction.

When I say "this test can be automated", I don't mean a computer can magically replace a sapient process to create a useful test.  All test design and methodology is devised using a sapient processes (obviously).  However, testcase generation, data generation, test execution, results analysis, etc, can often be done without human interaction.  This has been described using terms like: "Automated Testing, "Test Automation", etc.  Among some of the more vocal people in the testing community, these terms are considered confusing,  inaccurate, harmful, or just a cover up for what is actually a non-scripted test

Automated and Manual Testing are not mutually exclusive.  Rather, they are at either end of a continuum.  A test isn't "automated" or "manual".  It falls somewhere on that continuum.  Each step of the process has some mixture of the elements.

I have also heard the term: "Computer Assisted Testing".  I don't like this term because it tries to place a name on the entire testing process.  I am not referring to the entire process, so this becomes confusing talk to me.  I am referring to a style of test execution.

Therefore, to clear up ambiguity of terminology, I now refer to "Automation" as "Programmatic Testing".  This refers to a style of test execution where tests are executed in a non-interactive manner.

I you have a tool, a test framework, a software library, a program/script, functions for checking conditions, or a mechanism for reporting errors and stats, then you are doing a form of programmatic testing.

Sending HTTP Requests from a C# program seems unnecessarily hard.  I wrote a small helper class to deal with sending and timing GET requests:
http://www.goldb.org/httpgetcsharp.html

You use it like this:

public class Program
{
    static void Main(string[] args)
    {
        HTTPGet req = new HTTPGet();
        req.Request("http://www.google.com");
        Console.WriteLine(req.StatusLine);
        Console.WriteLine(req.ResponseTime);
    }
}

Convert a number of seconds into a human readable time string HH:MM:SS

7046 seconds is: 1 hour 57 mins 26 secs, or 01:57:26

The Function:

def humanize_time(secs):
    mins, secs = divmod(secs, 60)
    hours, mins = divmod(mins, 60)
    return '%02d:%02d:%02d' % (hours, mins, secs)

The Output:

print humanize_time(7046)
>>  01:57:26