Over the last several years, we have seen many big software players like IBM, HP, and Borland, acquire most of the larger automation tool vendors (Rational, Mercury, Segue, respectively).

Oracle just followed suit with their announcement to buy acquire the e-TEST suite products from Empirix:

http://www.oracle.com/empirix/index.html

I guess it was just a matter of time.  It will be interesting to see where all of this is going.

In the various testing forums and mailing lists I frequent, I see lots of questions asked about automated test tools.  A certain question about performance/load testing tools is asked over and over.  I would like to answer it here.

The question goes something like:
"Can I test my application with this performance tool?".

If you are asking this question, you really don't understand the nature of these tools and how they work.

There are many performance/load testing tools in the proprietary and open source worlds (LoadRunner, SilkPerformer, QALoad, JMeter, OpenSTA, etc).  Conceptually, they all work the same way.  Performance tools do not work by driving a GUI like many functional test tools do.  Instead, they are used to generate load against servers by simulating concurrent client requests at the protocol level.

So the key question you should be asking is: "Which protocols do the client and server use to communicate, and does the tool support these protocols?"  HTTP?  ODBC?  DCOM?  JMS?  So next time you want to ask: "Will the tool be able to test my AJAX/ASP.NET/JSP/PHP/etc application?"; the real question should go something like: "My application uses <protocol x>, does the tool support that protocol?".

Since you are working at the protocol level, the language/platform your system uses has no relevance. All of the technologies I just mentioned use HTTP for transport (layer 7), so any tool that supports HTTP can be used to test that system.

Interested in Web performance testing?
Check out my open source tool: Pylot

"Microsoft must stop polluting the downstream with patent encumbrances.  Period.  Full stop."

- Matt Assay (Microsoft's dilemma: The importance of the downstream)

Nice testing article from Titus Brown.  He makes some good points about automated tests:

The (Lack of) Testing Death Spiral

"So, automated tests are important for maintenance, and they are critical for making sure that your old code still works while you focus on new code. Without automated tests, you will be doomed to releasing increasingly buggy software as your body of code increases and the average level of testing decreases."

"compared to Java code, XML is agile and flexible. Compared to Python code, XML is a boat anchor, a ball and chain."
    - Phillip J. Eby

If you are new to Python and coming from Java (or C#, or other similar statically typed OO language), these classic articles from PJE and Ryan Tomayko are necessary reading:

• Python Is Not Java (PJE)
• Java is not Python, either (PJE)
• Python Interfaces are not Java Interfaces (PJE)
• The Static Method Thing (Tomayko)
• Getters/Setters/Fuxors (Tomayko)

Finally did the version 1.0 release! visit www.pylot.org to download.

Pylot is still lacking some features I want to add for it to become a serious performance/load testing tool, but the current release delivers very usable functionality.

Current Features:

• multi-threaded load generator
• HTTP and HTTPS (SSL) support
• response verification with regular expressions
• execution/monitoring console
• real-time stats
• results reports with graphs
• GUI mode
• shell/console mode
• cross-platform

Aside from the GUI, there is also a new shell/console interface mode with real-time output for quickly profiling performance your application/service under test from the command line. In this mode, Pylot can run cross-platform. (tested on Windows XP, Vista, Cygwin, Ubuntu, MacOS)

Note: Extra special thanks to Vasil Vangelovski for implementing the original console output and C++ extension

Screenshots of the GUI and new shell/console UI output:

Gnuplot is a free cross-platform program for scientific plotting, but is also useful for generating all sorts of other graphs.

There are lots of tutorials showing how to use Gnuplot in interactive mode.  However, Gnuplot can also be scripted from the command line for generating static images of graphs/plots.  I usually need to graph performance data, consisting of X,Y data points.  Gnuplot allows me to do this easily by passing a config file to the Gnuplot program in non-interactive mode.

Here is a sample data file (plot.data) containing x, y coordinates:

0 2
3 11
5 1
9 10
14 11
20 18
30 9

Here is a sample Gnuplot config file (plot.cfg)

set term png 
set output "plot.png"
set size 0.8,0.5
set yrange [0:]
set xlabel "Elapsed Time"
set ylabel "Throughput (requests/sec)"
plot "plot.data" using 1:2 title "" w lines

From the command line, you can run it like:

Windows:

> wgnuplot.exe plot.cfg

Unix/Linux:

$ gnuplot plot.cfg

This will create an image named "plot.png" containing the line graph:

Here is sample output of rendering a larger data set with an impulse graph and a line graph:

Now that you can run it from the CLI, it is easy to integrate with other scripts or programs.

I love this sarcastic rant by Ryan Tomayko from March 2006.  In the article, he rubutts some of James Gosling's comments regarding Java vs. dynamic languages.  Ryan pokes some fun at comments about dynamic languages and why they should be taken seriously; rather than thrown into the "scripting language ghetto".

My favorite part:

"Dealing with questions on dynamic languages:

First, call anything not statically compiled a “scripting language”. Attempt to insinuate that all languages without an explicit compilation step are not to be taken seriously and that they are all equivalently shitty. Best results are achieved when you provide no further explanation of the pros and cons of static and dynamic compilation and/or typing and instead allow the reader to simply assume that there are a wealth of benefits and very few, if any, disadvantages to static compilation. While the benefits of dynamic languages–first realized millions of years ago in LISP and Smalltalk–are well understood in academia, IT managers and Sun certified developers are perfectly accepting of our static = professional / dynamic = amateurish labeling scheme.

This technique is also known to result in dynamic language advocates going absolute bat-shit crazy and making complete fools of themselves. There have been no fewer than three head explosions recorded as a result of this technique.

Also, avoid the term “dynamic language” at all cost. It’s important that the reader not be exposed to the concepts separating scripting languages like bash, MS-DOS batch, and perl-in-the-eighties from general purpose dynamic languages like Ruby, Python, Smalltalk, and Perl present day."

When I release and publish code, I typically make it Free Software by copyrighting it and applying a license (usually GNU GPL).  In cases of smaller code bases or scripts I write, I don't bother to license them.  They are generally used as small tools or code snippets.

One choice is to release it as Public Domain.  Being in the public domain is not a license.  It means the material is not copyrighted and no license is needed. However, I want to retain copyrights but allow others the freedom to do anything.  This is based on trust of the author, not a license or an any official law speak.

To make the point clear that I don't care, I adapted Woody Guthrie's Copyright notice to apply to my software.

My new Copyright Notice for unlicensed code:

Copyright (c) 2008, Corey Goldberg (corey@goldb.org)

"This code is Copyrighted, and anybody caught usin' it without my permission, will be mighty good friends of mine, cause I don't give a dern.  Run it.  Study it.  Modify it.  Improve it.  Distribute it.  Yodel it.  I wrote it, that's all I wanted to do."

Feel free to use this notice on any software you publish, cause I don't give a dern.

Since I started this blog, my subscriber count keeps slowly increasing.  I'm still only at 231 subscribers, but the blog is also syndicated via Planet Python and testingReflections.

... I need more subscribers.  Go ahead and subscribe now!

I just quietly did a release of Pylot (my open source web performance tool).  You can grab it here.

The Charles River Skatepark is going to be built right down the street from my apartment.  I will definitely be a regular.

How insane is this setup?!

I just ordered an Asus Eee PC, the tiny portable 7" laptop.  I got the Black 4G Surf model.  I am psyched to get an ultraportable that runs Debian (Xandros) Linux!

I will blog about it if I do anything cool with it.  It comes with Python installed ;)

via Valleywag:

Quote from Bill Gates about Google:

"The Google tools that have tried to do productivity type things, they really don't have the richness, the responsiveness. Most of these Google products, to be frank, the day they announce them is their best day. I remember there was one called G Talk. I can barely remember the name but it was, you know, going to change the world."

Pretty funny comment since I use GTalk all day long and am slowly seeing my contacts fill up with other GTalk users.  I have never used MSN Messenger and never plan to.

This script will output bytes received and transmitted for a local Windows machine since the last reboot:

import re
from subprocess import Popen, PIPE

p = Popen('net statistics workstation', stdout=PIPE)
for line in p.stdout:
    m = re.search('Bytes received\W+(.*)', line)
    if m: print 'Bytes received: %s' % (m.group(1))
    m = re.search('Bytes transmitted\W+(.*)', line)
    if m: print 'Bytes transmitted: %s' % (m.group(1))

This script will output the last reboot date/time for a local Windows machine:

import re
from subprocess import Popen, PIPE

p = Popen('net statistics workstation', stdout=PIPE)
m = re.search('(\d+/\d+/\d{4}.*[A|P]M)', p.stdout.read())
if m: print 'Last Reboot: %s' % (m.group(1)) 

Output:

>> Last Reboot: 3/1/2008 1:51:41 PM

* updated the original script thanks to Ian's comment below

Here is how to zero-pad single digit days or months in a date string:

date = '3/2/2008'
padded_date = time.strftime('%m/%d/%Y', time.strptime(date,'%m/%d/%Y'))
print padded_date

>> 03/02/2008

A palindrome is a sequence that reads the same in either direction.

Here is function I wrote to check if a phrase is a palindrome:

import re

def is_palindrome(txt):
    txt = re.sub('\W+', '', txt).lower()
    return txt == txt[::-1]

phrase = "Go hang a salami, I'm a lasagna hog"
print is_palindrome(phrase)

>> True