goldb.org home

AS OF MAY 2008, THIS BLOG IS NO LONGER BEING UPDATED.
Visit the new blog at: http://coreygoldberg.blogspot.com


 Friday, June 01, 2007

PyLT - Scratching My Itch - New Web Performance/Load Test Tool (Open Source)

(Update: PyLT has been renamed to Pylot)

I have started development on a new web performance/load testing tool.  It is targeted at testing Web Services.


Here is some Q&A with myself:

You know you are reinventing the wheel, right?

Yes, I know.  There are already open source web load testing tools available (OpenSTA, JMeter, Grinder, WebLOAD, etc).  I have used all of these as well as proprietary tools for years.  I am a performance engineer and I feel like I need a tool set that I am intimately familiar with.  I need the ability to easily alter and tweak the tool at will.  I don't have the time, budget, or patience enough to wait on vendors when I need something.  I also want a tool that is fun to hack and adapt.  For this, I need to understand the code base deeply.

What language are you using?

Python.  The initial GUI uses Tk, but this may be changed down the road. I use Python's threading module for concurrency. If this doesn't scale well enough, I will be exploring other models of concurrency (perhaps generator based coroutines).

Why do you think you can write a tool like this?

I have worked in performance testing for nearly 10 years.  I have written many tools that work with various protocols to do distributed load generation and testing.  Creating a simple HTTP load generator is sort of my Hello World 2.0 for each language I try (I have written these from scratch in Python, Perl, Java, and C#).  This tool takes that basic concept and organizes it into a robust application.

Will it be Free and Open Source?

Of course!  Licensed under GNU GPL.


For an early look, check out the source repository at:  http://pylt.googlecode.com/svn/trunk

More details to come.

-Corey

 |  |  |  |  |  |  | 
#    Comments [6] |
Friday, June 01, 2007 10:26:37 AM (Eastern Standard Time, UTC-05:00)
Instead of threads have you thought of using parallel python (http://www.parallelpython.com/)? I have found it really easy to use and it would allow you to scale your load test using multiple processors or even multiple machines is trivial compared to trying to use threading. My experiments have lead me to believe it to have very low overhead.
Lateef
Friday, June 01, 2007 11:00:26 AM (Eastern Standard Time, UTC-05:00)
Lateef,
thanks for the tip. I already have a threaded version working, but I will certainly check out parallelpython.

-Corey
Corey
Friday, June 01, 2007 3:15:07 PM (Eastern Standard Time, UTC-05:00)
Two recommendations:

1) Drop Tk, and go with wxPython (The xwPython In Action book is actually pretty good)
2) Utilize pyCurl, it does a lot of stuff for you and the low level calls are C. Checkout the retriever.py and retriver-multi.py scripts the "examples" directory.

I had to write something to load test a site pretty quick and I just modified the retriever.py script and put some timing info in there, pretty simple. On the web services side I cheated and just grabbed the whole envelope (as a string) and stuck the values I needed into the string and then did a POST to the ws. Just a little warning...Python's SOAP support sucks when dealing with MSFT web services, at least I've never had any luck with it.
Steve | scragrahamAT NOSPAMgmail dot com
Friday, June 01, 2007 5:51:44 PM (Eastern Standard Time, UTC-05:00)
@ Steve:

> Drop Tk, and go with wxPython

I will probably eventually move to a richer toolkit (like wxPython) if I take this thing far. For right now, Tk works. It comes distributed with core python, its super fast and light, and its easy to use, and I know it pretty well. though it looks like crap and is limited in many ways.

> Utilize pyCurl ... low level calls are C

I might investigate that if I run into scaling problems doing it in pure python (if thats even true). I have written all (most) of the lower level http stuff already.. That is the stuff I actually like doing. I developed a tool (WebInject - www.webinject.org) in Perl. It also does all the low level stuff, but inside a hairball of Perl i wrote years ago. I want to apply the same thing here, but in Python.. and this is used for performance. I am just trying to take a lot of the one-offs and load test scripts/tools/ideas i have had over the years into one nice application that meets my own needs (and hopefully others).

> Python's SOAP support sucks when dealing with MSFT web services, at least I've never had any luck with it.
> I cheated and just grabbed the whole envelope (as a string) and stuck the values I needed into the string
> and then did a POST to the ws

my tool is agnostic to whats in the http payload. it can be POX, SOAP, query string, poems, whatever :)
you just shove envelopes in as strings (read from filesystem) and use the proper http heads and method.

thanks for the input.

-Corey
Corey
Sunday, June 03, 2007 8:20:02 AM (Eastern Standard Time, UTC-05:00)
Interesting to read the recommendations of toolsets here, but whatever you choose, I'm looking forward to PyLT: we can use all the help we can get testing web applications.
Ned Batchelder
Monday, June 11, 2007 2:03:09 PM (Eastern Standard Time, UTC-05:00)
>> Drop Tk, and go with wxPython
>I will probably eventually move to a richer toolkit (like wxPython)

well.. this came sooner than later. I mocked up the initial UI in Tk and quickly realized it wasn't a rich enough GUI Toolkit for what I want to do. As of today I am switching over to wxWindows and using wxPython for the UI.

-Corey
Corey
Comments are closed.