Sticky ToolLook - Tools and System Performance with Corey Goldberg

I was recently interview by Joseph McAllister for his Sticky ToolLook newsletter.  Sticky ToolLook is an extension of StickyMinds.com and Better Software magazine.  I mostly talk about Performance testing and tools.

The article can be found here: http://www.stickyminds.com/stickytoollook/index.asp?cd=5/10/2007

Transcript:

A Word with the Wise:
Tools and System Performance with Corey Goldberg
by Joseph McAllister

Corey Goldberg is a Boston-based software engineer who focuses on performance engineering and tool development. He also contributes to open source projects and has developed some of his own, such as WebInject. I spoke with him earlier this year about his passion for the craft of software tools.

Joseph McAllister:
What makes you passionate about performance test tools?

Corey Goldberg:
I am passionate about system performance, so tools are an integral part of that. Performance is an interesting and diverse space. It touches technology in so many ways, and the skill set it requires allows me to be close to several different technical areas at once: development, testing, analysis, design, operations, etc.

The thinking becomes pervasive, though. For example, last night I was standing in line for movie tickets, and all I could think about was how they could improve the queuing system to get better sales throughput. I regularly have conversations with my colleagues about service performance and input/output contention at our local burrito joint.

JM:
Is there a particular type of performance tool that is more "fun" to use? Is there a type that tends to offer more results?

CG:
The various types of tools all work together to form your full tool set or test suite. They all have their own fun parts. Load generation can be complex, as it involves software development alongside workload modeling. But it is also the fun part where you get to slam load through a system and watch it react.

The most satisfying tools to build are analysis and monitoring tools, especially tools with real-time monitoring and graphing. This enables you to look inside your test runs or production system and actually see what is happening in real time. Complex data sets and metrics collected from deep within a system are transformed into informative graphs as things happen. That is pretty exciting to work with.

JM:
Is there a clear division between the commercial tools you've used and the tools you've written? Do you prefer one or the other?

CG:
Yes and no. First, for terminology, I like to think of things in terms of proprietary vs. free tools. Proprietary tools tend to force you into a certain pattern of use and often don't provide the flexibility to change or extend them in ways you might want.

Lately I have been building a lot of my own tools that work alongside some commercial tools. I recently developed a reporting and analysis suite that replaced the analytics in a commercial tool we were using.

Commercial tools also offer some rich features that are sometimes not feasible to re-create in a reasonable amount of time. So you have to remember that building your own tools is only worthwhile if it is cost effective.

JM:
Describe your open source test tool, WebInject.

CG: WebInject is a test tool that I developed in Perl that is used for functional testing of Web application/services and ad-hoc monitoring of HTTP response times. It can run as its own GUI application with real-time graphing capabilities or can be integrated as a plug-in with other tools.

I was doing this type of stuff in various scripts for years, so I packaged a lot of it together and made a more generic interface that can be used across a variety of projects. I thought others might be interested in it, so I setup a SourceForge project and released it in January 2004.

The basic concept is that you define test cases in XML files that are fed through WebInject and executed against your system under test. WebInject provides a basic harness/framework that includes HTTP transport, parsing, cookie handling, authentication, SSL, etc. It gives you real-time response timing as well as functional verification using regex-based content verification and HTTP status codes.

I spend all of my time on newer tools these days, but I still keep on top of WebInject enough to facilitate others' using it and posting patches/updates to it. Other test tools have progressed a lot in the past few years, so I am sure there are lots of new options for doing this type of testing. Oddly, WebInject has become somewhat entrenched in monitoring systems. Most of the users lately seem to be people running Nagios (an open source monitoring system) that need an intelligent Web plug-in/agent.

JM:
What is your favorite element of creating and distributing an open source software tool?

CG:
Community feedback feels really good. I like sharing and collaborating. The feedback is also tremendously useful in terms of discovering bugs and offering suggestions, advice, or even patches of working code. I care about my craft, and I realize the only way to advance is through open collaboration.

I am also pretty influenced by the free software movement and do some volunteer work with the GNU Project. I have some core beliefs about the ethics of software freedom. Creating and distributing my own GPL-licensed software is my own little way to help that cause.