I was recently interview by Joseph McAllister for his Sticky ToolLook newsletter. Sticky ToolLook is an extension of StickyMinds.com and Better
Software magazine. I mostly talk about Performance testing and tools.
The article can be found here: http://www.stickyminds.com/stickytoollook/index.asp?cd=5/10/2007
Transcript:
A Word with the Wise:
Tools and System Performance with Corey Goldberg
by Joseph McAllister
Corey Goldberg is a Boston-based software engineer who focuses
on performance engineering and tool development. He also
contributes to open source projects and has developed some of
his own, such as WebInject. I spoke with him earlier this year
about his passion for the craft of software tools.
Joseph McAllister:
What makes you passionate about performance
test tools?
Corey Goldberg:
I am passionate about system performance, so
tools are an integral part of that. Performance is an
interesting and diverse space. It touches technology in so many
ways, and the skill set it requires allows me to be close to
several different technical areas at once: development, testing,
analysis, design, operations, etc.
The thinking becomes pervasive, though. For example, last night
I was standing in line for movie tickets, and all I could think
about was how they could improve the queuing system to get
better sales throughput. I regularly have conversations with my
colleagues about service performance and input/output contention
at our local burrito joint.
JM:
Is there a particular type of performance tool that is more
"fun" to use? Is there a type that tends to offer more results?
CG:
The various types of tools all work together to form your
full tool set or test suite. They all have their own fun parts.
Load generation can be complex, as it involves software
development alongside workload modeling. But it is also the fun
part where you get to slam load through a system and watch it
react.
The most satisfying tools to build are analysis and monitoring
tools, especially tools with real-time monitoring and graphing.
This enables you to look inside your test runs or production
system and actually see what is happening in real time. Complex
data sets and metrics collected from deep within a system are
transformed into informative graphs as things happen. That is
pretty exciting to work with.
JM:
Is there a clear division between the commercial tools
you've used and the tools you've written? Do you prefer one or
the other?
CG:
Yes and no. First, for terminology, I like to think of
things in terms of proprietary vs. free tools. Proprietary tools
tend to force you into a certain pattern of use and often don't
provide the flexibility to change or extend them in ways you
might want.
Lately I have been building a lot of my own tools that work
alongside some commercial tools. I recently developed a
reporting and analysis suite that replaced the analytics in a
commercial tool we were using.
Commercial tools also offer some rich features that are
sometimes not feasible to re-create in a reasonable amount of
time. So you have to remember that building your own tools is
only worthwhile if it is cost effective.
JM:
Describe your open source test tool, WebInject.
CG: WebInject is a test tool that I developed in Perl that is
used for functional testing of Web application/services and
ad-hoc monitoring of HTTP response times. It can run as its own
GUI application with real-time graphing capabilities or can be
integrated as a plug-in with other tools.
I was doing this type of stuff in various scripts for years, so
I packaged a lot of it together and made a more generic
interface that can be used across a variety of projects. I
thought others might be interested in it, so I setup a
SourceForge project and released it in January 2004.
The basic concept is that you define test cases in XML files
that are fed through WebInject and executed against your system
under test. WebInject provides a basic harness/framework that
includes HTTP transport, parsing, cookie handling,
authentication, SSL, etc. It gives you real-time response timing
as well as functional verification using regex-based content
verification and HTTP status codes.
I spend all of my time on newer tools these days, but I still
keep on top of WebInject enough to facilitate others' using it
and posting patches/updates to it. Other test tools have
progressed a lot in the past few years, so I am sure there are
lots of new options for doing this type of testing. Oddly,
WebInject has become somewhat entrenched in monitoring systems.
Most of the users lately seem to be people running Nagios (an
open source monitoring system) that need an intelligent Web
plug-in/agent.
JM:
What is your favorite element of creating and distributing
an open source software tool?
CG:
Community feedback feels really good. I like sharing and
collaborating. The feedback is also tremendously useful in terms
of discovering bugs and offering suggestions, advice, or even
patches of working code. I care about my craft, and I realize
the only way to advance is through open collaboration.
I am also pretty influenced by the free software movement and do
some volunteer work with the GNU Project. I have some core
beliefs about the ethics of software freedom. Creating and
distributing my own GPL-licensed software is my own little way
to help that cause.