I started a project on Google Code to create a set of Python tools to make dealing with Round Robin Databases (RRD) less painful.  Setting up RRD's can be tough if you don't know what you are doing.

anyone interested can check it out here:  rrdpy

I used it to create a simple HTTP monitoring script (included in source) to graph web response latency like this:

I just joined Twitter.  I have no idea how much I'll use it, but I guess I need to see what all the [good/bad] hype is about.

Follow me there: twitter.com/cgoldberg

When working with data sets, a common task I need to do is slurp a csv file into a nested data structure that contains a sequence of lists correlating to the rows and values in the csv file.

For example...

File contents (foo.csv):

10,20,30,40
19,29,39,55
16,21,31,59

Result:

[['10', '20', '30', '40'],
 ['19', '29', '39', '55'],
 ['16', '21', '31', '59']]

To accomplish this. you could parse it inside a big honkin' list comprehension and build our structure in one step:

csv_file = 'foo.csv'
value_lists = [line.split(',') for line in 
    [line.strip() for line in open(csv_file, 'r').readlines()]]

You could also use the csv module from Python's standard library:

import csv
csv_file = 'foo.csv'
value_lists = list(csv.reader(open(csv_file, 'r')))

The csv module has some useful tools for reading/writing csv files.  Check it out.

Jesse Noller just blogged "Finding Python people is hard":

Here is a good quote regarding the difficulty in finding skilled Test Engineers with Python experience:

"Either you teach QA people automation/test engineering, or you try to find a programmer who wants to learn/do test engineering and teaching them python. It's a hard sell either way. I technically view QA as one discipline, Development as another, but Test Engineering as the Hybrid of the two - and you need a strong background in both."

I have seen lots of QA Engineers and Testers with little to no development/programming experience. This seems to be such a valuable skill; why not learn some? The bar is set really low with today's dynamic languages. Getting into some quick scripting for data manipulation and building test harnesses is not a huge task. If a QA engineer can't learn some simple programming in a week, would you trust his efficiency and technical skills?

I agree with Jesse on this one. We need to see more Test Engineers and Developers In Test. Unfortunately, this hybrid roll often falls through cracks as many people view quality/testing vs. developing as a binary choice.

New Pylot 1.1 release is available
Visit: www.pylot.org/download.html

It contains some minor code cleanup and a new test case recorder contributed by David Solomon. The recorder works with Windows and IE only.

It is a script that launches your web browser and records HTTP requests as you navigate. While it records, it prints Pylot's XML test cases. The test cases are printed to STDOUT, so just redirect it to a file and you will have a valid testcases.xml file to use as Pylot input.

The pylot_recorder script is included in the lib directory of Pylot 1.1.

View the recorder's source code from the SVN trunk:
http://code.google.com/p/pylt/source/browse/trunk/lib/pylot_recorder.py

(It can't handle some complex scenarios, but is useful for recording simple GET and POST requests from web applications)

The Python Cookbook has a recipe for splitting a list into roughly equal-sized pieces:

http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/425397

In the comments, there are several alterate implementations. Sebastian Hempel has an interesting take on it using slicing for the calculation of the list lengths. It basically looks like this:

def split_seq(seq, num_pieces):
    start = 0
    for i in xrange(num_pieces):
        stop = start + len(seq[i::num_pieces])
        yield seq[start:stop]
        start = stop

This version of the function distributes the remaindered items evenly over the first few splits.

Example Usage:

seq = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]

for seq in split_seq(seq, 3):
    print seq

This script uses your system's ping utility to send an ICMP ECHO_REQUEST to a list of hosts or devices. It uses a separate thread to ping each host/device. This can be useful for measuring network latency and verifying hosts are alive.

Check out more info here: http://www.goldb.org/python_pinger.html

#!/usr/bin/env python

import re
from subprocess import Popen, PIPE
from threading import Thread


class Pinger(object):
    def __init__(self, hosts):
        for host in hosts:
            pa = PingAgent(host)
            pa.start()
        
class PingAgent(Thread):
    def __init__(self, host):
        Thread.__init__(self)        
        self.host = host

    def run(self):
        p = Popen('ping -n 1 ' + self.host, stdout=PIPE)
        m = re.search('Average = (.*)ms', p.stdout.read())
        if m: print 'Round Trip Time: %s ms -' % m.group(1), self.host
        else: print 'Error: Invalid Response -', self.host
              
                             
if __name__ == '__main__':
    hosts = [
        'www.pylot.org',
        'www.goldb.org',
        'www.google.com',
        'www.this_one_wont_work.com'
       ]
    Pinger(hosts)

Output:

Round Trip Time: 14 ms - www.yahoo.com
Round Trip Time: 17 ms - www.goldb.org
Round Trip Time: 30 ms - www.google.com
Round Trip Time: 82 ms - www.pylot.org
Error: Invalid Response - www.this_one_wont_work.com

Note: I only tested this on Windows. To run on other systems, it would only require a one-line change.

You can use this little Python function to see what type of web server a site is running.  All it does is send an HTTP request to the host and reads the 'server' header in the response.

import httplib

def get_server_type(host):
    conn = httplib.HTTPConnection(host)
    conn.request('GET', '/')
    resp = conn.getresponse()
    return resp.getheader('server')


print get_server_type('www.pylot.org')
print get_server_type('www.techcrunch.com')

Output:

lighttpd/1.4.19
Apache/2.0.52

Note: This doesn't work for all sites