goldb.org home

AS OF MAY 2008, THIS BLOG IS NO LONGER BEING UPDATED.
Visit the new blog at: http://coreygoldberg.blogspot.com



 Wednesday, September 26, 2007

Python - Tk Graph Example

I found a snippet to draw bar graphs in Python using Tk:
http://www.daniweb.com/code/snippet583.html

The output looks like this:


Here is a modified version that creates a bar graph in a Tk panel:

import Tkinter as tk

def graph_points(seq, width=375, height=325):
root = tk.Tk()
c = tk.Canvas(root, width=width, height=height, bg='white')
c.pack()
y_stretch = 15
y_gap = 20
x_stretch = 10
x_width = 20
x_gap = 20
for x, y in enumerate(data):
x0 = x * x_stretch + x * x_width + x_gap
y0 = height - (y * y_stretch + y_gap)
x1 = x * x_stretch + x * x_width + x_width + x_gap
y1 = height - y_gap
c.create_rectangle(x0, y0, x1, y1, fill="red")
c.create_text(x0+2, y0, anchor=tk.SW, text=str(y))
root.mainloop()

data = (18, 15, 10, 7, 5, 4, 2, 5, 8, 10, 13)
graph_points(data)
#    Comments [0] |
 Monday, September 24, 2007

Give One Get One - I Want My OLPC XO

OK.. looks like I need to plunk down $400 to get an XO.  When you buy one, they will give one away to a child in a developing nation.  Do some good, and get my own OLPC?  Too good to pass up.

For 2 weeks only, starting Nov. 12:  www.xogiving.org

#    Comments [0] |
 Monday, September 17, 2007

Old-School Pair Programming And My Inclination To Become A Tester

My first programming course was as an undergrad freshman in 1993. It was the basic introductory programming class for CS majors. The course was pretty difficult and was a great filter to separate the real CS students from the wannabes. About one-third of the students dropped the class, and out of the remaining two-thirds, many changed majors after this course was complete.

The course was taught using Scheme, with SICP as the text book. We programmed on a VAX cluster with Ultrix (DEC's Unix flavor) as the Operating System. We had to learn the Unix shells, VI, and all sorts of fun stuff to get us up and running.

The computer lab ("the cluster") was a large sterile room with rows of green-screen dumb terminals. I remember our professor told us that the VAX had 128 MB of memory and I was blown away by how huge that was (my rippin' fast PC had 4 MB at the time).

Spending hours in the computer lab was no fun at all. But I was one of the lucky ones. I owned a brand new 486-DX33 PC running Windows 3.1. I had a blazing fast 14.4 bps Zoom modem and could use Procomm Plus to dial into the VAX and program from the comfort of my own dorm room. I also found a Scheme interpreter that ran on DOS, giving me further options to do my work offline.

The programming assignments were brutal. All-nighters were the norm. Collaboration on the assignments was encouraged, but we were all expected to turn in our own original work. I found a fellow student that I got along with well and we decided to work together (unfortunately, I don't even remember his name... all I remember is that he was a lot smarter than me).

So the basic workflow was that we would get together, work out the basics of the assignment, get most of the algorithms working, then each take the code and finish it on our own. Since I had the bad-ass PC, we would work in my room. Two things quickly became apparent: He was a much better programmer than me, but I had a better eye for subtle details and debugging. Eventually we settled into a pattern where he would do the programming and I would look over his shoulder to give advice and input. Every few minutes, he would shoot a copy of the code to my dot-matrix printer. I would grab it, go through it line by line, and mark errors with my red pen and hand-write parts of the code that weren't correct. I would then hand the printout back to him and let him enter the changes. We iterated like this until we had all of the core code working.

For some reason, that instinct for attention to detail and debugging has always stuck with me. Because of that, my career has always been influenced by testing. I am a develop/tester, rather than just a developer. Most of the impact I have had in all of my jobs is from creating test tools and bringing in new ways to test software.

Just an interesting observation. I wonder how many others were naturally drawn to testing as soon as they started writing code?

#    Comments [2] |

Python - Yahoo Stock Quote Module

Last week I wrote a small Python module for retrieving stock prices.

It used screen scraping to get data from Google Finance.  Yahoo offers stock data in a much more digestible form which allowed me to get values without screen scraping and regular expressions.  So, I wrote a module based around this.

This new module is much more comprehensive and exposes a Python API for retrieving all sorts of stock data from Yahoo Finance.

My ystockquote module provides a Python API for retrieving stock data from Yahoo Finance.  This module contains the following functions:

  • get_all(symbol)
  • get_price(symbol)
  • get_change(symbol)
  • get_volume(symbol)
  • get_avg_daily_volume(symbol)
  • get_stock_exchange(symbol)
  • get_market_cap(symbol)
  • get_book_value(symbol)
  • get_ebitda(symbol)
  • get_dividend_per_share(symbol)
  • get_dividend_yield(symbol)
  • get_earnings_per_share(symbol)
  • get_52_week_high(symbol)
  • get_52_week_low(symbol)
  • get_50day_moving_avg(symbol)
  • get_200day_moving_avg(symbol)
  • get_price_earnings_ratio(symbol)
  • get_price_earnings_growth_ratio(symbol)
  • get_price_sales_ratio(symbol)
  • get_price_book_ratio(symbol)
  • get_short_ratio(symbol)

Sample Usage:


>>> import ystockquote
>>> print ystockquote.get_price('GOOG')
529.46
>>> print ystockquote.get_all('MSFT')
{'stock_exchange': '"NasdaqNM"', 'market_cap': '268.6B', 
'200day_moving_avg': '29.2879', '52_week_high': '31.84', 
'price_earnings_growth_ratio': '1.45', 'price_sales_ratio': '5.33',
'price': '28.65', 'earnings_per_share': '1.423', 
'50day_moving_avg': '28.7981', 'avg_daily_volume': '55579700',
'volume': '25330856', '52_week_low': '26.48', 'short_ratio': '1.60', 
'price_earnings_ratio': '28.65', 'dividend_yield': '1.38', 
'dividend_per_share': '0.40', 'price_book_ratio': '8.76', 
'ebitda': '20.441B', 'change': '-0.39', 'book_value': '3.315'}

The module is available here:  http://www.goldb.org/ystockquote.html

#    Comments [11] |
 Friday, September 14, 2007

Python - Stock Quote Module

I just wrote a tiny Python module for programmatically retrieving stock quotes from Google Finance:

The module:


import urllib
import re

def get_quote(symbol):
    base_url = 'http://finance.google.com/finance?q='
    content = urllib.urlopen(base_url + symbol).read()
    m = re.search('class="pr".*?>(.*?)<', content)
    if m:
        quote = m.group(1)
    else:
        quote = 'no quote available for: ' + symbol
    return quote


Sample usage:


#!/usr/bin/env python

import stockquote

print stockquote.get_quote('goog')


Output:


>> 529.56
#    Comments [8] |
 Tuesday, September 11, 2007

WebLOAD Open Source - Ain't So Open Source

"Open Source"

In the words of Inigo Montoya [The Princess Bride]:  "You keep using that word.  I do not think it means, what you think it means."

A few months back, Radview Software announced that they are releasing an open source version of WebLOAD, their web performance and load testing tool.  I was very excited about this and thought it was a fantastic move that would have a big impact in the test tool market.  I am a performance engineer and a huge Free/Open Source Software advocate, so I love to see companies in the space that interests me most come around to embrace openness.

In their press release, Radview stated:

"WebLOAD Open Source, licensed under the GNU Public License (GPL) version 2, is based on WebLOAD, the company's flagship product that is already deployed at 1,600 sites. Immediately available for free download and use, WebLOAD is a commercial-grade open source project with more than 250 engineering years of product development."

Ok cool.. they used the GPL and opened up the whole shebang.  Wow, this company actually "gets it"... right?

Umm.. not quite.

If you look through the source code that is available for WebLOAD Open Source, you will notice that only code for a small subset of the product is available.  In actuality, WebLOAD Open Source is a partially proprietary tool which is marketed as Open Source Software.  The software has significant limitations in functionality and scalability.  The source code which needs to be modified to remove these restrictions is not distributed.  So what we are left with is a crippled version of the tool.

In a recent post to the WebLOAD OS Forum, someone asked to see the source code for "proxynator", which is the recording feature in WebLOAD.

The response from the Forum Admin (Amir Shoval, a Radview employee) was this:

"Currently the source code for the proxynator is not available as part of the open source code of WebLOAD."

This is in direct contradiction to what their website states:

"WebLOAD Open Source introduces a unified script authoring environment for recording, editing and debugging."

When further probed about this, he stated the following:

WebLOAD Open Source is dual licensed:
  1. the WebLOAD Load Engine is totally open sourced and hence is licensed under the GPL
  2. but the complete WebLOAD is still licensed under a proprietary license, which grants free usage in WebLOAD Open Source.

wait.. wait.. WHAT?
I thought the press release said "licensed under the GNU Public License", and "WebLOAD Open Source is a fully functional, commercial-grade performance testing product"?  Nowhere on their website or marketing materials to they talk about this dual licensing and limited availability of source code.

Now, if we look at the End User License Agreement (EULA) that applies to WebLOAD Open Source, it gets worse:

2. License Restrictions. This License does not permit you or any third party to:
(i) modify, translate, reverse engineer, decompile, disassemble (except to the extent that this restriction is expressly prohibited by law) or otherwise attempt to discover the source code of all or any portion of the Software;
(ii) modify, translate or create derivative works of all or any portion of the Software;
(iii) copy the Software (other than a single copy solely for back-up or archival purposes);
(iv) rent, lease, sell, offer to sell, distribute, or otherwise transfer rights to the Software;

OK... so we have an "open source" product that is actually dual licensed, where a large portion of the toolset is proprietary.  And furthermore, by accepting the EULA, you give up all of your rights that were granted under the GPL.  Huh?

So... to reiterate, WebLOAD Open Source is *not* open source.  A subset of it is open source: the [crippled] load engine.  Contrary to what their press release and website says, it contains proprietary components that are released in binary form with no source code.  It is rather disappointing to see a company jump on the bandwagon of open source without respecting the freedom that is supposed to come with it.

In conclusion: Is WebLOAD Open Source currently Open Source?  No
Will WebLOAD Open Source actually become Open Source?  Well.. that's up to Radview

Hey, I'm all for Freedom.  I applaud Radview for any of the code they released under the GPL.  But lets be fair, if you want to call your product open source and reap any benefits that come along with that... you gotta walk the walk.

#    Comments [6] |

Python httplib2 - Handling Cookies in HTTP Form Posts

I often need to automate tasks in web based applications.  I like to do this at the protocol level by simulating a real user's interactions via HTTP.  Python comes with two built-in modules for this: urllib (higher level Web interface) and httplib (lower level HTTP interface).

However, I usually don't use either of these.  I prefer to use Joe Gregario's excellent httplib2 module (btw, I really wish this could make its way into Python's Standard Library).  It is a much richer library and has a lot of nice features for dealing with HTTP.  

When automating something, you often need to "login" to maintain some sort of session/state with the server.  This is usually achieved with form-based authentication. You post a form to the server, and it responds with a cookie in the incoming HTTP header.  You need to pass this cookie back to the server in subsequent requests to maintain state or to keep a session alive.

Here is an example of how to deal with cookies when doing your HTTP Post.


First, lets import the modules we will use:


import urllib
import httplib2


Now, lets define the data we will need: In this case, we are doing a form post with 2 fields representing a username and a password.


url = 'http://www.example.com/login'   
body = {'USERNAME': 'foo', 'PASSWORD': 'bar'}
headers = {'Content-type': 'application/x-www-form-urlencoded'}


Now we can send the HTTP request:


http = httplib2.Http()
response, content = http.request(url, 'POST', headers=headers, body=urllib.urlencode(body))


At this point, our "response" variable contains a dictionary of HTTP header fields that were returned by the server. If a cookie was returned, you would see a "set-cookie" field containing the cookie value. We want to take this value and put it into the outgoing HTTP header for our subsequent requests:


headers['Cookie'] = response['set-cookie']

Now we can send a request using this header and it will contain the cookie, so the server can recognize us.



So... here is the whole thing in a script. We login to a site and then make another request using the cookie we received:


#!/usr/bin/env python

import urllib
import httplib2

http = httplib2.Http()

url = 'http://www.example.com/login'   
body = {'USERNAME': 'foo', 'PASSWORD': 'bar'}
headers = {'Content-type': 'application/x-www-form-urlencoded'}
response, content = http.request(url, 'POST', headers=headers, body=urllib.urlencode(body))

headers = {'Cookie': response['set-cookie']}

url = 'http://www.example.com/home'   
response, content = http.request(url, 'GET', headers=headers)
#    Comments [2] |