goldb.org home

AS OF MAY 2008, THIS BLOG IS NO LONGER BEING UPDATED.
Visit the new blog at: http://coreygoldberg.blogspot.com



 Thursday, March 08, 2007

PLEAC - Programming Language Examples Alike Cookbook

I just stumbled across the PLEAC Project (Programming Language Examples Alike Cookbook).

Project Description:

"Following the great Perl Cookbook (by Tom Christiansen & Nathan Torkington, published by O'Reilly; you can freely browse an excerpt of the book here) which presents a suite of common programming problems solved in the Perl language, this project aims to gather fans of programming, in order to implement the solutions in other programming languages."


There is sample code in many popular languages.  The Python examples are really good.  They would serve as an excellent primer for someone moving from Perl to Python, or as a general Python reference with cookbook-style examples.

It is hosted at SourceForge and licensed under the GNU Free Documentation License (GFDL).

#    Comments [0] |
 Wednesday, March 07, 2007

Play Corey's Tunes - Last.fm

I got hooked on Last.fm last summer.  Since then, I've scrobbled over 6,800 tracks.. not bad!  (I've been a music junkie my entire life).

I submit my played tracks from all of my music players (Foobar2000, Winamp, iTunes, Squeezebox/SlimServer).  It populates fantastic statistics and charts of my listening habits and lets me listen to streams from people with similar tastes.

Here are my top 10 artists since I started using it:


Once you scrobble enough tracks, you can start streaming custom stations based on your listening habits.  Something new they just added is the ability to embed a Last.fm player widget into your own site. I just knocked up a quick web page with the embedded player so I can listen to my own station from anywhere that has a browser (requires Flash).  Check it out and have a listen.

#    Comments [0] |
 Tuesday, March 06, 2007

Show Us Your Rack!

Reverend Ted just started the "Show Us Your Rack" blog campaign ;)

Well.. I've seen a few nice racks in my day.  I took these pics last year.  Forget the tiny colo cage, this data center gives you some room to spread out:


plenty of room:


looking down the aisle:


got load balancing?:


the big dogs:

#    Comments [0] |
 Monday, March 05, 2007

.NET CLR - Covertly Throttling Thread Creation

Joe Duffy (from Microsoft) talking about the .NET 2.0 CLR:

"It's also worth noting that the threadpool throttles its creation of threads to 2/second once the count has exceeded the # of CPUs."


Yuck...  I don't like throttling like that behind the scenes.  It can make performance problems *very* hard to diagnose.

#    Comments [2] |
 Sunday, March 04, 2007

Python Parameters - Pass-By-Value or Pass-By-Reference?

Passing parameters to functions and methods.  Pass-by-value?  Pass-by-reference?  Which does your language use?

You probably learned this in your first CS class... so did I.

Then why did it take me a frakin' month to understand what Python does? :)

Well... if you look online, you will find some very ambiguous answers about Python being pass-by-reference or pass-by-value.  (which ends up boiling down to semantics and how you use certain terminology, but forget that for now)


To review, how do other languages handle this concept?

C is pass-by-value

Straightforward. You can simulate pass-by-reference with pointers.  Not much else to say here.

Java is pass-by-value

Primitive Types (non-object built-in types) are simply passed by value.  Passing Object References feels like pass-by-reference, but it isn't.  What you are really doing is passing references-to-objects by value.

OK, so what about Python?

Python passes references to objects by value (like Java), and everything in Python is an object. This sounds simple, but then you will notice that some data types seem to exhibit pass-by-value characteristics, while others seem to act like pass-by-reference... what's the deal?

It is important to understand mutable and immutable objects. Some objects, like strings, tuples, and numbers, are immutable.  Altering them inside a function/method will create a new instance and the original instance outside the function/method is not changed.  Other objects, like lists and dictionaries are mutable, which means you can change the object in-place.  Therefore, altering an object inside a function/method will also change the original object outside.



For entirely too much information about this topic in Python and across many other languages (Java, Scheme, C#, C, C++, Python), read the thread where these quotes come from:

Is Python By Value Or By Reference?

Alex Martelli:

The terminology problem may be due to the fact that, in python, the value of a name is a reference to an object. So, you always pass the value (no implicit copying), and that value is always a reference.
[...]
Now if you want to coin a name for that, such as "by object reference", "by uncopied value", or whatever, be my guest. Trying to reuse terminology that is more generally applied to languages where "variables are boxes" to a language where "variables are post-it tags" is, IMHO, more likely to confuse than to help.

Michael Hoffman:

Alex is right that trying to shoehorn Python into a "pass-by-reference" or "pass-by-value" paradigm is misleading and probably not very helpful. In Python every variable assignment (even an assignment of a small integer) is an assignment of a reference. Every function call involves passing the values of those references.


word.

#    Comments [0] |

Many Hats of a Performance Engineer

The many hats of a performance engineer:

  • tester
  • developer
  • toolsmith
  • dba
  • sysadmin
  • network/ops/tech
  • system integrator
  • architect
  • statistician
  • data visualizer
  • sadist



#!/usr/bin/env python


def main():
    me = PerformanceEngineer()
    print 'just another %s:' % me.role
    print me.skillz



class PerformanceEngineer(object):
   
    role = 'performance engineer'
    skillz= (
        'tester',
        'developer',
        'toolsmith',
        'sysadmin',
        'network/ops/tech',
        'system integrator',
        'architect',
        'statistician',
        'data visualizer',
        'sadist'
    )
  
    def __init__(self):
        pass      



if __name__ == '__main__':
    main()
#    Comments [0] |
 Wednesday, February 28, 2007

One Laptop Per Child - It's All About the Python!

Wow. I just read something interesting about the One Laptop Per Child (OLPC) project in Guido's PyCon writeup:

"The software is far from finished.  An early version of the GUI and window manager are available, and a few small demo applications: chat, video, two games, and a web browser, and that's about it!  The plan is to write all applications in Python (except for the web browser), and a "view source" button should show the Python source for the currently running application.  In the tradition of Smalltalk (Alan Kay is on the OLPC board, and has endorsed the project's use of Python) the user should be able to edit any part of a "live" application and see the effects of the change immediately in the application's behavior."


So... they are going to be running a GNU/Linux OS (a stripped down version of Fedora), with essentially all applications in Python.

This is very cool on many levels. It is the ultimate endorsement of Python.  It also makes me think about the future...  If OLPC is successful, a few years down the road we might be looking at several million young new Open Source/Python hackers.  Nice!


#    Comments [0] |

Reading Outlook/Exchange Email Programatically with Python

With Python's Windows Extensions, you can talk via COM to an Exchange Server and read/process your email.  You must have the Outlook Client installed on the box you are running this from.

Here is a sample script that will:

  • connect to your mailbox
  • print the inbox name
  • print the message count
  • print the subjects for all your email messages


#!/usr/bin/env python

from win32com.client import Dispatch

session = Dispatch("MAPI.session")
session.Logon('OUTLOOK')  # MAPI profile name
inbox = session.Inbox

print "Inbox name is:", inbox.Name
print "Number of messages:", inbox.Messages.Count

for i in range(inbox.Messages.Count):
    message = inbox.Messages.Item(i + 1)
    print message.Subject


#    Comments [0] |
 Monday, February 26, 2007

Python 3000 Video and Slides

Guido van Rossum just published his slides from the PyCon 2007 Keynote where he discusses Python 3000.

His talk is also available on Google Video.

I'm psyched to watch this!

#    Comments [0] |

Boston - Back Bay Snow Pics

New England weather is crazy. In Boston, we get pounded with snow some winters.  Other winters we hardly get any of the white stuff.  This year has had really minimal snowfall, but it was coming down this morning.

I snapped a few pics on my way to work (I live in Back Bay).

Looking out my apartment window:

Looking out from my front stoop:

The view down Comm. Ave:

Copley Square, Old Trinity Church, John Hancock Building, Boston Public Library:


#    Comments [2] |
 Sunday, February 25, 2007

Wes Dyer on Type Systems

Awesome post by Wes Dyer about Type Systems:
Types Of Confusion


He tackles a lot of misconceptions and myths about Typing.

To frame his points, he starts with some fantastic definitions.  I am all for clarifying vocabulary and I really like this:


Based on the apparent confusion, I think it is best to clarify what I mean by each of the following terms:

    Type Checking - Verifying that code respects type constraints.

    Statically Typed - Type checking occurs at compile time.

    Dynamically Typed - Type checking occurs at run time.

    Type Safe Language - A language which protects its own abstractions.

    Type Unsafe Language - A language which is not type safe.

    Strongly Typed and Weakly Typed - Depends on the author; The definitions are so many and so varied that the terms are practically useless. It seems that anyone can claim that language X is either strongly typed or weakly typed based on sound reasoning derived from one of the various definitions.

    Dynamic Language - A language which enables runtime inspection or modification of a program; most languages can do this but dynamic languages make it easy. It is common for people to refer to "dynamic languages" and mean "dynamically typed languages" as the term is defined here.

good read.

#    Comments [0] |

How To See Your Swallowed Exceptions In .NET (Visual Studio debugger)

Good to know..


See all the exceptions you are swallowing in .NET (Visual Studio debugger):

Turn on 'all exceptions' and watch the fireworks fly


One of the most glaring differences for me between Java and .NET is the difference between Checked/Unchecked Exceptions, so stuff like this has been helpful to me in figuring out how exception handling in C#/.NET really works.

#    Comments [0] |

4-Space Indents - 4Eva!

I personally use 4-space indents when I write code (in any language, period).


Oddly... in Python, where whitespace matters, there is no single common practice (which would fit in nicely with Python's TOOWTDI ideology).

Some observations of source code indentations in Python:

  • I mostly see 4-space indents in code I read (colleagues, libraries, 3rd party code)
  • Python's own source (the Python parts) use 4-space indents
  • Google uses 2-space indents in their Python
  • I hate tabs (if I randomly sucker-punch you, this is why)


Guido van Rossum (creator, lead developer, and BDFL of Python) writes

"If it uses two-space indents, it's corporate code; if it uses four-space indents, it's open source. (If it uses tabs, I didn't write it! :)"
#    Comments [0] |

IronPython Community Edition - Free IronPython

IronPython is the Python implementation that runs on the .NET platform... originally created by Jim Hugginin, but later backed (overtaken?) by Microsoft.

Microsoft's ambivalence towards Free software is a bit hard to follow sometimes and it really makes me question their entire approach to the software community  (wait.. have I ever *not* questioned that? :).

As a Python advocate and *nix geek trying to make my way working in a .NET shop, I am really excited about IronPython.  I was also initially impressed with Microsoft's embracement of the Python community and toe dipping into Open Source.  But then I hear Microsoft will not take patches from non-Microsoft developers and will not bundle IronPython with other applications which have certain Free licenses(LGPL, BSD). To me, this is really a shame. That is not how you approach a community.

Well... at least somebody has stepped up and is maintaining IronPython Community Edition (IPCE).

props.


So check out IPCE and the FePy Project!

#    Comments [0] |
 Saturday, February 24, 2007

The Web as a Data Integration and Machine-Oriented Publishing Layer

This is what I was talking about when I wrote:

"one of the most attractive things about the Web is the ability to use HTTP as a simple transport protocol abstraction.  [...]  with this additional transport abstraction in place, you can build another application layer protocol on top of this and use that as your API for distributed operations.  That is where the rubber meets the road in modern large scale systems, and that is where the action is taking place in the current debate about SOA, REST, Web Services, and distributed architectures.  Furthermore, the foundation for this style is built directly into HTTP 1.1."

Bill de hÓra states it better in his "Confederacy" post:

"the Web is not just the presentation tier anymore; it's becoming a data integration and machine-oriented publishing layer.  The presentation layer is being pushed down to the client machine in the form of AJAX, XUL and Flex."

A new web/middleware layer has been forming, and this is the engine that is driving Web 2.0 and creating a new level of integration and interoperability.

#    Comments [0] |
 Friday, February 23, 2007

“Humanity Lobotomy” - Net Neutrality Open Source Documentary

Spread Awareness.

Awesome new video (via Lessig): 

“Humanity Lobotomy” - Net Neutrality Open Source Documentary


What a great month for videos!  I feel inspired to be working in technology again.  We control the future.

Check out more here

#    Comments [0] |

New Degree - Psyched About A Piece Of Paper

A little while back I wrote about finishing my Master's Degree at Boston Universtiy. 

Well... this bad boy arrived in the mail today:



I guess I am now officially educated... or something.



O'Toole's Corollary of Finagle's Law: “The perversity of the Universe tends towards a maximum”
#    Comments [3] |

Open Source Feedback and Participation (The WebInject Story)

I posted a lengthy entry to the "getting feedback on tools" thread going on at the toolsmith-guild group

The thread is discussing how to get feedback and participation going in test tools projects.  This experience report sheds some light on this, but more generally can be applied to getting your open source project noticed and used.  I thought other open source developers might find this useful.


Here is my post:



Hi.. new to the list.

I have some experience in this area I want to comment on:

I had a specific need for a tool to test web services a few years ago ('02-'03). I had been rolling my own tools in various languages for quite a while and had some stuff that I thought others might be interested in; so I setup a SourceForge project and released it in Jan'04. The tool is called WebInject. Not many people used it at first. I just kept adding features and releasing. I was scratching my own itches and using my tool internally at a company I was working for. Eventually I got a few bites and some people started to download it and try it out.

As of today, the project site has served over a quarter million pageviews and the tool has been downloaded over 30,000 times. I've had feedback from all corners of the world.

See the Download Stats

In the early days, *any* bug that was reported, I would dive into immediately.. I was turning around fixes and new releases in crazy time. This personal attention sparked a lot of lengthy email conversations where I was starting to get real feedback. It wasn't the case where I could just sit back and expect quality feedback. I really engaged the users and asked questions. Pretty soon I was getting more feedback than I could reasonably handle. Around this time I started getting some patches sent to me. This accelerated development as you start to get various ideas from other people. A few silly bugs were also cleaned up.. stuff I never would have caught on my own. Encouraging patches is huge.. once someone gets a piece of their own code into your code base, they have an intimate connection.

I was also contacted by someone writing a book that wanted to use my tool as an example. I was really excited about this and put in some serious time getting the tool cleaned up and more useful. I never saw a published copy, but apparently it exists (I saw the PDF drafts :) It was also mentioned in a few testing magazines and articles which was cool for me to see.

I don't maintain WebInject much any more. I think it is cool and useful, but it's a rats nest of Perl code that needs some serious love. There is some stuff that bothers me so much about it that I barely want to touch it sometimes :) I spend all my time on newer tools these days, but I still keep on top of it enough to facilitate others using it and posting patches/updates to it (though I haven't released in over a year I think). Oddly, it has become somewhat entrenched in monitoring systems. Most of the users lately seem to be people running Nagios (open source monitoring system) that need an intelligent web plugin/agent.


Some basic thoughts on open source tool adoption and getting feedback:

  • A tool without documentation doesn't exist.. without quality documentation, don't even bother.. nobody will touch it.
  • If it takes someone more than 10 minutes to install, configure, run, and see the results from a simple test case.. nobody will use it. Once someone sees something work, it is tangible and they become engaged. I achieved this with my tool.. if you download and run it, you are presented with a GUI. Pressing "run" executes a preconfigured sample test case that hits my site. I wanted the most clueless of people to be able to immediately see a green "PASS", and say "hey, cool it actually did something". This satisfaction must come immediately or it will be abandoned. (unless of course you already have a user base that has made it past this hurdle and can encourage others to put in the effort to get it working)
  • Make your tool easy to get. The online presence of your tool should allow someone to visit the website and immediately download it. I have seen tools released that take 5 levels of navigation before you get to the download options, or don't even have a website or project page somewhere.
  • Announce your tools.. release early, release often.. who cares.. blog it, forum it, email it, digg it, usenet it, comment about it.. see if anyone has interest. Don't annoyingly spam it, bet let people know it exists. If you announce your tool once only, buried deep in a technical mailing list, don't expect people to come rushing.
  • Encourage participation. Go out of your way to let people know that you encourage and appreciate feedback, bug reports, offers for collaboration, patches, etc.
  • You need a forum or a mailing list with a web accessible archive. It has to be super simple to talk to you in public and let others see answers to questions and begin to participate on their own and to offer advice. Using a many-to-one scheme where you just tell people to email you isn't gonna work. It is too private.. people want to know what is going on and hear from each other and offer tips. I have had several thousand posts to my forums for this little tool.


- Corey Goldberg


#    Comments [0] |

Developer + Tester == Develester ?

I saw this posted in the toolsmith-guild group (Danny Faught):

"The developers were very surprised to find a whole room full of testers who didn't cringe at the thought of reading and writing code. The rather odd terms Developer-Tester/Tester-Developer emerged from AWTA." (Austin Workshop on Test Automation)

ahh, the elusive "develester"


... and.. the develester algorithm in Python :)

#!/usr/bin/env python

def what_am_i(skills):
    if 'developer' in skills:
        role = 'developer'
    if ('tester' in skills):
        role = 'tester'
    if ('developer' and 'tester') in skills:
        role = 'develester'
    return role
        

skills = 'developer-tester'
print 'you are a %s' % what_am_i(skills)
#    Comments [0] |

Google Apps Premier - The Office Battle Is On

Last year, I made a prediction/bet that Google was gonna make a huge push into office applications and we were gonna see the MS Office monopoly start to erode.  Well, its on!  We actually have quite a cool phenomenon brewing, with Open Office striking from one side and online office apps striking from the other.

It is a classic disruptive play.. still far from a tipping point, but serious shots were fired over Microsoft's bow.   Google is going at it pretty aggressively too.  I just got an invite for a seminar in Boston that explains the new enterprise office tools (Google@Work Seminar).

It will be interesting to watch this unfold.

#    Comments [3] |
 Thursday, February 22, 2007

64-bit WEP Is Better Than 128-bit WEP

I am setting up a new wireless network tonight and enabling WEP encryption on my cheapo router.

When using WEP, you have a choice of key strengths: 64-bit and 128-bit

In the past I always chose 128-bit encryption because it is "stronger".  This time I went with 64-bit, and here is my reasoning:

All I care about is keeping away casual snoopers and freeloaders from my wireless connection.  I live in a dense part of Boston and can pick up 10+ wireless signals from my apartment at any given time, so this is likely.  Anybody with skills who seriously wanted to penetrate my network could do so with either key strength. Cracking the 128-bit encryption isn't much more of a hurdle than the 64-bit, so why bother?

Why slow down all your packets, waste your batteries (decryption is processor hungry and therefore laptop/battery hungry), and have a longer key to keep track of, when the added security it gives you doesn't exist?

(note: If I wanted to block anything more than casual usage, I wouldn't use WEP in the first place)

#    Comments [0] |

Joe Gregario on MOM vs. RPC

RPC: Remote Procedure Call
MOM: Message Oriented Middleware

Both RPC and MOM are communication models for distributed systems.  Each has strengths and advantages. However, when you get into large heterogenous distributed systems, message passing is the way to achieve scalability.

I like this quote:

"In a large system you may be faced with either a multitude of clients or a menagerie of them; in either case you have to stop serializing objects and start exchanging documents."
- Joe Gregorio, 2007
#    Comments [0] |

Not Using ASP.NET Session State? Then Turn It Off

I am developing some small ASP.NET 2.0 web applications.  They are stateless and I am not doing anything with Session State.  However, I noticed that ASP.NET enables Session State by default (In-process mode is the default setting).  Therefore, if you have a truly stateless site or application, session state does nothing more than slow down performance.

In-process session state is still relatively fast, as the memory used to handle session is allocated by the same process on the local machine (no cross-process calls or data marshaling).  But this is needless overhead if you are not using your Session State.

So... to turn it off for the whole application, add the following line to your web.config, inside the system.web section:


<sessionState mode="Off" />


#    Comments [0] |
 Tuesday, February 20, 2007

Compiling Python Scripts to Windows Executables

I often write quick Python scripts that I need to run on other machines. It is sometimes easier to just drop a windows .EXE onto a machine (with a Python Interpreter compiled into it), rather than doing a full Python installation. To do this, I use py2exe

py2exe is a Python Distutils extension which converts Python scripts into executable Windows programs. This enables your Python scripts to be run on Windows platforms without a Python installation.

You can run py2exe directly from the command line, or you can script it. I wrote a small convenience script that I use for general compilation.

Let's call the compilation script: compile.py
Let's say we have a script we want to compile named: foo.py

You would then invoke it from the command line like this:

>python compile.py foo.py

This will create a 'dist' subdirectory containing the newly created executable along with some necessary DLL's.


Here is the code I use for my compile.py:


#!/usr/bin/env python
# Corey Goldberg

from distutils.core import setup
import py2exe
import sys

if len(sys.argv) == 2:
    entry_point = sys.argv[1]
    sys.argv.pop()
    sys.argv.append('py2exe')
    sys.argv.append('-q')
else:
    print 'usage: compile.py <python_script>\n'
    raw_input("press ENTER to exit...")
    sys.exit(1)

opts = {
    'py2exe': {
        'compressed': 1,
        'optimize': 2,
        'bundle_files': 1
    }
}

setup(console=[entry_point], options=opts, zipfile=None)

(note: you need to have Python and py2exe installed on a Windows box to run this)

#    Comments [0] |
 Saturday, February 17, 2007

The Summer That Mr. Gates Retired and Mr. Stallman Didn't

Eben Moglen [2006]:

"This will be remembered in history as the summer that Mr. Gates retired and Mr. Stallman didn't"

Another excellent and inspiring speech by Moglen


.. where he reminds us of the inevitable, unavoidable, endgame:

"Software that can do everything, runs everywhere, needs no additional testing, just works, can do anything human beings want, and costs $0.00 per unit."


It eludes me why people don't seem to grasp this:

"Excluding people from ideas works wells only if you think of ideas as something only one person at a time has.

When you live in our world and you know that ideas are created by people cooperating, sharing, yelling, screaming, waving their arms and typing code at 2 in the morning; the 20 year old monopoly on the ownership of an idea is an abomination."


(but Eben, tell us how you *really* feel :)


#    Comments [0] |