Geo Location Mashup | Python, Yahoo AJAX API

Geo Location Mashup | Python, Yahoo Maps AJAX API

Corey Goldberg - April 2007

This is a tutorial/example showing how to create a geolocation mashup by generating HTML/JavaScript code from a Python script. The resulting code is an HTML page with embedded JavaScript that you can open with your browser. It works with the Yahoo Maps AJAX API to plot markers at specified locations. I will also explain how this technique can be used to create a [near] real-time map of user concentration based on IP addresses.

Yahoo Maps AJAX API

The Yahoo Maps AJAX API is one of the options available from the Yahoo Maps Web Services Developer API's. Why Yahoo Maps? Because every mashup I see out there uses the Google Maps API. I wanted to try something different, I like the look of the Yahoo maps, and the Yahoo API's are very easy to use and well documented.

From Yahoo:
"The Yahoo! AJAX Maps API lets developers add maps to their web sites using DHTML and JavaScript. Maps are fully embeddable and scriptable using the JavaScript programming language. Yahoo! Maps API's built-in geocoder means that you can specify a physical address or latitude/longitude coordinates for your map's location, as you like."

We want to create a map of the US and plot our marker image at specified locations. We use the createYahooMarker method to create a new YMarker object for each data point, and add it to our map using the addOverlay method. This is where we define the size of the marker bubble and its location on the map. The API is pretty flexible and offers several options for specifying locations, including city/state, metro, address, zip code, etc:

map.addOverlay(new createYahooMarker('Boston, MA', 13))

(For more info, see the (Version 3 API Reference)

Python

The mashup I describe later needs a pre-processing step that parses and aggregates a lot of data. Python is a scripting language well suited for this. At the end of the script's processing, I just generate the markup (and embedded code) necessary to work with the API.

The Marker Bubble:

Static GIF image that gets resized and added as an overlay to be rendered on the map when the page loads:

marker bubble

Simple Geolocation Maps:

Hybrid Satellite/Map view with markers placed at specified locations
Since we are using the AJAX control, our map has a zooming, panning, dynamic, tiled interface

(click maps for actual map/interface)

The Code:

geo_loc.py

This script generates the HTML/JavaScript (AJAX) necessary to render our map.


#!/usr/bin/env python
# Corey Goldberg, April 2007 (corey@goldb.org)

marker_size = 13
zoom_level = 14
height = '550px' 
width = '800px' 
center = 'Topeka, KS'  # city or zip
map_type = 'YAHOO_MAP_HYB'  # YAHOO_MAP_REG, YAHOO_MAP_SAT, YAHOO_MAP_HYB 

fh = open('geomap.html', 'w')

fh.write("""\
    <html>
    <head>
        <title>Geolocation Map</title>
        <script type="text/javascript" 
            src="http://api.maps.yahoo.com/ajaxymap?v=3.0&appid=YahooDemo">
        </script>
        <style type="text/css">
            body { padding: 0; margin: 0; }
            #mapContainer { height: %s; width: %s; }
        </style>
    </head>
    <body>
    <div id="mapContainer"></div>
        <script type="text/javascript">
            function createYahooMarker(geopoint, size) { 
                var myImage = new YImage();
                myImage.src = './marker.gif';        
                myImage.size = new YSize(size, size);
                var marker = new YMarker(geopoint, myImage);
                return marker; 
            }
            var map = new  YMap(document.getElementById('mapContainer'), %s);
            map.addPanControl(); 
            map.addZoomLong(); 
            map.drawZoomAndCenter("%s", %d);""" 
    % (height, width, map_type, center, zoom_level))

for location in open('locations.txt', 'rb').readlines():    
    fh.write("map.addOverlay(new createYahooMarker('%s', %s))\n" % 
        (location.strip(), marker_size))
        
fh.write("""\
        </script>
    </body>
    </html>""")

Input File:

locations.txt

This is the data file we use in the script above (newline delimited ascii text file).

02116
Dallas, TX
Los Angeles, CA
Mexico City
Havana
03102
Providence, RI
Albany, NY
Worcester, MA
Toronto
Edmonton
Calgary
Atlanta
Ottawa
Augusta, ME
Philadelphia
Richmond, VA
Seattle

The Mashup

Now that I have shown how to build a simple map with image markers from a data source, we can use this same technique to create a [near] real-time map of user concentration based on IP addresses.

Data Source

To do a geolocation mashup, we first start with a data source that shows where each user's requests come from. In my case, I get this data directly from a monitoring appliance (Coradiant TrueSight) that does the IP to city/metro correlation for me. All I have to do is grab this data at regular intervals and do some processing to create my data source. Each time I generate the data, I generate a new HTML/JavaScript (AJAX) document with the new data points. I add a meta-refresh to the page and this updates regularly, giving the effect of a map showing real-time user concentration/distribution.

Server Logs

You can also obtain IP addresses of user requests from your server logs. I use a small parsing script to pull out the IP addresses to use as input data.

IP Geo Lookup

IP Geolocation data is available from various sources; some free and some commercial. Hostip.info is a community-based project to geolocate IP addresses, making the database freely available. We can use the Hostip API to do our geolocation lookups. We provide an IP address and use the API to translate it into a city/location. Sample code for doing this is shown below.

Simple Client For Geolocation Lookups Against The Hostip.info API:

We start with a list of IP Addresses and convert them to cities/locations by sending a request to the web service API and parsing the response for relevant data.

    
#!/usr/bin/env python
# Corey Goldberg, April 2007 (corey@goldb.org)

import urllib
import re

ip_addresses = (
    '165.123.243.168',
    '141.158.8.9',
    '198.190.156.119',
    '192.234.106.2',
    '66.151.227.20',
    '12.147.195.3',
)

for ip in ip_addresses:
    response = urllib.urlopen('http://api.hostip.info/get_html.php?ip=%s' % ip).read()
    m = re.search('City: (.*)', response)
    if m:
        print m.group(1)

Sizing Our Markers Based On User Concentration

When we place markers on the map, we specify the size:

map.addOverlay(new createYahooMarker('Boston, MA', 40))
map.addOverlay(new createYahooMarker('Manchester, NH', 20))
map.addOverlay(new createYahooMarker('Dallas, TX', 30))

So the basic approach is to aggregate the data and assign a size value to each marker. We want to render a marker bubble on the city/location with a size that is relative to the aggregate value we assign. To do this, I create an algorithm that chooses a size for each marker based on thresholds I supply.