Geographic Processing of Feeds

RSS feeds are ubiquitous. They are emerging as one of the most effective ways of keep track of changes to web resources.

They also illustrate the emergence of pull http://www.itworld.com/AppDev/nlsebusiness070410/index.html vs push (exemplified by email notification). Pull-centric designs are easy to implement and fault tolerant.

Add geographic location into the mix of standard RSS metadata and you have GeoRSS. GeoRSS has been embraced by the top 3 search engines (in Google Maps, Microsoft Live Local, and Yahoo Maps and Pipes) and is used in traffic, emergency response, geological, and social networking applications. In the near future, GeoRSS will be one of the primary means (KML the other) for telling users where their resources (pages, services) are concerned.

Pipes

Yahoo Pipes is a ground breaking web application for remixing RSS feeds, allowing users to find relationships between different sites and services, and most interestingly: without ever leaving the Web. To use a pipe you pull on its URL and that pipe in turn pulls on other source feeds, ultimately returning a new RSS feed. Neither you nor the data have ever left the Web.

RSS feeds are part of design for serendipitous reuse. Pipes is an engine for serendipitous reuse. RSS in, RSS out.

Here is a canonical location aware Pipe: Apartments Near Something.

Mush

The geo-processing capabilities of Pipes are meager, limited to "x within d(istance) of y" expressions. There's no capacity for processing lines or polygons, or creating new locations of other than point type.

To explore greater capabilities the Pleiades project has developed Mush, named after the command one yells to urge dogs to pull a sled. Mush has 2 simple built-in processes (or pipelines):

• Self Intersection
• Pair Intersection

These processing services buffer the locations of RSS feed entries by some amount and then return the sphere of influence intersections as entries of a new feed.

Example 1

Intersecting spheres of influence of human settlements in ancient Lycia and Pisidia attested to be occupied during the Classical period: [result feed, result map].

Example 2

Here's an application of Mush that might be useful to an antiquist: UK planning alerts within 1 kilometer of Celtic coin finds in Cambridgeshire [result feed, result map].

Ahead

Mush can benefit digital humanities projects in 2 ways:

• As a service hub, like a Pipes for the humanities;
• As open source software that could be easily deployed for special needs.

Features that Mush needs:

• KML processing (in/out);
• User-created pipelines;
• A small language for authoring pipelines (Javascript, most likely);
• Higher performance;
• Support for processing large feeds through callbacks.
• Resolution of locations linked to entries.