Yahoo! Pipes to the rescue. The JSONator: XML / RSS / ATOM to HTML with no server-side proxy.

We at GateHouse Media have lots of bloggers. We also have lots of newspapers. Naturally, we like to drive traffic in full-duplex. Unfortunately, because we have so many data 'sources' and 'destinations' always changing, it often becomes difficult for us to keep up with the massive amount of new blogs on a daily basis. We sought out to find a more dynamic process by which we can transport 'teases' of our blogs to our sites.

Enter Yahoo! Pipes. 'Rewire the web', they say, with their data processing tools made available to the public. Our main goal? To turn XML / RSS / ATOM into a usable format we can quickly and securely transport across our network of sites over the web, without server-side processing help.

Enter JSON. According to the guys at JSON.org:

Perfect! That's exactly what we need. JSON is easy to parse with JavaScript, and immediately becomes our format of choice for moving data between our sites. It also just so happens that any Yahoo! Pipe that you create can be exported as JSON. Awesome. So we wire up a pipe that takes a query string variable (which we'll put our blog feed URLs in), and grab the URL to the JSON'd feed. We now have our JSON data source, completely dynamic with whatever supported feed we stuff into the query string variable. Clutch.

Now all we need to do is process the JSON and turn it into HTML. The end-users of this script, most likely website editors or owners, will have a few basic needs for pulling feeds into their site. That said, there are a variety of possibilities for handling these feeds. So I wrote a quick script that will take a simple JSON object of META data describing the feed and it's properties. This is what the script will take as arguments:

var blogFeed = ProcessFeeds(
  {
    'feeds': [
      {
        'urls': [
          {'url':'http://blogs.gatehousenewsservice.com/suffragettecity'},
          {'url':'http://blogs.e-rockford.com/applesauce/feed/atom/'},
          {'url':'http://blogs.townonline.com/holmesandco/?feed=atom'}
        ],
        'handler':'list-titles',
        'target':'blogs_dump',
        'items':'3'
      },
      {
        'urls': [
          {'url':'http://pipes.yahoo.com/pipes/pipe.run?_id=ZF68eSDb3BGERhonLO2fWQ&_render=json&_callback=Transport'}
        ],
        'handler':'list-titles',
        'target':'pipe_dump',
        'items':'5',
        'type':'json'
      }
    ]
  }
);

All you'll need to do is include one JavaScript file (forthcoming), and declare that variable somewhere in your page. That variable up there says a few things:

• Feeds: Specify the feed URLs like you see there.
• Handler: This will specify how you'll format the output.
• Target: The ID of the DOM element where you want the output to go.
• Items: The number of items you want to grab from each feed.
• Type: In the case of feeding another Yahoo! Pipe feed, you'll need to use the JSON output of Yahoo! Pipes and use this flag to tell the script the feed is already in JSON format.

It really is that easy. It's also easy to extend the script. Adding additional handlers is intended to be straightforward and simple.

Here's the script:

http://global.static.ghm.zope.net/resources/global/jsonator.js

It does a few things. Most notably, it attempts to regulate the processing of the HTML until after all of the JSON feeds have been loaded. I need to find a new way to test the loaded feeds against the source feeds, but for now, it works fine as is. I'm no JavaScript guru, though, so all comments and suggestions are greatly appreciated.

The script provides one handler by default, the 'list-titles' handler, which will output the feeds into a listed format like so:

Of course, creating an additional handler is as simple as creating an additional 'case' in the 'Handler()' function, like so:

case 'new-handler': {
  for (var i = 0; i < argItems; i++) {
    if (feed.value.items[i]) {
      if (html[h] == undefined) {
        html[h] = '';
      }
      itemTitle = (feed.value.items[i].title).replace(/\"/g,'\'');
      html[h] += Process the feed item here!;
    }
  }
  break;
}

Then, just specify your new handler when you call the ProcessFeeds() function in your page.

So that's about it for the script. The script works excellently for displaying multiple feeds on different parts of the page, while only incurring one processing function. Again, all comments and suggestions are greatly appreciated. Thanks!

BTW: Some excellent references:

• Yahoo Pipes--RSS without Server Side Scripts
• Parsing Yahoo Pipes JSON Feeds with jQuery

1 Comment