Yahoo! Pipes could flow to exciting new places with web hooks

Yahoo Pipes Plus web Hooks

Sucking from the pipe...

Yahoo! Pipes, the excellent visual data mashup creator, has an unfortunate blockage problem. Specifically, it can pull in data from numerous sources, but the flow stops at the end of the pipe, waiting to be sucked out and consumed instead of automatically flowing out to its final destination.

When you create a Yahoo! Pipe, you can choose to consume sources such as RSS feeds, CSVs, or even regular web pages. You can then sort, filter, and generally mash up this data to create your own output feed.

Unfortunately, this is where the flow in Yahoo! Pipes stops. The output feed you create is made available in various formats, including RSS, JSON, and PHP. All formats that you need to pull in to consume.

... versus letting the pipe flow.

What Yahoo! Pipes could do is go one step further and use HTTP and the concept of web hooks to push the data out to custom endpoints. In other words, it could keep the data flowing.

In other words, once I have consumed my various data sources, mashed them up, and created an output feed, I should be able to push that to an HTTP endpoint on my web application. As the data changes (based on the frequency that Yahoo! updates the pipe), my application gets a notification and the latest data is pushed to it, removing the need for me to implement polling in my application and thus greatly simplifying things at the application level.

If you'd like to learn more about web hooks, Jeff Lindsay, recently gave a talk introducing web hooks at the <head> web conference in a presentation titled Web Hooks and the Programmable World of Tomorrow (<head> 2008 attendees can watch the video of Jeff's talk here).

Comments