Image may be NSFW.
Clik here to view.
Yahoo Pipes is a visual editor of mashups, allowing you to take data from sources on the net, transform them in various interesting ways and output the result as Atom, RSS or JSON. The primary downside Pipes of course is that you’re totally dependent on Yahoo for the infrastructure: it runs at Yahoo pulling feeds that have to be accessable through the public Internet.
It’s easy to use Pipes: just go to this page and start working with the sample example Pipe. You’ll need a Yahoo login ID, but most of us have that anyway. I’ve created an example that uses Yahoo Pipes to feed a Djolt template which you can see here.
We can analyze Pipes in the terms of the DQT paradigm we’ve outlined in the previous post.
Data Sources and Queries
Sources and Queries are merged (quite logically) in the Pipes interface. You can read in depth documentation here.
- Fetch CSV
- Feed Autodiscovery – outputs syndication feeds found on a page (RSS feeds on a CBC page)
- Fetch Feed
- Fetch Page – will read a page and parse the contents with a reg
- Fetch Site Feed – this is the logical combination of Fetch Feed and Fetch Autodiscovery
- Flickr – find images by tag near a location (photos of cats in Toronto)
- Google Base – look up information in Google Base
- Item Builder – a way of building new items from existing items
- Yahoo Local
- Yahoo Search
Transforms
The operator documentation can be read here.
- Count
- Filter
- Location Extractor – a geocoder that magically looks for locations
- Loop
- Regex
- Rename
- Reverse
- Sort
- Split
- Sub-element – pulls a particular sub-element of an item and makes that the item. This is very much like WORK path manipulation
- Tail
- Truncate
- Union
- Unique
- Web Service
Plus a number of specialized data services, for dealing with elements such as dates.
Templates
Pipes does not provide an arbitrary Djolt-like template producing HTML. Instead, they provide a number of pre-made code templates that output well known data types, including RSS, JSON and Atom (and some stranger choices, like PHP).
