org.apache.streams.rss.provider

Class RssStreamProviderTask

java.lang.Object

org.apache.streams.rss.provider.RssStreamProviderTask

All Implemented Interfaces:

Runnable

public class RssStreamProviderTask
extends Object
implements Runnable

A Runnable task that queues rss feed data.

RssStreamProviderTask reads the content of an rss feed and queues the articles from the feed inform of a ObjectNode wrapped in a StreamsDatum. The task can filter articles by a published date. If the task cannot parse the date of the article or the article does not contain a published date, by default the task will attempt to queue article.

A task can be run in perpetual mode which will store the article urls in a static variable. The next time a RssStreamProviderTask is run, it will not queue data that was seen the previous time the rss feed was read. This is an attempt to reduce multiple copies of an article from being output by a RssStreamProvider.

** Warning! ** It still is possible to output multiples of the same article. If multiple tasks executions for the same rss feed overlap in execution time, it possible that the previously seen articles static variable will not have been updated in time.

Field Summary

Fields

Modifier and Type	Field and Description
protected Map<String,Set<String>>	PREVIOUSLY_SEEN Map that contains the Set of previously seen articles by an rss feed.

Constructor Summary

Constructors

Constructor and Description
RssStreamProviderTask(BlockingQueue<StreamsDatum> queue, String rssFeed) Non-perpetual mode, no date filter, time out of 10 sec
RssStreamProviderTask(BlockingQueue<StreamsDatum> queue, String rssFeed, org.joda.time.DateTime publishedSince) Non-perpetual mode, time out of 10 sec
RssStreamProviderTask(BlockingQueue<StreamsDatum> queue, String rssFeed, org.joda.time.DateTime publishedSince, int timeOut, boolean perpetual) RssStreamProviderTask that reads an rss feed url and queues the resulting articles as StreamsDatums with the documents being object nodes.
RssStreamProviderTask(BlockingQueue<StreamsDatum> queue, String rssFeed, int timeOut) Non-perpetual mode, no date filter.

Method Summary

Modifier and Type	Method and Description
String	getRssFeed() The rss feed url that this task is responsible for reading.
protected Set<String>	queueFeedEntries(URL feedUrl) Reads the url and queues the data
void	run()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

PREVIOUSLY_SEEN

protected Map<String,Set<String>> PREVIOUSLY_SEEN

Map that contains the Set of previously seen articles by an rss feed.

Constructor Detail

RssStreamProviderTask

public RssStreamProviderTask(BlockingQueue<StreamsDatum> queue,
                             String rssFeed)

Non-perpetual mode, no date filter, time out of 10 sec

Parameters:

queue - queue

rssFeed - rssFeed

See Also:

org.apache.streams.rss.provider.RssStreamProviderTask #RssStreamProviderTask(java.util.concurrent.BlockingQueue, String, org.joda.time.DateTime, int, boolean)}

RssStreamProviderTask

public RssStreamProviderTask(BlockingQueue<StreamsDatum> queue,
                             String rssFeed,
                             int timeOut)

Non-perpetual mode, no date filter.

Parameters:

queue - queue

rssFeed - rssFeed

timeOut - timeOut

See Also:

org.apache.streams.rss.provider.RssStreamProviderTask #RssStreamProviderTask(java.util.concurrent.BlockingQueue, String, org.joda.time.DateTime, int, boolean)}

RssStreamProviderTask

public RssStreamProviderTask(BlockingQueue<StreamsDatum> queue,
                             String rssFeed,
                             org.joda.time.DateTime publishedSince)

Non-perpetual mode, time out of 10 sec

Parameters:

queue - queue

rssFeed - rssFeed

publishedSince - publishedSince

See Also:

org.apache.streams.rss.provider.RssStreamProviderTask #RssStreamProviderTask(java.util.concurrent.BlockingQueue, String, org.joda.time.DateTime, int, boolean)}

RssStreamProviderTask

public RssStreamProviderTask(BlockingQueue<StreamsDatum> queue,
                             String rssFeed,
                             org.joda.time.DateTime publishedSince,
                             int timeOut,
                             boolean perpetual)

RssStreamProviderTask that reads an rss feed url and queues the resulting articles as StreamsDatums with the documents being object nodes.

Parameters:

queue - Queue to push data to

rssFeed - url of rss feed to read

publishedSince - DateTime to filter articles by, will queue articles with published times after this

timeOut - url connection timeout in milliseconds

perpetual - true, if you want to run in perpetual mode. NOT RECOMMENDED

Method Detail

getRssFeed

public String getRssFeed()

The rss feed url that this task is responsible for reading.

Returns:

rss feed url

run

public void run()

Specified by:

run in interface Runnable

queueFeedEntries

protected Set<String> queueFeedEntries(URL feedUrl)
                                throws IOException,
                                       com.rometools.rome.io.FeedException

Reads the url and queues the data

Parameters:

feedUrl - rss feed url

Returns:

set of all article urls that were read from the feed

Throws:

IOException - when it cannot connect to the url or the url is malformed

com.rometools.rome.io.FeedException - when it cannot reed the feed.