public class SysomosProvider extends Object implements StreamsProvider
Modifier and Type | Class and Description |
---|---|
static class |
SysomosProvider.Mode |
Modifier and Type | Field and Description |
---|---|
static long |
API_BATCH_SIZE |
static String |
ENDING_TIME_KEY |
static int |
LATENCY |
static String |
MODE_KEY |
static long |
PROVIDER_BATCH_SIZE |
protected Queue<StreamsDatum> |
providerQueue |
static String |
STARTING_DOCS_KEY |
static String |
STARTING_TIME_KEY |
static String |
STREAMS_ID |
Constructor and Description |
---|
SysomosProvider(SysomosConfiguration sysomosConfiguration)
SysomosProvider constructor.
|
Modifier and Type | Method and Description |
---|---|
void |
cleanUp() |
protected SysomosHeartbeatStream |
createStream(String heartbeatId) |
protected void |
enqueueItem(StreamsDatum datum) |
protected void |
extractConfigFromMap(Map configMap) |
SysomosClient |
getClient() |
SysomosConfiguration |
getConfig() |
int |
getCount() |
String |
getId() |
long |
getMaxApiBatch() |
long |
getMinLatency() |
SysomosProvider.Mode |
getMode() |
boolean |
isRunning() |
static void |
main(String[] args)
To use from command line:
Supply configuration similar to src/test/resources/rss.conf
Launch using:
mvn exec:java -Dexec.mainClass=org.apache.streams.rss.provider.RssStreamProvider -Dexec.args="rss.conf articles.json"
|
protected void |
pauseForSpace()
Wait for the queue size to be below threshold before allowing execution to continue on this thread.
|
void |
prepare(Object configurationObject) |
StreamsResultSet |
readCurrent() |
StreamsResultSet |
readNew(BigInteger bigInteger) |
StreamsResultSet |
readRange(org.joda.time.DateTime dateTime,
org.joda.time.DateTime dateTime2) |
void |
setConfig(SysomosConfiguration config) |
void |
signalComplete(String heartbeatId)
signalComplete.
|
void |
startStream() |
public static final String STREAMS_ID
public static final String ENDING_TIME_KEY
public static final String STARTING_TIME_KEY
public static final String MODE_KEY
public static final String STARTING_DOCS_KEY
public static final int LATENCY
public static final long PROVIDER_BATCH_SIZE
public static final long API_BATCH_SIZE
protected volatile Queue<StreamsDatum> providerQueue
public SysomosProvider(SysomosConfiguration sysomosConfiguration)
sysomosConfiguration
- SysomosConfigurationpublic static void main(String[] args) throws Exception
args
- argsException
- Exceptionpublic SysomosConfiguration getConfig()
public void setConfig(SysomosConfiguration config)
public SysomosProvider.Mode getMode()
public long getMinLatency()
public long getMaxApiBatch()
public SysomosClient getClient()
public String getId()
getId
in interface StreamsOperation
public void startStream()
startStream
in interface StreamsProvider
public StreamsResultSet readCurrent()
readCurrent
in interface StreamsProvider
public StreamsResultSet readNew(BigInteger bigInteger)
readNew
in interface StreamsProvider
public StreamsResultSet readRange(org.joda.time.DateTime dateTime, org.joda.time.DateTime dateTime2)
readRange
in interface StreamsProvider
public boolean isRunning()
isRunning
in interface StreamsProvider
public void prepare(Object configurationObject)
prepare
in interface StreamsOperation
public void cleanUp()
cleanUp
in interface StreamsOperation
public void signalComplete(String heartbeatId)
heartbeatId
- heartbeatIdprotected void enqueueItem(StreamsDatum datum)
protected SysomosHeartbeatStream createStream(String heartbeatId)
protected void pauseForSpace()
protected void extractConfigFromMap(Map configMap)
public int getCount()
Copyright © 2018 The Apache Software Foundation. All rights reserved.