org.apache.streams.hdfs

Class WebHdfsPersistWriter

java.lang.Object

org.apache.streams.hdfs.WebHdfsPersistWriter

All Implemented Interfaces:

Closeable, Flushable, Serializable, AutoCloseable, DatumStatusCountable, StreamsOperation, StreamsPersistWriter

public class WebHdfsPersistWriter
extends Object
implements StreamsPersistWriter, Flushable, Closeable, DatumStatusCountable

WebHdfsPersistWriter writes to hdfs.

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field and Description
protected HdfsWriterConfiguration	hdfsConfiguration
protected Queue<StreamsDatum>	persistQueue
static String	STREAMS_ID
boolean	terminate

Constructor Summary

Constructors

Constructor and Description
WebHdfsPersistWriter()
WebHdfsPersistWriter(HdfsWriterConfiguration hdfsConfiguration)

Method Summary

Modifier and Type	Method and Description
void	cleanUp()
void	close()
void	flush()
DatumStatusCounter	getDatumStatusCounter()
org.apache.hadoop.fs.FileSystem	getFileSystem() getFileSystem.
String	getId()
URI	getURI() getURI from hdfsConfiguration.
boolean	isConnected() isConnected.
void	prepare(Object configurationObject)
void	write(StreamsDatum streamsDatum)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

STREAMS_ID

public static final String STREAMS_ID

See Also:

Constant Field Values

terminate

public boolean terminate

persistQueue

protected volatile Queue<StreamsDatum> persistQueue

hdfsConfiguration

protected HdfsWriterConfiguration hdfsConfiguration

Constructor Detail

WebHdfsPersistWriter

public WebHdfsPersistWriter()

WebHdfsPersistWriter

public WebHdfsPersistWriter(HdfsWriterConfiguration hdfsConfiguration)

Method Detail

getURI

public URI getURI()
           throws URISyntaxException

getURI from hdfsConfiguration.

Returns:

URI

Throws:

URISyntaxException - URISyntaxException

isConnected

public boolean isConnected()

isConnected.

Returns:

true if connected, false otherwise

getFileSystem

public final org.apache.hadoop.fs.FileSystem getFileSystem()

getFileSystem.

Returns:

FileSystem

getId

public String getId()

Specified by:

getId in interface StreamsOperation

write

public void write(StreamsDatum streamsDatum)

Specified by:

write in interface StreamsPersistWriter

flush

public void flush()
           throws IOException

Specified by:

flush in interface Flushable

Throws:

IOException

close

public void close()
           throws IOException

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable

Throws:

IOException

prepare

public void prepare(Object configurationObject)

Specified by:

prepare in interface StreamsOperation

cleanUp

public void cleanUp()

Specified by:

cleanUp in interface StreamsOperation

getDatumStatusCounter

public DatumStatusCounter getDatumStatusCounter()

Specified by:

getDatumStatusCounter in interface DatumStatusCountable