com.datastax.oss.dsbulk.connectors.json

Class JsonConnector

java.lang.Object

com.datastax.oss.dsbulk.connectors.commons.AbstractFileBasedConnector

com.datastax.oss.dsbulk.connectors.json.JsonConnector

All Implemented Interfaces:

Connector, AutoCloseable

public class JsonConnector
extends AbstractFileBasedConnector

A connector for Json files.

It is capable of reading from any URL, provided that there is a handler installed for it. For file URLs, it is also capable of reading several files at once from a given root directory.

This connector is highly configurable; see its dsbulk-reference.conf file, bundled within its jar archive, for detailed information.

Nested Class Summary

Nested classes/interfaces inherited from class com.datastax.oss.dsbulk.connectors.commons.AbstractFileBasedConnector

AbstractFileBasedConnector.RecordReader, AbstractFileBasedConnector.RecordWriter

Field Summary

Fields inherited from class com.datastax.oss.dsbulk.connectors.commons.AbstractFileBasedConnector

compression, COMPRESSION, encoding, ENCODING, FILE_NAME_FORMAT, FILE_NAME_PATTERN, fileCounter, fileNameFormat, files, MAX_CONCURRENT_FILES, MAX_RECORDS, maxConcurrentFiles, maxRecords, nextWriterIndex, pattern, read, recursive, RECURSIVE, resourceCount, retainRecordSources, roots, singleWriter, SKIP_RECORDS, skipRecords, URL, URLFILE, urls, writers

Constructor Summary

Constructors

Constructor and Description
JsonConnector()

Method Summary

Modifier and Type	Method and Description
void	configure(Config settings, boolean read, boolean retainRecordSources) Configures the connector.
String	getConnectorName() Returns the connector name, e.g.
RecordMetadata	getRecordMetadata() Returns metadata about the records that this connector can read or write.
void	init() Initializes the connector.
protected AbstractFileBasedConnector.RecordReader	newSingleFileReader(URL url) Returns a new AbstractFileBasedConnector.RecordReader instance; cannot be null.
protected AbstractFileBasedConnector.RecordWriter	newSingleFileWriter() Returns a new AbstractFileBasedConnector.RecordWriter instance; cannot be null.
boolean	supports(ConnectorFeature feature) Whether or not the connector supports the given feature.

Methods inherited from class com.datastax.oss.dsbulk.connectors.commons.AbstractFileBasedConnector

applyPerFileLimits, close, getOrCreateDestinationURL, isDataSizeSamplingAvailable, loadURLs, processURLsForRead, processURLsForWrite, read, readConcurrency, readSingleFile, scanRootDirectory, write, writeConcurrency

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

JsonConnector

public JsonConnector()

Method Detail

getConnectorName

@NonNull
public String getConnectorName()

Description copied from class: AbstractFileBasedConnector

Returns the connector name, e.g. "csv" for the CSV connector. This should match the programmatic connector name and also its configuration namespace; that is, if this method returns "foo", then the connector configuration namespace is expected to be connector.foo.

Specified by:

getConnectorName in class AbstractFileBasedConnector

configure

public void configure(@NonNull
                      Config settings,
                      boolean read,
                      boolean retainRecordSources)

Description copied from interface: Connector

Configures the connector.

Specified by:

configure in interface Connector

Overrides:

configure in class AbstractFileBasedConnector

Parameters:

settings - the connector settings.

read - whether the connector should be configured for reading or writing.

retainRecordSources - whether the connector should retain sources when emitting records; only applicable when the connector is being configured for reads.

init

public void init()
          throws URISyntaxException,
                 IOException

Description copied from interface: Connector

Initializes the connector.

This method should only be called after the connector is properly configured.

Specified by:

init in interface Connector

Overrides:

init in class AbstractFileBasedConnector

Throws:

URISyntaxException

IOException

getRecordMetadata

@NonNull
public RecordMetadata getRecordMetadata()

Description copied from interface: Connector

Returns metadata about the records that this connector can read or write.

This method should only be called after the connector is properly configured and initialized.

Returns:

the metadata about the records that this connector can read or write.

supports

public boolean supports(@NonNull
                        ConnectorFeature feature)

Description copied from interface: Connector

Whether or not the connector supports the given feature.

Parameters:

feature - the feature to check.

Returns:

true if this connector supports the feature, false otherwise.

newSingleFileReader

@NonNull
protected AbstractFileBasedConnector.RecordReader newSingleFileReader(@NonNull
                                                                               URL url)
                                                                        throws IOException

Description copied from class: AbstractFileBasedConnector

Returns a new AbstractFileBasedConnector.RecordReader instance; cannot be null. Only used when reading. Each invocation of this method is expected to return a newly-allocated instance. The reader is expected to be initialized already, and ready to emit its first record. It is possible to throw IOException if the reader cannot be initialized.

Specified by:

newSingleFileReader in class AbstractFileBasedConnector

Throws:

IOException

newSingleFileWriter

@NonNull
protected AbstractFileBasedConnector.RecordWriter newSingleFileWriter()

Description copied from class: AbstractFileBasedConnector

Returns a new AbstractFileBasedConnector.RecordWriter instance; cannot be null. Only used when writing. Each invocation of this method is expected to return a newly-allocated instance.

Specified by:

newSingleFileWriter in class AbstractFileBasedConnector