com.datastax.oss.dsbulk.workflow.commons.log

Class LogManager

java.lang.Object

com.datastax.oss.dsbulk.workflow.commons.log.LogManager

All Implemented Interfaces:

AutoCloseable

public class LogManager
extends Object
implements AutoCloseable

Constructor Summary

Constructors

Constructor and Description
LogManager(CqlSession session, Path operationDirectory, ErrorThreshold errorThreshold, ErrorThreshold queryWarningsThreshold, boolean trackPositions, StatementFormatter statementFormatter, StatementFormatVerbosity statementFormatVerbosity, RowFormatter rowFormatter)

Method Summary

Modifier and Type	Method and Description
void	close()
Path	getOperationDirectory()
int	getTotalErrors()
void	init()
Function<Flux<ReadResult>,Flux<ReadResult>>	newFailedReadsHandler() Handler for unsuccessful ReadResults.
Function<Flux<Record>,Flux<Record>>	newFailedRecordsHandler() Handler for failed records.
Function<Flux<WriteResult>,Flux<WriteResult>>	newFailedWritesHandler() Handler for unsuccessful WriteResults.
<T extends Result> Function<Flux<T>,Flux<T>>	newQueryWarningsHandler() Handler for query warnings.
Function<Flux<WriteResult>,Flux<Void>>	newResultPositionsHandler() Handler for result positions.
Function<Flux<Void>,Flux<Void>>	newTerminationHandler() Handler that is meant to be executed at the very end of the main workflow.
<T> Function<Flux<T>,Flux<T>>	newTotalItemsCounter()
Function<Flux<Record>,Flux<Record>>	newUnmappableRecordsHandler() Handler for unmappable records produced by the result mapper.
Function<Flux<BatchableStatement<?>>,Flux<BatchableStatement<?>>>	newUnmappableStatementsHandler() Handler for unmappable statements produced by the record mapper.
void	reportLastLocations()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

LogManager

public LogManager(CqlSession session,
                  Path operationDirectory,
                  ErrorThreshold errorThreshold,
                  ErrorThreshold queryWarningsThreshold,
                  boolean trackPositions,
                  StatementFormatter statementFormatter,
                  StatementFormatVerbosity statementFormatVerbosity,
                  RowFormatter rowFormatter)

Method Detail

init

public void init()

getOperationDirectory

public Path getOperationDirectory()

getTotalErrors

public int getTotalErrors()

close

public void close()
           throws IOException

Specified by:

close in interface AutoCloseable

Throws:

IOException

reportLastLocations

public void reportLastLocations()

newTerminationHandler

@NonNull
public Function<Flux<Void>,Flux<Void>> newTerminationHandler()

Handler that is meant to be executed at the very end of the main workflow.

It detects when the main workflow emits a terminal signal and reacts by triggering the completion of all processors managed by this component.

Returns:

a handler that is meant to be executed at the very end of the main workflow.

newUnmappableStatementsHandler

@NonNull
public Function<Flux<BatchableStatement<?>>,Flux<BatchableStatement<?>>> newUnmappableStatementsHandler()

Handler for unmappable statements produced by the record mapper.

Used only in load workflows.

Increments the number of errors and forwards unmappable statements to the unmappable statement processor for further processing.

Returns:

a handler for unmappable statements.

newFailedRecordsHandler

@NonNull
public Function<Flux<Record>,Flux<Record>> newFailedRecordsHandler()

Handler for failed records. A failed record is a record that the connector could not read or write.

Used by both load and unload workflows.

Increments the number of errors and forwards failed records to the failed record processor for further processing.

Returns:

a handler for failed records.

newUnmappableRecordsHandler

@NonNull
public Function<Flux<Record>,Flux<Record>> newUnmappableRecordsHandler()

Handler for unmappable records produced by the result mapper.

Used only in unload workflows.

Increments the number of errors and forwards unmappable records to the unmappable record processor for further processing.

Returns:

a handler for unmappable records.

newFailedWritesHandler

@NonNull
public Function<Flux<WriteResult>,Flux<WriteResult>> newFailedWritesHandler()

Handler for unsuccessful WriteResults.

Used only by the load workflow.

Increments the number of errors and forwards unsuccessful write results to the write result processor for further processing.

Returns:

a handler for unsuccessful write results.

newFailedReadsHandler

@NonNull
public Function<Flux<ReadResult>,Flux<ReadResult>> newFailedReadsHandler()

Handler for unsuccessful ReadResults.

Used only by the unload workflow.

Increments the number of errors and forwards unsuccessful read results to the read result processor for further processing.

Returns:

a handler for unsuccessful read results.

newQueryWarningsHandler

public <T extends Result> Function<Flux<T>,Flux<T>> newQueryWarningsHandler()

Handler for query warnings.

Used by all workflows.

Increments the number of query warnings; if the threshold is exceeded, logs one last warning than mutes subsequent query warnings.

Returns:

a handler for for query warnings.

newResultPositionsHandler

public Function<Flux<WriteResult>,Flux<Void>> newResultPositionsHandler()

Handler for result positions.

Used only by the load workflow.

Extracts the result's Record and updates the positions.

Returns:

A handler for result positions.

newTotalItemsCounter

public <T> Function<Flux<T>,Flux<T>> newTotalItemsCounter()