org.apache.ignite.internal.processors.dataload

Class IgniteDataLoaderImpl<K,V>

java.lang.Object

org.apache.ignite.internal.processors.dataload.IgniteDataLoaderImpl<K,V>

All Implemented Interfaces:

AutoCloseable, Comparable<Delayed>, Delayed, IgniteDataLoader<K,V>

public class IgniteDataLoaderImpl<K,V>
extends Object
implements IgniteDataLoader<K,V>, Delayed

Data loader implementation.

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.ignite.IgniteDataLoader

IgniteDataLoader.Updater<K,V>

Field Summary

Fields inherited from interface org.apache.ignite.IgniteDataLoader

DFLT_MAX_PARALLEL_OPS, DFLT_PER_NODE_BUFFER_SIZE

Constructor Summary

Constructors

Constructor and Description
IgniteDataLoaderImpl(GridKernalContext ctx, String cacheName, DelayQueue<IgniteDataLoaderImpl<K,V>> flushQ, boolean compact)

Method Summary

Methods

Modifier and Type	Method and Description
IgniteFuture<?>	addData(Collection<? extends Map.Entry<K,V>> entries) Adds data for loading on remote node.
IgniteFuture<?>	addData(K key, V val) Adds data for loading on remote node.
IgniteFuture<?>	addData(Map.Entry<K,V> entry) Adds data for loading on remote node.
IgniteFuture<?>	addData(Map<K,V> entries) Adds data for loading on remote node.
boolean	allowOverwrite() Gets flag value indicating that this data loader assumes that there are no other concurrent updates to the cache.
void	allowOverwrite(boolean allow) Sets flag indicating that this data loader should assume that there are no other concurrent updates to the cache.
long	autoFlushFrequency() Gets automatic flush frequency.
void	autoFlushFrequency(long autoFlushFreq) Sets automatic flush frequency.
String	cacheName() Name of cache to load data to.
void	close() Closes data loader.
void	close(boolean cancel) Loads any remaining data and closes this loader.
void	closeEx(boolean cancel)
int	compareTo(Delayed o)
void	deployClass(Class<?> depCls) Optional deploy class for peer deployment.
void	flush() Loads any remaining data, but doesn't close the loader.
IgniteFuture<?>	future() Gets future for this loading process.
long	getDelay(TimeUnit unit)
IgniteInternalFuture<?>	internalFuture()
int	perNodeBufferSize() Gets size of per node key-value pairs buffer.
void	perNodeBufferSize(int bufSize) Sets size of per node key-value pairs buffer.
int	perNodeParallelLoadOperations() Gets maximum number of parallel load operations for a single node.
void	perNodeParallelLoadOperations(int parallelOps) Sets maximum number of parallel load operations for a single node.
IgniteFuture<?>	removeData(K key) Adds key for removal on remote node.
boolean	skipStore() Gets flag indicating that write-through behavior should be disabled for data loading.
void	skipStore(boolean skipStore) Sets flag indicating that write-through behavior should be disabled for data loading.
String	toString()
void	tryFlush() Flushes every internal buffer if buffer was flushed before passed in threshold.
void	updater(IgniteDataLoader.Updater<K,V> updater) Sets custom cache updater to this data loader.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

IgniteDataLoaderImpl

public IgniteDataLoaderImpl(GridKernalContext ctx,
                    @Nullable
                    String cacheName,
                    DelayQueue<IgniteDataLoaderImpl<K,V>> flushQ,
                    boolean compact)

Parameters:

ctx - Grid kernal context.

cacheName - Cache name.

flushQ - Flush queue.

compact - If true data is transferred in compact mode (only keys and values). Otherwise full map entry will be transferred (this is required by DR internal logic).

Method Detail

future

public IgniteFuture<?> future()

Gets future for this loading process. This future completes whenever method IgniteDataLoader.close(boolean) completes. By attaching listeners to this future it is possible to get asynchronous notifications for completion of this loading process.

Specified by:

future in interface IgniteDataLoader<K,V>

Returns:

Future for this loading process.

internalFuture

public IgniteInternalFuture<?> internalFuture()

Returns:

Internal future.

deployClass

public void deployClass(Class<?> depCls)

Optional deploy class for peer deployment. All classes loaded by a data loader must be class-loadable from the same class-loader. Ignite will make the best effort to detect the most suitable class-loader for data loading. However, in complex cases, where compound or deeply nested class-loaders are used, it is best to specify a deploy class which can be any class loaded by the class-loader for given data.

Specified by:

deployClass in interface IgniteDataLoader<K,V>

Parameters:

depCls - Any class loaded by the class-loader for given data.

updater

public void updater(IgniteDataLoader.Updater<K,V> updater)

Sets custom cache updater to this data loader.

Specified by:

updater in interface IgniteDataLoader<K,V>

Parameters:

updater - Cache updater.

allowOverwrite

public boolean allowOverwrite()

Gets flag value indicating that this data loader assumes that there are no other concurrent updates to the cache. Default is true.

Specified by:

allowOverwrite in interface IgniteDataLoader<K,V>

Returns:

Flag value.

allowOverwrite

public void allowOverwrite(boolean allow)

Sets flag indicating that this data loader should assume that there are no other concurrent updates to the cache. Should not be used when custom cache updater set using IgniteDataLoader.updater(IgniteDataLoader.Updater) method. Default is true. When this flag is set, updates will not be propagated to the cache store.

Specified by:

allowOverwrite in interface IgniteDataLoader<K,V>

Parameters:

allow - Flag value.

skipStore

public boolean skipStore()

Gets flag indicating that write-through behavior should be disabled for data loading. Default is false.

Specified by:

skipStore in interface IgniteDataLoader<K,V>

Returns:

Skip store flag.

skipStore

public void skipStore(boolean skipStore)

Sets flag indicating that write-through behavior should be disabled for data loading. Default is false.

Specified by:

skipStore in interface IgniteDataLoader<K,V>

Parameters:

skipStore - Skip store flag.

cacheName

@Nullable
public String cacheName()

Name of cache to load data to.

Specified by:

cacheName in interface IgniteDataLoader<K,V>

Returns:

Cache name or null for default cache.

perNodeBufferSize

public int perNodeBufferSize()

Gets size of per node key-value pairs buffer.

Specified by:

perNodeBufferSize in interface IgniteDataLoader<K,V>

Returns:

Per node buffer size.

perNodeBufferSize

public void perNodeBufferSize(int bufSize)

Sets size of per node key-value pairs buffer.

This method should be called prior to IgniteDataLoader.addData(Object, Object) call.

If not provided, default value is IgniteDataLoader.DFLT_PER_NODE_BUFFER_SIZE.

Specified by:

perNodeBufferSize in interface IgniteDataLoader<K,V>

Parameters:

bufSize - Per node buffer size.

perNodeParallelLoadOperations

public int perNodeParallelLoadOperations()

Gets maximum number of parallel load operations for a single node.

Specified by:

perNodeParallelLoadOperations in interface IgniteDataLoader<K,V>

Returns:

Maximum number of parallel load operations for a single node.

perNodeParallelLoadOperations

public void perNodeParallelLoadOperations(int parallelOps)

Sets maximum number of parallel load operations for a single node.

This method should be called prior to IgniteDataLoader.addData(Object, Object) call.

If not provided, default value is IgniteDataLoader.DFLT_MAX_PARALLEL_OPS.

Specified by:

perNodeParallelLoadOperations in interface IgniteDataLoader<K,V>

Parameters:

parallelOps - Maximum number of parallel load operations for a single node.

autoFlushFrequency

public long autoFlushFrequency()

Gets automatic flush frequency. Essentially, this is the time after which the loader will make an attempt to submit all data added so far to remote nodes. Note that there is no guarantee that data will be delivered after this concrete attempt (e.g., it can fail when topology is changing), but it won't be lost anyway.

If set to 0, automatic flush is disabled.

Automatic flush is disabled by default (default value is 0).

Specified by:

autoFlushFrequency in interface IgniteDataLoader<K,V>

Returns:

Flush frequency or 0 if automatic flush is disabled.

See Also:

IgniteDataLoader.flush()

autoFlushFrequency

public void autoFlushFrequency(long autoFlushFreq)

Sets automatic flush frequency. Essentially, this is the time after which the loader will make an attempt to submit all data added so far to remote nodes. Note that there is no guarantee that data will be delivered after this concrete attempt (e.g., it can fail when topology is changing), but it won't be lost anyway.

If set to 0, automatic flush is disabled.

Automatic flush is disabled by default (default value is 0).

Specified by:

autoFlushFrequency in interface IgniteDataLoader<K,V>

Parameters:

autoFlushFreq - Flush frequency or 0 to disable automatic flush.

See Also:

IgniteDataLoader.flush()

addData

public IgniteFuture<?> addData(Map<K,V> entries)
                        throws IllegalStateException

Adds data for loading on remote node. This method can be called from multiple threads in parallel to speed up loading if needed.

Note that loader will load data concurrently by multiple internal threads, so the data may get to remote nodes in different order from which it was added to the loader.

Specified by:

addData in interface IgniteDataLoader<K,V>

Parameters:

entries - Map to be loaded.

Returns:

Future for this load operation.

Throws:

IllegalStateException - If grid has been concurrently stopped or IgniteDataLoader.close(boolean) has already been called on loader.

addData

public IgniteFuture<?> addData(Collection<? extends Map.Entry<K,V>> entries)

Adds data for loading on remote node. This method can be called from multiple threads in parallel to speed up loading if needed.

Note that loader will load data concurrently by multiple internal threads, so the data may get to remote nodes in different order from which it was added to the loader.

Specified by:

addData in interface IgniteDataLoader<K,V>

Parameters:

entries - Collection of entries to be loaded.

Returns:

Future for this load operation.

addData

public IgniteFuture<?> addData(Map.Entry<K,V> entry)

Adds data for loading on remote node. This method can be called from multiple threads in parallel to speed up loading if needed.

Note that loader will load data concurrently by multiple internal threads, so the data may get to remote nodes in different order from which it was added to the loader.

Specified by:

addData in interface IgniteDataLoader<K,V>

Parameters:

entry - Entry.

Returns:

Future fo this operation.

addData

public IgniteFuture<?> addData(K key,
                      V val)

Adds data for loading on remote node. This method can be called from multiple threads in parallel to speed up loading if needed.

Note that loader will load data concurrently by multiple internal threads, so the data may get to remote nodes in different order from which it was added to the loader.

Specified by:

addData in interface IgniteDataLoader<K,V>

Parameters:

key - Key.

val - Value or null if respective entry must be removed from cache.

Returns:

Future fo this operation.

removeData

public IgniteFuture<?> removeData(K key)

Adds key for removal on remote node. Equivalent to addData(key, null).

Specified by:

removeData in interface IgniteDataLoader<K,V>

Parameters:

key - Key.

Returns:

Future fo this operation.

flush

public void flush()
           throws IgniteException

Loads any remaining data, but doesn't close the loader. Data can be still added after flush is finished. This method blocks and doesn't allow to add any data until all data is loaded.

If another thread is already performing flush, this method will block, wait for another thread to complete flush and exit. If you don't want to wait in this case, use IgniteDataLoader.tryFlush() method.

Specified by:

flush in interface IgniteDataLoader<K,V>

Throws:

IgniteException - If failed to map key to node.

IgniteInterruptedException - If thread has been interrupted.

See Also:

IgniteDataLoader.tryFlush()

tryFlush

public void tryFlush()
              throws IgniteInterruptedException

Flushes every internal buffer if buffer was flushed before passed in threshold.

Does not wait for result and does not fail on errors assuming that this method should be called periodically.

Specified by:

tryFlush in interface IgniteDataLoader<K,V>

Throws:

IgniteInterruptedException - If thread has been interrupted.

See Also:

IgniteDataLoader.flush()

close

public void close(boolean cancel)
           throws IgniteException

Description copied from interface: IgniteDataLoader

Loads any remaining data and closes this loader.

Specified by:

close in interface IgniteDataLoader<K,V>

Parameters:

cancel - True to close with cancellation.

Throws:

IgniteException - If failed.

closeEx

public void closeEx(boolean cancel)
             throws IgniteCheckedException

Parameters:

cancel - True to close with cancellation.

Throws:

IgniteCheckedException - If failed.

close

public void close()
           throws IgniteException

Closes data loader. This method is identical to calling close(false) method.

The method is invoked automatically on objects managed by the try-with-resources statement.

Specified by:

close in interface AutoCloseable

Specified by:

close in interface IgniteDataLoader<K,V>

Throws:

IgniteException - If failed to close data loader.

IgniteInterruptedException - If thread has been interrupted.

toString

public String toString()

Overrides:

toString in class Object

getDelay

public long getDelay(TimeUnit unit)

Specified by:

getDelay in interface Delayed

compareTo

public int compareTo(Delayed o)

Specified by:

compareTo in interface Comparable<Delayed>