org.apache.ignite

## Interface IgniteDataLoader<K,V>

All Superinterfaces:
  AutoCloseable

All Known Implementing Classes:
  IgniteDataLoaderImpl

    public interface IgniteDataLoader<K,V>
    extends AutoCloseable

Data loader is responsible for loading external data into cache. It achieves it by properly buffering updates and properly mapping keys to nodes responsible for the data to make sure that there is the least amount of data movement possible and optimal network and memory utilization.

Note that loader will load data concurrently by multiple internal threads, so the data may get to remote nodes in different order from which it was added to the loader.

Also note that `GridDataLoader` is not the only way to load data into cache. Alternatively you can use `IgniteCache.loadCache(IgniteBiPredicate, Object...)` method to load data from underlying data store. You can also use standard cache `put(...)` and `putAll(...)` operations as well, but they most likely will not perform as well as this class for loading data. And finally, data can be loaded from underlying data store on demand, whenever it is accessed - for this no explicit data loading step is needed.

`IgniteDataLoader` supports the following configuration properties:

- `perNodeBufferSize(int)` - when entries are added to data loader via `addData(Object, Object)` method, they are not sent to in-memory data grid right away and are buffered internally for better performance and network utilization. This setting controls the size of internal per-node buffer before buffered data is sent to remote node. Default is defined by `DFLT_PER_NODE_BUFFER_SIZE` value.
- `perNodeParallelLoadOperations(int)` - sometimes data may be added to the data loader via `addData(Object, Object)` method faster than it can be put in cache. In this case, new buffered load messages are sent to remote nodes before responses from previous ones are received. This could cause unlimited heap memory utilization growth on local and remote nodes. To control memory utilization, this setting limits maximum allowed number of parallel buffered load messages that are being processed on remote nodes. If this number is exceeded, then `addData(Object, Object)` method will block to control memory utilization. Default is defined by `DFLT_MAX_PARALLEL_OPS` value.
- `autoFlushFrequency(long)` - automatic flush frequency in milliseconds. Essentially, this is the time after which the loader will make an attempt to submit all data added so far to remote nodes. Note that there is no guarantee that data will be delivered after this concrete attempt (e.g., it can fail when topology is changing), but it won't be lost anyway. Disabled by default (default value is `0`).
- `allowOverwrite(boolean)` - defines if data loader will assume that there are no other concurrent updates and allow data loader choose most optimal concurrent implementation.
- `updater(IgniteDataLoader.Updater)` - defines how cache will be updated with loaded entries. It allows to provide user-defined custom logic to update the cache in the most effective and flexible way.
- `deployClass(Class)` - optional deploy class for peer deployment. All classes loaded by a data loader must be class-loadable from the same class-loader. Ignite will make the best effort to detect the most suitable class-loader for data loading. However, in complex cases, where compound or deeply nested class-loaders are used, it is best to specify a deploy class which can be any class loaded by the class-loader for given data.

### Nested Class Summary

Nested Classes
`static interface` `IgniteDataLoader.Updater<K,V>` Updates cache with batch of entries.

### Field Summary

Fields
`static int` `DFLT_MAX_PARALLEL_OPS` Default max concurrent put operations count.
`static int` `DFLT_PER_NODE_BUFFER_SIZE` Default per node buffer size.

### Method Summary

Methods
`IgniteFuture<?>` `addData(Collection<? extends Map.Entry<K,V>> entries)` Adds data for loading on remote node.
`IgniteFuture<?>` `addData(K key, V val)` Adds data for loading on remote node.
`IgniteFuture<?>` `addData(Map.Entry<K,V> entry)` Adds data for loading on remote node.
`IgniteFuture<?>` `addData(Map<K,V> entries)` Adds data for loading on remote node.
`boolean` `allowOverwrite()` Gets flag value indicating that this data loader assumes that there are no other concurrent updates to the cache.
`void` `allowOverwrite(boolean allowOverwrite)` Sets flag indicating that this data loader should assume that there are no other concurrent updates to the cache.
`long` `autoFlushFrequency()` Gets automatic flush frequency.
`void` `autoFlushFrequency(long autoFlushFreq)` Sets automatic flush frequency.
`String` `cacheName()` Name of cache to load data to.
`void` `close()` Closes data loader.
`void` `close(boolean cancel)` Loads any remaining data and closes this loader.
`void` `deployClass(Class<?> depCls)` Optional deploy class for peer deployment.
`void` `flush()` Loads any remaining data, but doesn't close the loader.
`IgniteFuture<?>` `future()` Gets future for this loading process.
`int` `perNodeBufferSize()` Gets size of per node key-value pairs buffer.
`void` `perNodeBufferSize(int bufSize)` Sets size of per node key-value pairs buffer.
`int` `perNodeParallelLoadOperations()` Gets maximum number of parallel load operations for a single node.
`void` `perNodeParallelLoadOperations(int parallelOps)` Sets maximum number of parallel load operations for a single node.
`IgniteFuture<?>` `removeData(K key)` Adds key for removal on remote node.
`boolean` `skipStore()` Gets flag indicating that write-through behavior should be disabled for data loading.
`void` `skipStore(boolean skipStore)` Sets flag indicating that write-through behavior should be disabled for data loading.
`void` `tryFlush()` Makes an attempt to load remaining data.
`void` `updater(IgniteDataLoader.Updater<K,V> updater)` Sets custom cache updater to this data loader.

### Field Detail

#### DFLT_MAX_PARALLEL_OPS

    static final int DFLT_MAX_PARALLEL_OPS

Default max concurrent put operations count.

See Also:
  Constant Field Values

#### DFLT_PER_NODE_BUFFER_SIZE

    static final int DFLT_PER_NODE_BUFFER_SIZE

Default per node buffer size.

See Also:
  Constant Field Values

### Method Detail

#### cacheName

    String cacheName()

Name of cache to load data to.

Returns:
  Cache name or `null` for default cache.

#### allowOverwrite

    boolean allowOverwrite()

Gets flag value indicating that this data loader assumes that there are no other concurrent updates to the cache. Default is `true`.

Returns:
  Flag value.

#### allowOverwrite

    void allowOverwrite(boolean allowOverwrite)
                        throws IgniteException

Sets flag indicating that this data loader should assume that there are no other concurrent updates to the cache. Should not be used when custom cache updater set using `updater(IgniteDataLoader.Updater)` method. Default is `true`. When this flag is set, updates will not be propagated to the cache store.

Parameters:
  `allowOverwrite` - Flag value.
Throws:
  `IgniteException` - If failed.

#### skipStore

    boolean skipStore()

Gets flag indicating that write-through behavior should be disabled for data loading. Default is `false`.

Returns:
  Skip store flag.

#### skipStore

    void skipStore(boolean skipStore)

Sets flag indicating that write-through behavior should be disabled for data loading. Default is `false`.

Parameters:
  `skipStore` - Skip store flag.

#### perNodeBufferSize

    int perNodeBufferSize()

Gets size of per node key-value pairs buffer.

Returns:
  Per node buffer size.

#### perNodeBufferSize

    void perNodeBufferSize(int bufSize)

Sets size of per node key-value pairs buffer.

This method should be called prior to `addData(Object, Object)` call.

If not provided, default value is `DFLT_PER_NODE_BUFFER_SIZE`.

Parameters:
  `bufSize` - Per node buffer size.

#### perNodeParallelLoadOperations

    int perNodeParallelLoadOperations()

Gets maximum number of parallel load operations for a single node.

Returns:
  Maximum number of parallel load operations for a single node.

#### perNodeParallelLoadOperations

    void perNodeParallelLoadOperations(int parallelOps)

Sets maximum number of parallel load operations for a single node.

This method should be called prior to `addData(Object, Object)` call.

If not provided, default value is `DFLT_MAX_PARALLEL_OPS`.

Parameters:
  `parallelOps` - Maximum number of parallel load operations for a single node.

#### autoFlushFrequency

    long autoFlushFrequency()

Gets automatic flush frequency. Essentially, this is the time after which the loader will make an attempt to submit all data added so far to remote nodes. Note that there is no guarantee that data will be delivered after this concrete attempt (e.g., it can fail when topology is changing), but it won't be lost anyway.

If set to `0`, automatic flush is disabled.

Automatic flush is disabled by default (default value is `0`).

Returns:
  Flush frequency or `0` if automatic flush is disabled.
See Also:
  `flush()`

#### autoFlushFrequency

    void autoFlushFrequency(long autoFlushFreq)

Sets automatic flush frequency. Essentially, this is the time after which the loader will make an attempt to submit all data added so far to remote nodes. Note that there is no guarantee that data will be delivered after this concrete attempt (e.g., it can fail when topology is changing), but it won't be lost anyway.

If set to `0`, automatic flush is disabled.

Automatic flush is disabled by default (default value is `0`).

Parameters:
  `autoFlushFreq` - Flush frequency or `0` to disable automatic flush.
See Also:
  `flush()`

#### future

    IgniteFuture<?> future()

Gets future for this loading process. This future completes whenever method `close(boolean)` completes. By attaching listeners to this future it is possible to get asynchronous notifications for completion of this loading process.

Returns:
  Future for this loading process.

#### deployClass

    void deployClass(Class<?> depCls)

Optional deploy class for peer deployment. All classes loaded by a data loader must be class-loadable from the same class-loader. Ignite will make the best effort to detect the most suitable class-loader for data loading. However, in complex cases, where compound or deeply nested class-loaders are used, it is best to specify a deploy class which can be any class loaded by the class-loader for given data.

Parameters:
  `depCls` - Any class loaded by the class-loader for given data.

#### updater

    void updater(IgniteDataLoader.Updater<K,V> updater)

Sets custom cache updater to this data loader.

Parameters:
  `updater` - Cache updater.

#### removeData

    IgniteFuture<?> removeData(K key)
                               throws IgniteException,
                                      IgniteInterruptedException,
                                      IllegalStateException

Adds key for removal on remote node. Equivalent to `addData(key, null)`.

Parameters:
  `key` - Key.
Returns:
  Future fo this operation.
Throws:
  `IgniteException` - If failed to map key to node.
  `IgniteInterruptedException` - If thread has been interrupted.
  `IllegalStateException` - If grid has been concurrently stopped or `close(boolean)` has already been called on loader.

#### addData

    IgniteFuture<?> addData(K key,
                          @Nullable
                          V val)
                            throws IgniteException,
                                   IgniteInterruptedException,
                                   IllegalStateException

Adds data for loading on remote node. This method can be called from multiple threads in parallel to speed up loading if needed.

Note that loader will load data concurrently by multiple internal threads, so the data may get to remote nodes in different order from which it was added to the loader.

Parameters:
  `key` - Key.
  `val` - Value or `null` if respective entry must be removed from cache.
Returns:
  Future fo this operation.
Throws:
  `IgniteException` - If failed to map key to node.
  `IgniteInterruptedException` - If thread has been interrupted.
  `IllegalStateException` - If grid has been concurrently stopped or `close(boolean)` has already been called on loader.

#### addData

    IgniteFuture<?> addData(Map.Entry<K,V> entry)
                            throws IgniteException,
                                   IgniteInterruptedException,
                                   IllegalStateException

Adds data for loading on remote node. This method can be called from multiple threads in parallel to speed up loading if needed.

Note that loader will load data concurrently by multiple internal threads, so the data may get to remote nodes in different order from which it was added to the loader.

Parameters:
  `entry` - Entry.
Returns:
  Future fo this operation.
Throws:
  `IgniteException` - If failed to map key to node.
  `IgniteInterruptedException` - If thread has been interrupted.
  `IllegalStateException` - If grid has been concurrently stopped or `close(boolean)` has already been called on loader.

#### addData

    IgniteFuture<?> addData(Collection<? extends Map.Entry<K,V>> entries)
                            throws IllegalStateException

Adds data for loading on remote node. This method can be called from multiple threads in parallel to speed up loading if needed.

Note that loader will load data concurrently by multiple internal threads, so the data may get to remote nodes in different order from which it was added to the loader.

Parameters:
  `entries` - Collection of entries to be loaded.
Returns:
  Future for this load operation.
Throws:
  `IllegalStateException` - If grid has been concurrently stopped or `close(boolean)` has already been called on loader.

#### addData

    IgniteFuture<?> addData(Map<K,V> entries)
                            throws IllegalStateException

Adds data for loading on remote node. This method can be called from multiple threads in parallel to speed up loading if needed.

Note that loader will load data concurrently by multiple internal threads, so the data may get to remote nodes in different order from which it was added to the loader.

Parameters:
  `entries` - Map to be loaded.
Returns:
  Future for this load operation.
Throws:
  `IllegalStateException` - If grid has been concurrently stopped or `close(boolean)` has already been called on loader.

#### flush

    void flush()
               throws IgniteException,
                      IgniteInterruptedException,
                      IllegalStateException

Loads any remaining data, but doesn't close the loader. Data can be still added after flush is finished. This method blocks and doesn't allow to add any data until all data is loaded.

If another thread is already performing flush, this method will block, wait for another thread to complete flush and exit. If you don't want to wait in this case, use `tryFlush()` method.

Throws:
  `IgniteException` - If failed to map key to node.
  `IgniteInterruptedException` - If thread has been interrupted.
  `IllegalStateException` - If grid has been concurrently stopped or `close(boolean)` has already been called on loader.
See Also:
  `tryFlush()`

#### tryFlush

    void tryFlush()
                  throws IgniteException,
                         IgniteInterruptedException,
                         IllegalStateException

Makes an attempt to load remaining data. This method is mostly similar to `flush()`, with the difference that it won't wait and will exit immediately.

Throws:
  `IgniteException` - If failed to map key to node.
  `IgniteInterruptedException` - If thread has been interrupted.
  `IllegalStateException` - If grid has been concurrently stopped or `close(boolean)` has already been called on loader.
See Also:
  `flush()`

#### close

    void close(boolean cancel)
               throws IgniteException,
                      IgniteInterruptedException

Loads any remaining data and closes this loader.

Parameters:
  `cancel` - `True` to cancel ongoing loading operations.
Throws:
  `IgniteException` - If failed to map key to node.
  `IgniteInterruptedException` - If thread has been interrupted.

#### close

    void close()
               throws IgniteException,
                      IgniteInterruptedException

Closes data loader. This method is identical to calling `close(false)` method.

The method is invoked automatically on objects managed by the `try-with-resources` statement.

Specified by:
  `close` in interface `AutoCloseable`
Throws:
  `IgniteException` - If failed to close data loader.
  `IgniteInterruptedException` - If thread has been interrupted.

2015 Copyright © Apache Software Foundation

Follow @ApacheIgnite

Ignite Fabric | : ver. 1.0.0-RC1
Release Date | : February 17 2015