Package org.apache.ignite.internal.processors.cache.persistence.checkpoint

Class CheckpointManager

java.lang.Object

org.apache.ignite.internal.processors.cache.persistence.checkpoint.CheckpointManager

public class CheckpointManager
extends Object

Main class to abstract checkpoint-related processes and actions and hide them from higher-level components. Implements default checkpointing algorithm which is sharp checkpoint but can be replaced by other implementations if needed. Represents only an intermediate step in refactoring of checkpointing component and may change in the future. This checkpoint ensures that all pages marked as dirty under () will be consistently saved to disk. Configuration of this checkpoint allows the following:

Collecting all pages from configured dataRegions which was marked as dirty under ().

*

Marking the start of checkpoint in WAL and on disk.

Notifying the subscribers of different checkpoint states through CheckpointListener.

*

Synchronizing collected pages with disk using FilePageStoreManager.

Restoring memory in consistent state if the node failed in the middle of checkpoint.

Constructor Summary

Constructors

Constructor

Description

CheckpointManager(Function<Class<?>,IgniteLogger> logger, String igniteInstanceName, String checkpointThreadName, IgniteWriteAheadLogManager wal, WorkersRegistry workersRegistry, DataStorageConfiguration persistenceCfg, FilePageStoreManager pageStoreManager, IgniteThrowableBiPredicate<Long,Integer> checkpointInapplicableChecker, Supplier<Collection<DataRegion>> dataRegions, Supplier<Collection<CacheGroupContext>> cacheGroupContexts, IgniteThrowableFunction<Integer,PageMemoryEx> pageMemoryGroupResolver, PageMemoryImpl.ThrottlingPolicy throttlingPolicy, DataStorageMetricsImpl persStoreMetrics, LongJVMPauseDetector longJvmPauseDetector, FailureProcessor failureProcessor, GridCacheProcessor cacheProcessor, Supplier<Integer> cpFreqDeviation, Executor checkpointMapSnapshotExecutor, JdkMarshaller marsh)

Method Summary

Modifier and Type	Method	Description
void	addCheckpointListener(CheckpointListener lsnr, DataRegion dataRegion)
boolean	cancelOrWaitPartitionDestroy(int grpId, int partId)
File	checkpointDirectory()
CheckpointHistory	checkpointHistory()
CheckpointMarkersStorage	checkpointMarkerStorage()
CheckpointTimeoutLock	checkpointTimeoutLock()
void	cleanupCheckpointDirectory()	Clean checkpoint directory CheckpointMarkersStorage.cpDir.
void	cleanupTempCheckpointDirectory()	Cleanup checkpoint directory from all temporary files.
IgniteInternalFuture<Void>	enableCheckpoints(boolean enable)	For test use only.
void	finalizeCheckpointOnRecovery(long ts, UUID id, WALPointer ptr, StripedExecutor exec)
<R> CheckpointProgress	forceCheckpoint(String reason, IgniteInClosure<? super IgniteInternalFuture<R>> lsnr)	Start the new checkpoint immediately.
Checkpointer	getCheckpointer()	Current checkpointer implementation.
void	init()	Initialize the checkpoint and prepare it to work.
void	initializeStorage()	Initialize checkpoint storage.
void	memoryRecoveryRecordPtr(WALPointer memoryRecoveryRecordPtr)
CheckpointStatus	readCheckpointStatus()
void	removeCheckpointListener(CheckpointListener lsnr)
void	removeCheckpointsUntil(@Nullable WALPointer highBound)	Wal truncate callback.
void	schedulePartitionDestroy(@Nullable CacheGroupContext context, int groupId, int partId)
void	start()	Checkpoint starts to do their work after this method.
void	stop(boolean cancel)
void	threadBuf(ThreadLocal<ByteBuffer> threadBuf)	Replace thread local with buffers.
void	unblockCheckpointLock()	Checkpoint lock blocks when stop method is called.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

CheckpointManager

public CheckpointManager(Function<Class<?>,IgniteLogger> logger,
                         String igniteInstanceName,
                         String checkpointThreadName,
                         IgniteWriteAheadLogManager wal,
                         WorkersRegistry workersRegistry,
                         DataStorageConfiguration persistenceCfg,
                         FilePageStoreManager pageStoreManager,
                         IgniteThrowableBiPredicate<Long,Integer> checkpointInapplicableChecker,
                         Supplier<Collection<DataRegion>> dataRegions,
                         Supplier<Collection<CacheGroupContext>> cacheGroupContexts,
                         IgniteThrowableFunction<Integer,PageMemoryEx> pageMemoryGroupResolver,
                         PageMemoryImpl.ThrottlingPolicy throttlingPolicy,
                         DataStorageMetricsImpl persStoreMetrics,
                         LongJVMPauseDetector longJvmPauseDetector,
                         FailureProcessor failureProcessor,
                         GridCacheProcessor cacheProcessor,
                         Supplier<Integer> cpFreqDeviation,
                         Executor checkpointMapSnapshotExecutor,
                         JdkMarshaller marsh)
                  throws IgniteCheckedException

Parameters:

logger - Logger producer.

igniteInstanceName - Ignite instance name.

checkpointThreadName - Name of main checkpoint thread.

wal - Write ahead log manager.

workersRegistry - Workers registry.

persistenceCfg - Persistence configuration.

pageStoreManager - File page store manager.

checkpointInapplicableChecker - Checker of checkpoints.

dataRegions - Data regions.

cacheGroupContexts - Cache group contexts.

pageMemoryGroupResolver - Page memory resolver.

throttlingPolicy - Throttling policy.

persStoreMetrics - Persistence metrics.

longJvmPauseDetector - Long JVM pause detector.

failureProcessor - Failure processor.

cacheProcessor - Cache processor.

cpFreqDeviation - Distributed checkpoint frequency deviation.

checkpointMapSnapshotExecutor - Checkpoint map snapshot executor.

marsh - JDK marshaller.

Throws:

IgniteCheckedException - if fail.

Method Detail

checkpointTimeoutLock

public CheckpointTimeoutLock checkpointTimeoutLock()

Returns:

Checkpoint lock which can be used for protection of writing to memory.

threadBuf

public void threadBuf(ThreadLocal<ByteBuffer> threadBuf)

Replace thread local with buffers. Thread local should provide direct buffer with one page in length.

Parameters:

threadBuf - new thread-local with buffers for the checkpoint threads.

addCheckpointListener

public void addCheckpointListener(CheckpointListener lsnr,
                                  DataRegion dataRegion)

Parameters:

lsnr - Listener.

dataRegion - Data region for which listener is corresponded to.

removeCheckpointListener

public void removeCheckpointListener(CheckpointListener lsnr)

Parameters:

lsnr - Listener.

memoryRecoveryRecordPtr

public void memoryRecoveryRecordPtr(WALPointer memoryRecoveryRecordPtr)

Parameters:

memoryRecoveryRecordPtr - Memory recovery record pointer.

checkpointDirectory

public File checkpointDirectory()

Returns:

Checkpoint directory.

checkpointMarkerStorage

public CheckpointMarkersStorage checkpointMarkerStorage()

Returns:

Checkpoint storage.

readCheckpointStatus

public CheckpointStatus readCheckpointStatus()
                                      throws IgniteCheckedException

Returns:

Read checkpoint status.

Throws:

IgniteCheckedException - If failed to read checkpoint status page.

forceCheckpoint

public <R> CheckpointProgress forceCheckpoint(String reason,
                                              IgniteInClosure<? super IgniteInternalFuture<R>> lsnr)

Start the new checkpoint immediately.

Parameters:

reason - Reason.

lsnr - Listener which will be called on finish.

Returns:

Triggered checkpoint progress.

checkpointHistory

public CheckpointHistory checkpointHistory()

Returns:

Checkpoint history.

initializeStorage

public void initializeStorage()
                       throws IgniteCheckedException

Initialize checkpoint storage.

Throws:

IgniteCheckedException

removeCheckpointsUntil

public void removeCheckpointsUntil(@Nullable
                                   @Nullable WALPointer highBound)
                            throws IgniteCheckedException

Wal truncate callback.

Parameters:

highBound - Upper bound.

Throws:

IgniteCheckedException - If failed.

cleanupTempCheckpointDirectory

public void cleanupTempCheckpointDirectory()
                                    throws IgniteCheckedException

Cleanup checkpoint directory from all temporary files.

Throws:

IgniteCheckedException

cleanupCheckpointDirectory

public void cleanupCheckpointDirectory()
                                throws IgniteCheckedException

Clean checkpoint directory CheckpointMarkersStorage.cpDir. The operation is necessary when local node joined to baseline topology with different consistentId.

Throws:

IgniteCheckedException

getCheckpointer

public Checkpointer getCheckpointer()

Current checkpointer implementation.

schedulePartitionDestroy

public void schedulePartitionDestroy(@Nullable
                                     @Nullable CacheGroupContext context,
                                     int groupId,
                                     int partId)

Parameters:

context - Group context. Can be null in case of crash recovery.

groupId - Group ID.

partId - Partition ID.

enableCheckpoints

public IgniteInternalFuture<Void> enableCheckpoints(boolean enable)

For test use only.

finalizeCheckpointOnRecovery

public void finalizeCheckpointOnRecovery(long ts,
                                         UUID id,
                                         WALPointer ptr,
                                         StripedExecutor exec)
                                  throws IgniteCheckedException

Throws:

IgniteCheckedException - If failed.

cancelOrWaitPartitionDestroy

public boolean cancelOrWaitPartitionDestroy(int grpId,
                                            int partId)
                                     throws IgniteCheckedException

Parameters:

grpId - Group ID.

partId - Partition ID.

Returns:

True if the request to destroy the partition was canceled.

Throws:

IgniteCheckedException

stop

public void stop(boolean cancel)

Parameters:

cancel - Cancel flag.

init

public void init()

Initialize the checkpoint and prepare it to work. It should be called if the stop was called before.

start

public void start()

Checkpoint starts to do their work after this method.

unblockCheckpointLock

public void unblockCheckpointLock()

Checkpoint lock blocks when stop method is called. This method allows continuing the work with a checkpoint lock if needed.