org.apache.ignite.internal.processors.cache.persistence.wal

Class FileWriteAheadLogManager

java.lang.Object

org.apache.ignite.internal.processors.cache.GridCacheSharedManagerAdapter

org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager

All Implemented Interfaces:

IgniteWriteAheadLogManager, GridCacheSharedManager, IgniteChangeGlobalStateSupport

public class FileWriteAheadLogManager
extends GridCacheSharedManagerAdapter
implements IgniteWriteAheadLogManager

File WAL manager.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	FileWriteAheadLogManager.ReadFileHandle

Field Summary

Fields

Modifier and Type	Field and Description
static long	DFLT_WAL_SEGMENT_SYNC_TIMEOUT Dfault wal segment sync timeout.
static Pattern	WAL_NAME_PATTERN Pattern for segment file names
static FileFilter	WAL_SEGMENT_COMPACTED_OR_RAW_FILE_FILTER WAL segment file filter, see WAL_NAME_PATTERN
static Pattern	WAL_SEGMENT_FILE_COMPACTED_PATTERN
static FileFilter	WAL_SEGMENT_FILE_FILTER WAL segment file filter, see WAL_NAME_PATTERN
static Pattern	WAL_TEMP_NAME_PATTERN

Fields inherited from class org.apache.ignite.internal.processors.cache.GridCacheSharedManagerAdapter

cctx, diagnosticLog, log

Constructor Summary

Constructors

Constructor and Description
FileWriteAheadLogManager(GridKernalContext ctx)

Method Summary

Modifier and Type	Method and Description
void	cleanupWalDirectories() Cleanup all directories relating to WAL (e.g. work WAL dir, archive WAL dir).
boolean	disabled(int grpId) Checks WAL disabled for cache group.
void	flush(WALPointer ptr, boolean explicitFsync) Makes sure that all log entries written to the log up until the specified pointer are actually written to the underlying storage.
Collection<File>	getAndReserveWalFiles(FileWALPointer low, FileWALPointer high) Collect wal segment files from low pointer (include) to high pointer (not include) and reserve low pointer.
boolean	isAlwaysWriteFullPages()
boolean	isFullSync()
long	lastArchivedSegment()
long	lastCompactedSegment()
static FileDescriptor[]	loadFileDescriptors(File walFilesDir) Scans provided folder for a WAL segment files
WALPointer	log(WALRecord rec) Appends the given log entry to the write-ahead log.
long	maxArchivedSegmentToDelete()
long	maxWalSegmentSize()
void	notchLastCheckpointPtr(WALPointer ptr) Notifies this about latest checkpoint pointer.
void	onActivate(GridKernalContext kctx) Called when cluster performing activation.
void	onDeActivate(GridKernalContext kctx) Called when cluster performing deactivation.
void	release(WALPointer start) Invoke this method to release WAL history since provided pointer that was previously reserved.
WALIterator	replay(WALPointer start) Invoke this method to iterate over the written log entries.
boolean	reserve(WALPointer start) Invoke this method to reserve WAL history since provided pointer and prevent it's deletion.
boolean	reserved(WALPointer ptr) Checks if WAL segment is under lock or reserved
int	reserved(WALPointer low, WALPointer high) Checks if WAL segments is under lock or reserved.
void	resumeLogging(WALPointer lastPtr) Resumes logging after start.
static FileDescriptor[]	scan(File[] allFiles)
int	serializerVersion()
void	setCreateWalFileListener(IgniteInClosure<FileIO> createWalFileListener) Setup listener for WAL segment write File IO creation.
void	setFileIOFactory(FileIOFactory ioFactory) For test purposes only.
void	start0()
protected void	stop0(boolean cancel)
int	truncate(WALPointer low, WALPointer high) Gives a hint to WAL manager to clear entries logged before the given pointer.
int	walArchiveSegments()

Methods inherited from class org.apache.ignite.internal.processors.cache.GridCacheSharedManagerAdapter

context, isStopping, kernalStartInfo, kernalStopInfo, log, onDisconnected, onKernalStop, onKernalStop0, onReconnected, printMemoryStats, start, startInfo, stop, stopInfo, toString

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.ignite.internal.processors.cache.GridCacheSharedManager

onDisconnected, onKernalStop, onReconnected, printMemoryStats, start, stop

Field Detail

DFLT_WAL_SEGMENT_SYNC_TIMEOUT

public static final long DFLT_WAL_SEGMENT_SYNC_TIMEOUT

Dfault wal segment sync timeout.

See Also:

Constant Field Values

WAL_NAME_PATTERN

public static final Pattern WAL_NAME_PATTERN

Pattern for segment file names

WAL_TEMP_NAME_PATTERN

public static final Pattern WAL_TEMP_NAME_PATTERN

WAL_SEGMENT_FILE_FILTER

public static final FileFilter WAL_SEGMENT_FILE_FILTER

WAL segment file filter, see WAL_NAME_PATTERN

WAL_SEGMENT_FILE_COMPACTED_PATTERN

public static final Pattern WAL_SEGMENT_FILE_COMPACTED_PATTERN

WAL_SEGMENT_COMPACTED_OR_RAW_FILE_FILTER

public static final FileFilter WAL_SEGMENT_COMPACTED_OR_RAW_FILE_FILTER

WAL segment file filter, see WAL_NAME_PATTERN

Constructor Detail

FileWriteAheadLogManager

public FileWriteAheadLogManager(@NotNull
                                GridKernalContext ctx)

Parameters:

ctx - Kernal context.

Method Detail

setFileIOFactory

public void setFileIOFactory(FileIOFactory ioFactory)

For test purposes only.

Parameters:

ioFactory - IO factory.

start0

public void start0()
            throws IgniteCheckedException

Overrides:

start0 in class GridCacheSharedManagerAdapter

Throws:

IgniteCheckedException - If failed.

getAndReserveWalFiles

public Collection<File> getAndReserveWalFiles(FileWALPointer low,
                                              FileWALPointer high)
                                       throws IgniteCheckedException

Collect wal segment files from low pointer (include) to high pointer (not include) and reserve low pointer.

Parameters:

low - Low bound.

high - High bound.

Throws:

IgniteCheckedException

stop0

protected void stop0(boolean cancel)

Overrides:

stop0 in class GridCacheSharedManagerAdapter

Parameters:

cancel - Cancel flag.

onActivate

public void onActivate(GridKernalContext kctx)
                throws IgniteCheckedException

Called when cluster performing activation.

Specified by:

onActivate in interface IgniteChangeGlobalStateSupport

Parameters:

kctx - Kernal context.

Throws:

IgniteCheckedException - If failed.

onDeActivate

public void onDeActivate(GridKernalContext kctx)

Called when cluster performing deactivation.

Specified by:

onDeActivate in interface IgniteChangeGlobalStateSupport

Parameters:

kctx - Kernal context.

isAlwaysWriteFullPages

public boolean isAlwaysWriteFullPages()

Specified by:

isAlwaysWriteFullPages in interface IgniteWriteAheadLogManager

Returns:

true If we have to always write full pages.

isFullSync

public boolean isFullSync()

Specified by:

isFullSync in interface IgniteWriteAheadLogManager

Returns:

true if WAL will perform fair syncs on fsync call.

resumeLogging

public void resumeLogging(WALPointer lastPtr)
                   throws IgniteCheckedException

Resumes logging after start. When WAL manager is started, it will skip logging any updates until this method is called to avoid logging changes induced by the state restore procedure.

Specified by:

resumeLogging in interface IgniteWriteAheadLogManager

Throws:

IgniteCheckedException

serializerVersion

public int serializerVersion()

Specified by:

serializerVersion in interface IgniteWriteAheadLogManager

Returns:

Current serializer version.

log

public WALPointer log(WALRecord rec)
               throws IgniteCheckedException,
                      StorageException

Appends the given log entry to the write-ahead log.

Specified by:

log in interface IgniteWriteAheadLogManager

Parameters:

rec - entry to log.

Returns:

WALPointer that may be passed to IgniteWriteAheadLogManager.flush(WALPointer, boolean) method to make sure the record is written to the log.

Throws:

IgniteCheckedException - If failed to construct log entry.

StorageException - If IO error occurred while writing log entry.

flush

public void flush(WALPointer ptr,
                  boolean explicitFsync)
           throws IgniteCheckedException,
                  StorageException

Makes sure that all log entries written to the log up until the specified pointer are actually written to the underlying storage.

Specified by:

flush in interface IgniteWriteAheadLogManager

Parameters:

ptr - Optional pointer to write. If null, will sync up to the latest record.

explicitFsync - If true, data will be synced to the storage device on hardware level.

Throws:

IgniteCheckedException - If failed to write.

StorageException - If IO exception occurred during the write. If an exception is thrown from this method, the WAL will be invalidated and the node will be stopped.

replay

public WALIterator replay(WALPointer start)
                   throws IgniteCheckedException,
                          StorageException

Invoke this method to iterate over the written log entries.

Specified by:

replay in interface IgniteWriteAheadLogManager

Parameters:

start - Optional WAL pointer from which to start iteration.

Returns:

Records iterator.

Throws:

StorageException - If IO error occurred while reading WAL entries.

IgniteCheckedException

reserve

public boolean reserve(WALPointer start)
                throws IgniteCheckedException

Invoke this method to reserve WAL history since provided pointer and prevent it's deletion.

Specified by:

reserve in interface IgniteWriteAheadLogManager

Parameters:

start - WAL pointer.

Throws:

IgniteCheckedException

release

public void release(WALPointer start)

Invoke this method to release WAL history since provided pointer that was previously reserved.

Specified by:

release in interface IgniteWriteAheadLogManager

Parameters:

start - WAL pointer.

truncate

public int truncate(WALPointer low,
                    WALPointer high)

Gives a hint to WAL manager to clear entries logged before the given pointer. Some entries before the the given pointer will be kept because there is a configurable WAL history size. Those entries may be used for partial partition rebalancing.

Specified by:

truncate in interface IgniteWriteAheadLogManager

Parameters:

low - Pointer since which WAL will be truncated. If null, WAL will be truncated from the oldest segment.

high - Pointer for which it is safe to clear the log.

Returns:

Number of deleted WAL segments.

notchLastCheckpointPtr

public void notchLastCheckpointPtr(WALPointer ptr)

Notifies this about latest checkpoint pointer.

Current implementations, in fact, react by keeping all WAL segments uncompacted starting from index prior to the index of ptr. Compaction implies filtering out physical records and ZIP compression.

Specified by:

notchLastCheckpointPtr in interface IgniteWriteAheadLogManager

Parameters:

ptr - Pointer for which it is safe to compact the log.

walArchiveSegments

public int walArchiveSegments()

Specified by:

walArchiveSegments in interface IgniteWriteAheadLogManager

Returns:

Total number of segments in the WAL archive.

lastArchivedSegment

public long lastArchivedSegment()

Specified by:

lastArchivedSegment in interface IgniteWriteAheadLogManager

Returns:

Last archived segment index.

lastCompactedSegment

public long lastCompactedSegment()

Specified by:

lastCompactedSegment in interface IgniteWriteAheadLogManager

Returns:

Last compacted segment index.

reserved

public boolean reserved(WALPointer ptr)

Checks if WAL segment is under lock or reserved

Specified by:

reserved in interface IgniteWriteAheadLogManager

Parameters:

ptr - Pointer to check.

Returns:

True if given pointer is located in reserved segment.

reserved

public int reserved(WALPointer low,
                    WALPointer high)

Checks if WAL segments is under lock or reserved.

Specified by:

reserved in interface IgniteWriteAheadLogManager

Parameters:

low - Pointer since which WAL is locked or reserved. If null, checks from the oldest segment.

high - Pointer for which WAL is locked or reserved.

Returns:

Number of reserved WAL segments.

disabled

public boolean disabled(int grpId)

Checks WAL disabled for cache group.

Specified by:

disabled in interface IgniteWriteAheadLogManager

Parameters:

grpId - Group id.

cleanupWalDirectories

public void cleanupWalDirectories()
                           throws IgniteCheckedException

Cleanup all directories relating to WAL (e.g. work WAL dir, archive WAL dir).

Specified by:

cleanupWalDirectories in interface IgniteWriteAheadLogManager

Throws:

IgniteCheckedException

maxArchivedSegmentToDelete

public long maxArchivedSegmentToDelete()

Specified by:

maxArchivedSegmentToDelete in interface IgniteWriteAheadLogManager

Returns:

Max allowed index of archived segment to delete or -1 if it does not exist.

scan

public static FileDescriptor[] scan(File[] allFiles)

Returns:

Sorted WAL files descriptors.

setCreateWalFileListener

public void setCreateWalFileListener(@Nullable
                                     IgniteInClosure<FileIO> createWalFileListener)

Setup listener for WAL segment write File IO creation.

Parameters:

createWalFileListener - Listener to be invoked for new segment file IO creation.

maxWalSegmentSize

public long maxWalSegmentSize()

Returns:

maxWalSegmentSize.

loadFileDescriptors

public static FileDescriptor[] loadFileDescriptors(@NotNull
                                                   File walFilesDir)
                                            throws IgniteCheckedException

Scans provided folder for a WAL segment files

Parameters:

walFilesDir - directory to scan

Returns:

found WAL file descriptors

Throws:

IgniteCheckedException