org.apache.ignite.internal.processors.cache.persistence.wal

Class FileWriteAheadLogManager

java.lang.Object

org.apache.ignite.internal.processors.cache.GridCacheSharedManagerAdapter

org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager

All Implemented Interfaces:

IgniteWriteAheadLogManager, GridCacheSharedManager, IgniteChangeGlobalStateSupport

public class FileWriteAheadLogManager
extends GridCacheSharedManagerAdapter
implements IgniteWriteAheadLogManager

File WAL manager.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	FileWriteAheadLogManager.ReadFileHandle

Field Summary

Fields

Modifier and Type	Field and Description
static Pattern	WAL_NAME_PATTERN Pattern for segment file names.
static FileFilter	WAL_SEGMENT_COMPACTED_OR_RAW_FILE_FILTER WAL segment file filter, see WAL_NAME_PATTERN
static Pattern	WAL_SEGMENT_FILE_COMPACTED_PATTERN
static FileFilter	WAL_SEGMENT_FILE_FILTER WAL segment file filter, see WAL_NAME_PATTERN
static Pattern	WAL_TEMP_NAME_PATTERN Pattern for WAL temp files - these files will be cleared at startup.

Fields inherited from class org.apache.ignite.internal.processors.cache.GridCacheSharedManagerAdapter

cctx, diagnosticLog, log

Constructor Summary

Constructors

Constructor and Description
FileWriteAheadLogManager(@NotNull GridKernalContext ctx)

Method Summary

Modifier and Type	Method and Description
boolean	disabled(int grpId) Checks WAL disabled for cache group.
void	flush(WALPointer ptr, boolean explicitFsync) Makes sure that all log entries written to the log up until the specified pointer are actually written to the underlying storage.
Collection<File>	getAndReserveWalFiles(FileWALPointer low, FileWALPointer high) Collect wal segment files from low pointer (include) to high pointer (not include) and reserve low pointer.
SegmentRouter	getSegmentRouter()
boolean	isAlwaysWriteFullPages()
boolean	isFullSync()
long	lastArchivedSegment()
long	lastCompactedSegment()
static FileDescriptor[]	loadFileDescriptors(@NotNull File walFilesDir) Scans provided folder for a WAL segment files
WALPointer	log(WALRecord rec) Appends the given log entry to the write-ahead log.
WALPointer	log(WALRecord rec, RolloverType rolloverType) Appends the given log entry to the write-ahead log.
long	maxArchivedSegmentToDelete()
long	maxWalSegmentSize()
void	notchLastCheckpointPtr(WALPointer ptr) Notifies this about latest checkpoint pointer.
void	onActivate(GridKernalContext kctx) Called when cluster performing activation.
void	onDeActivate(GridKernalContext kctx) Called when cluster performing deactivation.
static @NotNull ByteBuffer	prepareSerializerVersionBuffer(long idx, int ver, boolean compacted, ByteBuffer buf) Needs only for WAL compaction.
WALRecord	read(WALPointer ptr) Reads WAL record by the specified pointer.
void	release(WALPointer start) Invoke this method to release WAL history since provided pointer that was previously reserved.
WALIterator	replay(WALPointer start) Invoke this method to iterate over the written log entries.
WALIterator	replay(WALPointer start, @Nullable IgniteBiPredicate<WALRecord.RecordType,WALPointer> recordDeserializeFilter) Invoke this method to iterate over the written log entries.
boolean	reserve(WALPointer start) Invoke this method to reserve WAL history since provided pointer and prevent it's deletion.
boolean	reserved(WALPointer ptr) Checks if WAL segment is under lock or reserved
int	reserved(WALPointer low, WALPointer high) Checks if WAL segments is under lock or reserved.
void	resumeLogging(WALPointer lastPtr) Resumes logging after start.
static FileDescriptor[]	scan(File[] allFiles)
int	serializerVersion()
void	setCreateWalFileListener(@Nullable IgniteInClosure<FileIO> createWalFileListener) Setup listener for WAL segment write File IO creation.
void	setFileIOFactory(FileIOFactory ioFactory) For test purposes only.
void	start0()
protected void	stop0(boolean cancel) Method is called twice on deactivate and stop.
int	truncate(WALPointer low, WALPointer high) Gives a hint to WAL manager to clear entries logged before the given pointer.
int	walArchiveSegments()

Methods inherited from class org.apache.ignite.internal.processors.cache.GridCacheSharedManagerAdapter

context, isStopping, kernalStartInfo, kernalStopInfo, log, onDisconnected, onKernalStart, onKernalStart0, onKernalStop, onKernalStop0, onReconnected, printMemoryStats, start, startInfo, stop, stopInfo, toString

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.ignite.internal.processors.cache.GridCacheSharedManager

onDisconnected, onKernalStart, onKernalStop, onReconnected, printMemoryStats, start, stop

Field Detail

WAL_NAME_PATTERN

public static final Pattern WAL_NAME_PATTERN

Pattern for segment file names.

WAL_TEMP_NAME_PATTERN

public static final Pattern WAL_TEMP_NAME_PATTERN

Pattern for WAL temp files - these files will be cleared at startup.

WAL_SEGMENT_FILE_FILTER

public static final FileFilter WAL_SEGMENT_FILE_FILTER

WAL segment file filter, see WAL_NAME_PATTERN

WAL_SEGMENT_FILE_COMPACTED_PATTERN

public static final Pattern WAL_SEGMENT_FILE_COMPACTED_PATTERN

WAL_SEGMENT_COMPACTED_OR_RAW_FILE_FILTER

public static final FileFilter WAL_SEGMENT_COMPACTED_OR_RAW_FILE_FILTER

WAL segment file filter, see WAL_NAME_PATTERN

Constructor Detail

FileWriteAheadLogManager

public FileWriteAheadLogManager(@NotNull
                                @NotNull GridKernalContext ctx)

Parameters:

ctx - Kernal context.

Method Detail

setFileIOFactory

public void setFileIOFactory(FileIOFactory ioFactory)

For test purposes only.

Parameters:

ioFactory - IO factory.

start0

public void start0()
            throws IgniteCheckedException

Overrides:

start0 in class GridCacheSharedManagerAdapter

Throws:

IgniteCheckedException - If failed.

getSegmentRouter

public SegmentRouter getSegmentRouter()

Returns:

Info about of WAL paths.

getAndReserveWalFiles

public Collection<File> getAndReserveWalFiles(FileWALPointer low,
                                              FileWALPointer high)
                                       throws IgniteCheckedException

Collect wal segment files from low pointer (include) to high pointer (not include) and reserve low pointer.

Parameters:

low - Low bound.

high - High bound.

Throws:

IgniteCheckedException

stop0

protected void stop0(boolean cancel)

Method is called twice on deactivate and stop. It shutdown workers but do not deallocate them to avoid duplication.

Overrides:

stop0 in class GridCacheSharedManagerAdapter

Parameters:

cancel - Cancel flag.

onActivate

public void onActivate(GridKernalContext kctx)
                throws IgniteCheckedException

Called when cluster performing activation.

Specified by:

onActivate in interface IgniteChangeGlobalStateSupport

Parameters:

kctx - Kernal context.

Throws:

IgniteCheckedException - If failed.

onDeActivate

public void onDeActivate(GridKernalContext kctx)

Called when cluster performing deactivation.

Specified by:

onDeActivate in interface IgniteChangeGlobalStateSupport

Parameters:

kctx - Kernal context.

isAlwaysWriteFullPages

public boolean isAlwaysWriteFullPages()

Specified by:

isAlwaysWriteFullPages in interface IgniteWriteAheadLogManager

Returns:

true If we have to always write full pages.

isFullSync

public boolean isFullSync()

Specified by:

isFullSync in interface IgniteWriteAheadLogManager

Returns:

true if WAL will perform fair syncs on fsync call.

resumeLogging

public void resumeLogging(WALPointer lastPtr)
                   throws IgniteCheckedException

Resumes logging after start. When WAL manager is started, it will skip logging any updates until this method is called to avoid logging changes induced by the state restore procedure.

Specified by:

resumeLogging in interface IgniteWriteAheadLogManager

Throws:

IgniteCheckedException - If fails.

serializerVersion

public int serializerVersion()

Specified by:

serializerVersion in interface IgniteWriteAheadLogManager

Returns:

Current serializer version.

log

public WALPointer log(WALRecord rec)
               throws IgniteCheckedException

Appends the given log entry to the write-ahead log.

Specified by:

log in interface IgniteWriteAheadLogManager

Parameters:

rec - Entry to log.

Returns:

WALPointer that may be passed to IgniteWriteAheadLogManager.flush(WALPointer, boolean) method to make sure the record is written to the log.

Throws:

IgniteCheckedException - If failed to construct log entry.

StorageException - If IO error occurred while writing log entry.

log

public WALPointer log(WALRecord rec,
                      RolloverType rolloverType)
               throws IgniteCheckedException

Appends the given log entry to the write-ahead log. If entry logging leads to rollover, caller can specify whether to write the entry to the current segment or to th next one.

Specified by:

log in interface IgniteWriteAheadLogManager

Parameters:

rec - Entry to log.

rolloverType - Rollover type.

Returns:

WALPointer that may be passed to IgniteWriteAheadLogManager.flush(WALPointer, boolean) method to make sure the record is written to the log.

Throws:

IgniteCheckedException - If failed to construct log entry.

StorageException - If IO error occurred while writing log entry.

See Also:

RolloverType

flush

public void flush(WALPointer ptr,
                  boolean explicitFsync)
           throws IgniteCheckedException,
                  StorageException

Makes sure that all log entries written to the log up until the specified pointer are actually written to the underlying storage.

Specified by:

flush in interface IgniteWriteAheadLogManager

Parameters:

ptr - Optional pointer to write. If null, will sync up to the latest record.

explicitFsync - If true, data will be synced to the storage device on hardware level.

Throws:

IgniteCheckedException - If failed to write.

StorageException - If IO exception occurred during the write. If an exception is thrown from this method, the WAL will be invalidated and the node will be stopped.

read

public WALRecord read(WALPointer ptr)
               throws IgniteCheckedException,
                      StorageException

Reads WAL record by the specified pointer.

Specified by:

read in interface IgniteWriteAheadLogManager

Parameters:

ptr - WAL pointer.

Returns:

WAL record.

Throws:

IgniteCheckedException - If failed to read.

StorageException - If IO error occurred while reading WAL entries.

replay

public WALIterator replay(WALPointer start)
                   throws IgniteCheckedException,
                          StorageException

Invoke this method to iterate over the written log entries.

Specified by:

replay in interface IgniteWriteAheadLogManager

Parameters:

start - Optional WAL pointer from which to start iteration.

Returns:

Records iterator.

Throws:

StorageException - If IO error occurred while reading WAL entries.

IgniteCheckedException

replay

public WALIterator replay(WALPointer start,
                          @Nullable
                          @Nullable IgniteBiPredicate<WALRecord.RecordType,WALPointer> recordDeserializeFilter)
                   throws IgniteCheckedException,
                          StorageException

Invoke this method to iterate over the written log entries.

Specified by:

replay in interface IgniteWriteAheadLogManager

Parameters:

start - Optional WAL pointer from which to start iteration.

recordDeserializeFilter - Specify a filter to skip WAL records. Those records will not be explicitly deserialized.

Returns:

Records iterator.

Throws:

StorageException - If IO error occurred while reading WAL entries.

IgniteCheckedException

reserve

public boolean reserve(WALPointer start)

Invoke this method to reserve WAL history since provided pointer and prevent it's deletion.

Specified by:

reserve in interface IgniteWriteAheadLogManager

Parameters:

start - WAL pointer.

release

public void release(WALPointer start)

Invoke this method to release WAL history since provided pointer that was previously reserved.

Specified by:

release in interface IgniteWriteAheadLogManager

Parameters:

start - WAL pointer.

truncate

public int truncate(WALPointer low,
                    WALPointer high)

Gives a hint to WAL manager to clear entries logged before the given pointer. Some entries before the the given pointer will be kept because there is a configurable WAL history size. Those entries may be used for partial partition rebalancing.

Specified by:

truncate in interface IgniteWriteAheadLogManager

Parameters:

low - Pointer since which WAL will be truncated. If null, WAL will be truncated from the oldest segment.

high - Pointer for which it is safe to clear the log.

Returns:

Number of deleted WAL segments.

notchLastCheckpointPtr

public void notchLastCheckpointPtr(WALPointer ptr)

Notifies this about latest checkpoint pointer.

Current implementations, in fact, react by keeping all WAL segments uncompacted starting from index prior to the index of ptr. Compaction implies filtering out physical records and ZIP compression.

Specified by:

notchLastCheckpointPtr in interface IgniteWriteAheadLogManager

Parameters:

ptr - Pointer for which it is safe to compact the log.

walArchiveSegments

public int walArchiveSegments()

Specified by:

walArchiveSegments in interface IgniteWriteAheadLogManager

Returns:

Total number of segments in the WAL archive.

lastArchivedSegment

public long lastArchivedSegment()

Specified by:

lastArchivedSegment in interface IgniteWriteAheadLogManager

Returns:

Last archived segment index.

lastCompactedSegment

public long lastCompactedSegment()

Specified by:

lastCompactedSegment in interface IgniteWriteAheadLogManager

Returns:

Last compacted segment index.

reserved

public boolean reserved(WALPointer ptr)

Checks if WAL segment is under lock or reserved

Specified by:

reserved in interface IgniteWriteAheadLogManager

Parameters:

ptr - Pointer to check.

Returns:

True if given pointer is located in reserved segment.

reserved

public int reserved(WALPointer low,
                    WALPointer high)

Checks if WAL segments is under lock or reserved.

Specified by:

reserved in interface IgniteWriteAheadLogManager

Parameters:

low - Pointer since which WAL is locked or reserved. If null, checks from the oldest segment.

high - Pointer for which WAL is locked or reserved.

Returns:

Number of reserved WAL segments.

disabled

public boolean disabled(int grpId)

Checks WAL disabled for cache group.

Specified by:

disabled in interface IgniteWriteAheadLogManager

Parameters:

grpId - Group id.

maxArchivedSegmentToDelete

public long maxArchivedSegmentToDelete()

Specified by:

maxArchivedSegmentToDelete in interface IgniteWriteAheadLogManager

Returns:

Max allowed index of archived segment to delete or -1 if it does not exist.

scan

public static FileDescriptor[] scan(File[] allFiles)

Returns:

Sorted WAL files descriptors.

setCreateWalFileListener

public void setCreateWalFileListener(@Nullable
                                     @Nullable IgniteInClosure<FileIO> createWalFileListener)

Setup listener for WAL segment write File IO creation.

Parameters:

createWalFileListener - Listener to be invoked for new segment file IO creation.

maxWalSegmentSize

public long maxWalSegmentSize()

Returns:

maxWalSegmentSize.

prepareSerializerVersionBuffer

@NotNull
public static @NotNull ByteBuffer prepareSerializerVersionBuffer(long idx,
                                                                          int ver,
                                                                          boolean compacted,
                                                                          ByteBuffer buf)

Needs only for WAL compaction.

Parameters:

idx - Index.

ver - Version.

compacted - Compacted flag.

loadFileDescriptors

public static FileDescriptor[] loadFileDescriptors(@NotNull
                                                   @NotNull File walFilesDir)
                                            throws IgniteCheckedException

Scans provided folder for a WAL segment files

Parameters:

walFilesDir - directory to scan

Returns:

found WAL file descriptors

Throws:

IgniteCheckedException