org.apache.ignite.internal.processors.cache.persistence.file

Class FilePageStoreManager

java.lang.Object

org.apache.ignite.internal.processors.cache.GridCacheSharedManagerAdapter

org.apache.ignite.internal.processors.cache.persistence.file.FilePageStoreManager

All Implemented Interfaces:

IgnitePageStoreManager, GridCacheSharedManager, IgniteChangeGlobalStateSupport

public class FilePageStoreManager
extends GridCacheSharedManagerAdapter
implements IgnitePageStoreManager

File page store manager.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
protected static class	FilePageStoreManager.LongOperationAsyncExecutor Synchronization wrapper for long operations that should be executed asynchronously and operations that can not be executed in parallel with long operation.

Field Summary

Fields

Modifier and Type	Field and Description
static String	CACHE_DATA_FILENAME
static String	CACHE_DATA_TMP_FILENAME
static String	CACHE_DIR_PREFIX
static String	CACHE_GRP_DIR_PREFIX
static String	DFLT_STORE_DIR
static String	FILE_SUFFIX File suffix.
static String	INDEX_FILE_NAME
static String	META_STORAGE_NAME
static String	PART_FILE_PREFIX Partition file prefix.
static String	PART_FILE_TEMPLATE
static PathMatcher	TMP_FILE_MATCHER Matcher for searching of *.tmp files.
static String	TMP_SUFFIX Suffix for tmp files
static String	ZIP_SUFFIX Suffix for zip files

Fields inherited from class org.apache.ignite.internal.processors.cache.GridCacheSharedManagerAdapter

cctx, diagnosticLog, log

Constructor Summary

Constructors

Constructor and Description
FilePageStoreManager(GridKernalContext ctx)

Method Summary

Modifier and Type	Method and Description
void	addConfigurationChangeListener(BiConsumer<String,File> lsnr)
long	allocatePage(int grpId, int partId, byte flags) Allocates a page for the given page space.
void	beforeCacheGroupStart(CacheGroupDescriptor grpDesc)
void	beginRecover() Invoked before starting checkpoint recover.
static String	cacheDirName(boolean isSharedGroup, String cacheOrGroupName)
static String	cacheDirName(CacheConfiguration<?,?> ccfg)
File	cacheWorkDir(boolean isSharedGroup, String cacheOrGroupName)
File	cacheWorkDir(CacheConfiguration<?,?> ccfg)
static File	cacheWorkDir(File storeWorkDir, String cacheDirName)
boolean	checkAndInitCacheWorkDir(CacheConfiguration cacheCfg) Creates and initializes cache work directory retrieved from cacheCfg.
void	cleanupPageStoreIfMatch(Predicate<Integer> cacheGrpPred, boolean cleanFiles) Cleanup cache store whether it matches the provided predicate and if matched store was previously initizlized.
void	cleanupPersistentSpace() Cleanup persistent space for all caches except metastore.
void	cleanupPersistentSpace(CacheConfiguration cacheConfiguration) Cleanup persistent space for cache.
void	ensure(int grpId, int partId)
boolean	exists(int grpId, int partId) Checks if partition store exists.
void	finishRecover() Invoked after checkpoint recover is finished.
FilePageStoreFactory	getPageStoreFactory(int grpId, boolean encrypted)
FileIOFactory	getPageStoreFileIoFactory()
static @NotNull File	getPartitionFile(File workDir, String cacheDirName, int partId)
static String	getPartitionFileName(int partId)
Path	getPath(boolean isSharedGroup, String cacheOrGroupName, int partId)
PageStore	getStore(int grpId, int partId)
Collection<PageStore>	getStores(int grpId)
boolean	hasIndexStore(int grpId)
void	initialize(int cacheId, int partitions, String workingDir, LongAdderMetric tracker) Initializes disk store structures.
void	initializeForCache(CacheGroupDescriptor grpDesc, StoredCacheData cacheData) Callback called when a cache is starting.
void	initializeForMetastorage() Initializes disk cache store structures.
long	metaPageId(int grpId) Gets meta page ID for specified cache.
void	onActivate(GridKernalContext kctx) Called when cluster performing activation.
void	onDeActivate(GridKernalContext kctx) Called when cluster performing deactivation.
void	onKernalStop0(boolean cancel)
void	onPartitionCreated(int grpId, int partId) Callback called when a partition is created on the local node.
void	onPartitionDestroyed(int grpId, int partId, int tag) Callback called when a partition for the given cache is evicted from the local node.
long	pageOffset(int grpId, long pageId) Gets page offset within the page store file.
int	pages(int grpId, int partId) Gets total number of allocated pages for the given space.
long	pagesAllocated(int grpId) Calculates number of pages currently allocated for given cache group.
int	pageSize()
void	read(int grpId, long pageId, ByteBuffer pageBuf) Reads a page for the given cache ID.
void	read(int grpId, long pageId, ByteBuffer pageBuf, boolean keepCrc) Will preserve crc in buffer if keepCrc is true.
Map<String,StoredCacheData>	readCacheConfigurations()
void	readConfigurationFiles(List<CacheConfiguration<?,?>> ccfgs, BiConsumer<CacheConfiguration<?,?>,File> ccfgCons)
void	readHeader(int grpId, int partId, ByteBuffer buf) Reads a header of a page store.
void	removeCacheData(StoredCacheData cacheData) Remove cache configuration data file.
void	removeConfigurationChangeListener(BiConsumer<String,File> lsnr)
void	setPageStoreFileIOFactories(FileIOFactory pageStoreFileIoFactory, FileIOFactory pageStoreV1FileIoFactory)
void	shutdownForCacheGroup(CacheGroupContext grp, boolean destroy) Callback called when a cache is stopping.
void	start0()
void	stop0(boolean cancel)
void	storeCacheData(StoredCacheData cacheData, boolean overwrite)
void	sync(int grpId, int partId) Makes sure that all previous writes to the store has been written to disk.
File	workDir()
void	write(int grpId, long pageId, ByteBuffer pageBuf, int tag) Writes the page for the given cache ID.
PageStore	writeInternal(int cacheId, long pageId, ByteBuffer pageBuf, int tag, boolean calculateCrc)

Methods inherited from class org.apache.ignite.internal.processors.cache.GridCacheSharedManagerAdapter

context, isStopping, kernalStartInfo, kernalStopInfo, log, onDisconnected, onKernalStart, onKernalStart0, onKernalStop, onReconnected, printMemoryStats, start, startInfo, stop, stopInfo, toString

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.ignite.internal.processors.cache.GridCacheSharedManager

onDisconnected, onKernalStart, onKernalStop, onReconnected, printMemoryStats, start, stop

Field Detail

FILE_SUFFIX

public static final String FILE_SUFFIX

File suffix.

See Also:

Constant Field Values

ZIP_SUFFIX

public static final String ZIP_SUFFIX

Suffix for zip files

See Also:

Constant Field Values

TMP_SUFFIX

public static final String TMP_SUFFIX

Suffix for tmp files

See Also:

Constant Field Values

PART_FILE_PREFIX

public static final String PART_FILE_PREFIX

Partition file prefix.

See Also:

Constant Field Values

INDEX_FILE_NAME

public static final String INDEX_FILE_NAME

See Also:

Constant Field Values

PART_FILE_TEMPLATE

public static final String PART_FILE_TEMPLATE

See Also:

Constant Field Values

CACHE_DIR_PREFIX

public static final String CACHE_DIR_PREFIX

See Also:

Constant Field Values

CACHE_GRP_DIR_PREFIX

public static final String CACHE_GRP_DIR_PREFIX

See Also:

Constant Field Values

CACHE_DATA_FILENAME

public static final String CACHE_DATA_FILENAME

See Also:

Constant Field Values

CACHE_DATA_TMP_FILENAME

public static final String CACHE_DATA_TMP_FILENAME

See Also:

Constant Field Values

DFLT_STORE_DIR

public static final String DFLT_STORE_DIR

See Also:

Constant Field Values

META_STORAGE_NAME

public static final String META_STORAGE_NAME

See Also:

Constant Field Values

TMP_FILE_MATCHER

public static final PathMatcher TMP_FILE_MATCHER

Matcher for searching of *.tmp files.

Constructor Detail

FilePageStoreManager

public FilePageStoreManager(GridKernalContext ctx)

Parameters:

ctx - Kernal context.

Method Detail

start0

public void start0()
            throws IgniteCheckedException

Overrides:

start0 in class GridCacheSharedManagerAdapter

Throws:

IgniteCheckedException - If failed.

cleanupPersistentSpace

public void cleanupPersistentSpace(CacheConfiguration cacheConfiguration)
                            throws IgniteCheckedException

Cleanup persistent space for cache.

Specified by:

cleanupPersistentSpace in interface IgnitePageStoreManager

Parameters:

cacheConfiguration - Cache configuration of cache which should be cleanup.

Throws:

IgniteCheckedException

cleanupPersistentSpace

public void cleanupPersistentSpace()
                            throws IgniteCheckedException

Cleanup persistent space for all caches except metastore.

Specified by:

cleanupPersistentSpace in interface IgnitePageStoreManager

Throws:

IgniteCheckedException

cleanupPageStoreIfMatch

public void cleanupPageStoreIfMatch(Predicate<Integer> cacheGrpPred,
                                    boolean cleanFiles)

Cleanup cache store whether it matches the provided predicate and if matched store was previously initizlized.

Specified by:

cleanupPageStoreIfMatch in interface IgnitePageStoreManager

Parameters:

cacheGrpPred - Predicate to match by id cache group stores to clean.

cleanFiles - True to delete all persisted files related to particular store.

stop0

public void stop0(boolean cancel)

Overrides:

stop0 in class GridCacheSharedManagerAdapter

Parameters:

cancel - Cancel flag.

onKernalStop0

public void onKernalStop0(boolean cancel)

Overrides:

onKernalStop0 in class GridCacheSharedManagerAdapter

Parameters:

cancel - Cancel flag.

onActivate

public void onActivate(GridKernalContext kctx)
                throws IgniteCheckedException

Called when cluster performing activation.

Specified by:

onActivate in interface IgniteChangeGlobalStateSupport

Parameters:

kctx - Kernal context.

Throws:

IgniteCheckedException - If failed.

onDeActivate

public void onDeActivate(GridKernalContext kctx)

Called when cluster performing deactivation.

Specified by:

onDeActivate in interface IgniteChangeGlobalStateSupport

Parameters:

kctx - Kernal context.

beginRecover

public void beginRecover()

Invoked before starting checkpoint recover.

Specified by:

beginRecover in interface IgnitePageStoreManager

finishRecover

public void finishRecover()
                   throws IgniteCheckedException

Invoked after checkpoint recover is finished.

Specified by:

finishRecover in interface IgnitePageStoreManager

Throws:

IgniteCheckedException

initialize

public void initialize(int cacheId,
                       int partitions,
                       String workingDir,
                       LongAdderMetric tracker)
                throws IgniteCheckedException

Initializes disk store structures.

Specified by:

initialize in interface IgnitePageStoreManager

Parameters:

cacheId - Cache id.

partitions - Partitions count.

workingDir - Working directory.

tracker - Allocation tracker.

Throws:

IgniteCheckedException - If failed.

initializeForCache

public void initializeForCache(CacheGroupDescriptor grpDesc,
                               StoredCacheData cacheData)
                        throws IgniteCheckedException

Callback called when a cache is starting.

Specified by:

initializeForCache in interface IgnitePageStoreManager

Parameters:

grpDesc - Cache group descriptor.

cacheData - Cache data of the cache being started.

Throws:

IgniteCheckedException - If failed to handle cache start callback.

initializeForMetastorage

public void initializeForMetastorage()
                              throws IgniteCheckedException

Initializes disk cache store structures.

Specified by:

initializeForMetastorage in interface IgnitePageStoreManager

Throws:

IgniteCheckedException

storeCacheData

public void storeCacheData(StoredCacheData cacheData,
                           boolean overwrite)
                    throws IgniteCheckedException

Specified by:

storeCacheData in interface IgnitePageStoreManager

Parameters:

cacheData - Cache configuration.

overwrite - Whether stored configuration should be overwritten if it exists.

Throws:

IgniteCheckedException - If failed.

addConfigurationChangeListener

public void addConfigurationChangeListener(BiConsumer<String,File> lsnr)

Parameters:

lsnr - Instance of listener to add.

removeConfigurationChangeListener

public void removeConfigurationChangeListener(BiConsumer<String,File> lsnr)

Parameters:

lsnr - Instance of listener to remove.

shutdownForCacheGroup

public void shutdownForCacheGroup(CacheGroupContext grp,
                                  boolean destroy)
                           throws IgniteCheckedException

Callback called when a cache is stopping. After this callback is invoked, no data associated with the given cache will be stored on disk.

Specified by:

shutdownForCacheGroup in interface IgnitePageStoreManager

Parameters:

grp - Cache group being stopped.

destroy - Flag indicating if the cache is being destroyed and data should be cleaned.

Throws:

IgniteCheckedException - If failed to handle cache destroy callback.

onPartitionCreated

public void onPartitionCreated(int grpId,
                               int partId)

Callback called when a partition is created on the local node.

Specified by:

onPartitionCreated in interface IgnitePageStoreManager

Parameters:

grpId - Cache group ID where the partition is being created.

partId - ID of the partition being created.

onPartitionDestroyed

public void onPartitionDestroyed(int grpId,
                                 int partId,
                                 int tag)
                          throws IgniteCheckedException

Callback called when a partition for the given cache is evicted from the local node. After this callback is invoked, no data associated with the partition will be stored on disk.

Specified by:

onPartitionDestroyed in interface IgnitePageStoreManager

Parameters:

grpId - Cache group ID of the evicted partition.

partId - Partition ID.

tag - Partition tag (growing 1-based partition file version).

Throws:

IgniteCheckedException - If failed to handle partition destroy callback.

read

public void read(int grpId,
                 long pageId,
                 ByteBuffer pageBuf)
          throws IgniteCheckedException

Reads a page for the given cache ID. Cache ID may be 0 if the page is a meta page.

Specified by:

read in interface IgnitePageStoreManager

Parameters:

grpId - Cache group ID.

pageId - PageID to read.

pageBuf - Page buffer to write to.

Throws:

IgniteCheckedException - If failed to read the page.

read

public void read(int grpId,
                 long pageId,
                 ByteBuffer pageBuf,
                 boolean keepCrc)
          throws IgniteCheckedException

Will preserve crc in buffer if keepCrc is true.

Parameters:

grpId - Group ID.

pageId - Page ID.

pageBuf - Page buffer.

keepCrc - Keep CRC flag.

Throws:

IgniteCheckedException - If failed.

exists

public boolean exists(int grpId,
                      int partId)
               throws IgniteCheckedException

Checks if partition store exists.

Specified by:

exists in interface IgnitePageStoreManager

Parameters:

grpId - Cache group ID.

partId - Partition ID.

Returns:

True if partition store exists.

Throws:

IgniteCheckedException - If failed.

readHeader

public void readHeader(int grpId,
                       int partId,
                       ByteBuffer buf)
                throws IgniteCheckedException

Reads a header of a page store.

Specified by:

readHeader in interface IgnitePageStoreManager

Parameters:

grpId - Cache group ID.

partId - Partition ID.

buf - Buffer to write to.

Throws:

IgniteCheckedException - If failed.

write

public void write(int grpId,
                  long pageId,
                  ByteBuffer pageBuf,
                  int tag)
           throws IgniteCheckedException

Writes the page for the given cache ID. Cache ID may be 0 if the page is a meta page.

Specified by:

write in interface IgnitePageStoreManager

Parameters:

grpId - Cache group ID.

pageId - Page ID.

pageBuf - Page buffer to write.

Throws:

IgniteCheckedException - If failed to write page.

pageOffset

public long pageOffset(int grpId,
                       long pageId)
                throws IgniteCheckedException

Gets page offset within the page store file.

Specified by:

pageOffset in interface IgnitePageStoreManager

Parameters:

grpId - Cache group ID.

pageId - Page ID.

Returns:

Page offset.

Throws:

IgniteCheckedException - If failed.

writeInternal

public PageStore writeInternal(int cacheId,
                               long pageId,
                               ByteBuffer pageBuf,
                               int tag,
                               boolean calculateCrc)
                        throws IgniteCheckedException

Parameters:

cacheId - Cache ID to write.

pageId - Page ID.

pageBuf - Page buffer.

tag - Partition tag (growing 1-based partition file version). Used to validate page is not outdated

calculateCrc - if False crc calculation will be forcibly skipped.

Returns:

PageStore to which the page has been written.

Throws:

IgniteCheckedException - If IO error occurred.

getPath

public Path getPath(boolean isSharedGroup,
                    String cacheOrGroupName,
                    int partId)

getPageStoreFactory

public FilePageStoreFactory getPageStoreFactory(int grpId,
                                                boolean encrypted)

Parameters:

grpId - Cache group id.

encrypted - true if cache group encryption enabled.

Returns:

Factory to create page stores.

getPartitionFile

@NotNull
public static @NotNull File getPartitionFile(File workDir,
                                                      String cacheDirName,
                                                      int partId)

Parameters:

workDir - Cache work directory.

cacheDirName - Cache directory name.

partId - Partition id.

Returns:

Partition file.

getPartitionFileName

public static String getPartitionFileName(int partId)

Parameters:

partId - Partition id.

Returns:

File name.

checkAndInitCacheWorkDir

public boolean checkAndInitCacheWorkDir(CacheConfiguration cacheCfg)
                                 throws IgniteCheckedException

Creates and initializes cache work directory retrieved from cacheCfg.

Specified by:

checkAndInitCacheWorkDir in interface IgnitePageStoreManager

Parameters:

cacheCfg - Cache configuration.

Returns:

True if work directory already exists.

Throws:

IgniteCheckedException - If failed.

sync

public void sync(int grpId,
                 int partId)
          throws IgniteCheckedException

Makes sure that all previous writes to the store has been written to disk.

Specified by:

sync in interface IgnitePageStoreManager

Parameters:

grpId - Cache group ID to sync.

partId - Partition ID to sync.

Throws:

IgniteCheckedException - If IO error occurred while running sync.

ensure

public void ensure(int grpId,
                   int partId)
            throws IgniteCheckedException

Specified by:

ensure in interface IgnitePageStoreManager

Parameters:

grpId - Cache group ID.

partId - Partition ID.

Throws:

IgniteCheckedException - If failed.

allocatePage

public long allocatePage(int grpId,
                         int partId,
                         byte flags)
                  throws IgniteCheckedException

Allocates a page for the given page space.

Specified by:

allocatePage in interface IgnitePageStoreManager

Parameters:

grpId - Cache group ID.

partId - Partition ID. Used only if flags is equal to PageIdAllocator.FLAG_DATA.

flags - Page allocation flags.

Returns:

Allocated page ID.

Throws:

IgniteCheckedException - If IO exception occurred while allocating a page ID.

metaPageId

public long metaPageId(int grpId)

Gets meta page ID for specified cache.

Specified by:

metaPageId in interface IgnitePageStoreManager

Parameters:

grpId - Cache group ID.

Returns:

Meta page ID.

pages

public int pages(int grpId,
                 int partId)
          throws IgniteCheckedException

Gets total number of allocated pages for the given space.

Specified by:

pages in interface IgnitePageStoreManager

Parameters:

grpId - Cache group ID.

partId - Partition ID.

Returns:

Number of allocated pages.

Throws:

IgniteCheckedException - If failed.

readConfigurationFiles

public void readConfigurationFiles(List<CacheConfiguration<?,?>> ccfgs,
                                   BiConsumer<CacheConfiguration<?,?>,File> ccfgCons)

Parameters:

ccfgs - List of cache configurations to process.

ccfgCons - Consumer which accepts found configurations files.

readCacheConfigurations

public Map<String,StoredCacheData> readCacheConfigurations()
                                                    throws IgniteCheckedException

Specified by:

readCacheConfigurations in interface IgnitePageStoreManager

Returns:

Saved cache configurations.

Throws:

IgniteCheckedException - If failed.

hasIndexStore

public boolean hasIndexStore(int grpId)

Specified by:

hasIndexStore in interface IgnitePageStoreManager

Parameters:

grpId - Cache group ID.

Returns:

True if index store for given cache group existed before node started.

pagesAllocated

public long pagesAllocated(int grpId)

Calculates number of pages currently allocated for given cache group.

Specified by:

pagesAllocated in interface IgnitePageStoreManager

Parameters:

grpId - cache group id.

Returns:

number of pages.

workDir

public File workDir()

Returns:

Store work dir. Includes consistent-id based folder

cacheWorkDir

public File cacheWorkDir(CacheConfiguration<?,?> ccfg)

Parameters:

ccfg - Cache configuration.

Returns:

Store dir for given cache.

cacheWorkDir

public File cacheWorkDir(boolean isSharedGroup,
                         String cacheOrGroupName)

Parameters:

isSharedGroup - True if cache is sharing the same `underlying` cache.

cacheOrGroupName - Cache name.

Returns:

Store directory for given cache.

cacheWorkDir

public static File cacheWorkDir(File storeWorkDir,
                                String cacheDirName)

Parameters:

cacheDirName - Cache directory name.

Returns:

Store directory for given cache.

cacheDirName

public static String cacheDirName(boolean isSharedGroup,
                                  String cacheOrGroupName)

Parameters:

isSharedGroup - True if cache is sharing the same `underlying` cache.

cacheOrGroupName - Cache name.

Returns:

The full cache directory name.

cacheDirName

public static String cacheDirName(CacheConfiguration<?,?> ccfg)

Parameters:

ccfg - Cache configuration.

Returns:

The full cache directory name.

removeCacheData

public void removeCacheData(StoredCacheData cacheData)
                     throws IgniteCheckedException

Remove cache configuration data file.

Specified by:

removeCacheData in interface IgnitePageStoreManager

Parameters:

cacheData - Cache configuration.

Throws:

IgniteCheckedException - If failed.

getStores

public Collection<PageStore> getStores(int grpId)
                                throws IgniteCheckedException

Parameters:

grpId - Cache group ID.

Returns:

Collection of related page stores.

Throws:

IgniteCheckedException - If failed.

getStore

public PageStore getStore(int grpId,
                          int partId)
                   throws IgniteCheckedException

Parameters:

grpId - Cache group ID.

partId - Partition ID.

Returns:

Page store for the corresponding parameters.

Throws:

IgniteCheckedException - If cache or partition with the given ID was not created. Note: visible for testing.

beforeCacheGroupStart

public void beforeCacheGroupStart(CacheGroupDescriptor grpDesc)

Specified by:

beforeCacheGroupStart in interface IgnitePageStoreManager

Parameters:

grpDesc - Cache group descriptor.

setPageStoreFileIOFactories

public void setPageStoreFileIOFactories(FileIOFactory pageStoreFileIoFactory,
                                        FileIOFactory pageStoreV1FileIoFactory)

Parameters:

pageStoreFileIoFactory - File IO factory to override default, may be used for blocked read-write.

pageStoreV1FileIoFactory - File IO factory for reading V1 page store and for fast touching page files (non blocking).

getPageStoreFileIoFactory

public FileIOFactory getPageStoreFileIoFactory()

Returns:

File IO factory currently selected for page store.

pageSize

public int pageSize()

Returns:

Durable memory page size in bytes.