org.apache.ignite.internal.processors.cache.persistence.file

Class FilePageStoreManager

java.lang.Object

org.apache.ignite.internal.processors.cache.GridCacheSharedManagerAdapter

org.apache.ignite.internal.processors.cache.persistence.file.FilePageStoreManager

All Implemented Interfaces:

IgnitePageStoreManager, PageStoreCollection, GridCacheSharedManager, PageReadWriteManager, IgniteChangeGlobalStateSupport

public class FilePageStoreManager
extends GridCacheSharedManagerAdapter
implements IgnitePageStoreManager, PageStoreCollection

File page store manager.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
protected static class	FilePageStoreManager.LongOperationAsyncExecutor Synchronization wrapper for long operations that should be executed asynchronously and operations that can not be executed in parallel with long operation.

Field Summary

Fields

Modifier and Type	Field and Description
static String	CACHE_DATA_FILENAME
static String	CACHE_DATA_TMP_FILENAME
static String	CACHE_DIR_PREFIX
static String	CACHE_GRP_DIR_PREFIX
static String	CORRUPTED_DATA_FILES_MNTC_TASK_NAME Unique name for corrupted data files maintenance task.
static Predicate<File>	DATA_DIR_FILTER
static String	DFLT_STORE_DIR
static String	FILE_SUFFIX File suffix.
static String	INDEX_FILE_NAME
static String	INDEX_FILE_PREFIX
static String	PART_FILE_PREFIX Partition file prefix.
static String	PART_FILE_TEMPLATE
static PathMatcher	TMP_FILE_MATCHER Matcher for searching of *.tmp files.
static String	TMP_SUFFIX Suffix for tmp files
static String	ZIP_SUFFIX Suffix for zip files

Fields inherited from class org.apache.ignite.internal.processors.cache.GridCacheSharedManagerAdapter

cctx, diagnosticLog, log

Constructor Summary

Constructors

Constructor and Description
FilePageStoreManager(GridKernalContext ctx)

Method Summary

Modifier and Type	Method and Description
long	allocatePage(int grpId, int partId, byte flags) Allocates a page for the given page space.
void	beginRecover() Invoked before starting checkpoint recover.
static File[]	cacheDataFiles(File root)
static List<File>	cacheDirectories(File dir, Predicate<String> names)
static File	cacheDirectory(File dir, int grpId)
static String	cacheDirName(boolean isSharedGroup, String cacheOrGroupName)
static String	cacheDirName(CacheConfiguration<?,?> ccfg)
String	cacheDirName(int grpId)
static String	cacheGroupName(File dir)
static List<File>	cachePartitionFiles(File cacheDir)
File	cacheWorkDir(boolean isSharedGroup, String cacheOrGroupName)
File	cacheWorkDir(CacheConfiguration<?,?> ccfg)
static File	cacheWorkDir(File storeWorkDir, String cacheDirName)
static boolean	checkAndInitCacheWorkDir(File cacheWorkDir, IgniteLogger log)
void	cleanupPageStoreIfMatch(Predicate<Integer> cacheGrpPred, boolean cleanFiles) Cleanup cache store whether it matches the provided predicate and if matched store was previously initizlized.
void	cleanupPersistentSpace() Cleanup persistent space for all caches except metastore.
void	cleanupPersistentSpace(CacheConfiguration cacheConfiguration) Cleanup persistent space for cache.
EncryptedFileIOFactory	encryptedFileIoFactory(FileIOFactory plainFileIOFactory, int cacheGrpId)
EncryptedFileIOFactory	encryptedFileIoFactory(FileIOFactory plainFileIOFactory, int cacheGrpId, EncryptionCacheKeyProvider encrKeyProvider)
void	ensure(int grpId, int partId)
boolean	exists(int grpId, int partId) Checks if partition store exists.
void	finishRecover() Invoked after checkpoint recover is finished.
FileVersionCheckingFactory	getPageStoreFactory(int grpId, boolean encrypted)
FileVersionCheckingFactory	getPageStoreFactory(int grpId, EncryptionCacheKeyProvider encrKeyProvider)
FileIOFactory	getPageStoreFileIoFactory()
static @NotNull File	getPartitionFile(File workDir, String cacheDirName, int partId)
static String	getPartitionFileName(int partId)
Path	getPath(boolean isSharedGroup, String cacheOrGroupName, int partId)
PageStore	getStore(int grpId, int partId)
Collection<PageStore>	getStores(int grpId)
boolean	hasIndexStore(int grpId)
void	initialize(int cacheId, int partitions, String cacheName, PageMetrics pageMetrics) Initializes disk store structures.
void	initializeForCache(CacheGroupDescriptor grpDesc, CacheConfiguration<?,?> ccfg) Callback called when a cache is starting.
void	initializeForMetastorage() Initializes disk cache store structures.
void	onActivate(GridKernalContext kctx) Called when cluster performing activation.
void	onDeActivate(GridKernalContext kctx) Called when cluster performing deactivation.
void	onKernalStop0(boolean cancel)
long	pageOffset(int grpId, long pageId) Gets page offset within the page store file.
int	pages(int grpId, int partId) Gets total number of allocated pages for the given space.
long	pagesAllocated(int grpId) Calculates number of pages currently allocated for given cache group.
int	pageSize()
static int	partId(String partFileName)
void	read(int grpId, long pageId, ByteBuffer pageBuf, boolean keepCrc) Reads a page for the given cache ID.
void	readHeader(int grpId, int partId, ByteBuffer buf) Reads a header of a page store.
void	setPageStoreFileIOFactories(FileIOFactory pageStoreFileIoFactory, FileIOFactory pageStoreV1FileIoFactory)
void	shutdownForCacheGroup(CacheGroupContext grp, boolean destroy) Callback called when a cache is stopping.
void	start0()
void	stop0(boolean cancel)
void	sync(int grpId, int partId) Makes sure that all previous writes to the store has been written to disk.
void	truncate(int grpId, int partId, int tag) Callback called when a partition for the given cache is evicted from the local node.
File	workDir()
PageStore	write(int grpId, long pageId, ByteBuffer pageBuf, int tag, boolean calculateCrc) Writes the page for the given cache ID.

Methods inherited from class org.apache.ignite.internal.processors.cache.GridCacheSharedManagerAdapter

context, isStopping, kernalStartInfo, kernalStopInfo, log, onDisconnected, onKernalStart, onKernalStart0, onKernalStop, onReconnected, printMemoryStats, start, startInfo, stop, stopInfo, toString

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.ignite.internal.processors.cache.GridCacheSharedManager

onDisconnected, onKernalStart, onKernalStop, onReconnected, printMemoryStats, start, stop

Field Detail

FILE_SUFFIX

public static final String FILE_SUFFIX

File suffix.

See Also:

Constant Field Values

ZIP_SUFFIX

public static final String ZIP_SUFFIX

Suffix for zip files

See Also:

Constant Field Values

TMP_SUFFIX

public static final String TMP_SUFFIX

Suffix for tmp files

See Also:

Constant Field Values

PART_FILE_PREFIX

public static final String PART_FILE_PREFIX

Partition file prefix.

See Also:

Constant Field Values

INDEX_FILE_PREFIX

public static final String INDEX_FILE_PREFIX

See Also:

Constant Field Values

INDEX_FILE_NAME

public static final String INDEX_FILE_NAME

See Also:

Constant Field Values

PART_FILE_TEMPLATE

public static final String PART_FILE_TEMPLATE

See Also:

Constant Field Values

CACHE_DIR_PREFIX

public static final String CACHE_DIR_PREFIX

See Also:

Constant Field Values

CACHE_GRP_DIR_PREFIX

public static final String CACHE_GRP_DIR_PREFIX

See Also:

Constant Field Values

DATA_DIR_FILTER

public static final Predicate<File> DATA_DIR_FILTER

CACHE_DATA_FILENAME

public static final String CACHE_DATA_FILENAME

See Also:

Constant Field Values

CACHE_DATA_TMP_FILENAME

public static final String CACHE_DATA_TMP_FILENAME

See Also:

Constant Field Values

DFLT_STORE_DIR

public static final String DFLT_STORE_DIR

See Also:

Constant Field Values

TMP_FILE_MATCHER

public static final PathMatcher TMP_FILE_MATCHER

Matcher for searching of *.tmp files.

CORRUPTED_DATA_FILES_MNTC_TASK_NAME

public static final String CORRUPTED_DATA_FILES_MNTC_TASK_NAME

Unique name for corrupted data files maintenance task.

See Also:

Constant Field Values

Constructor Detail

FilePageStoreManager

public FilePageStoreManager(GridKernalContext ctx)

Parameters:

ctx - Kernal context.

Method Detail

start0

public void start0()
            throws IgniteCheckedException

Overrides:

start0 in class GridCacheSharedManagerAdapter

Throws:

IgniteCheckedException - If failed.

cleanupPersistentSpace

public void cleanupPersistentSpace(CacheConfiguration cacheConfiguration)
                            throws IgniteCheckedException

Cleanup persistent space for cache.

Specified by:

cleanupPersistentSpace in interface IgnitePageStoreManager

Parameters:

cacheConfiguration - Cache configuration of cache which should be cleanup.

Throws:

IgniteCheckedException

cleanupPersistentSpace

public void cleanupPersistentSpace()
                            throws IgniteCheckedException

Cleanup persistent space for all caches except metastore.

Specified by:

cleanupPersistentSpace in interface IgnitePageStoreManager

Throws:

IgniteCheckedException

cleanupPageStoreIfMatch

public void cleanupPageStoreIfMatch(Predicate<Integer> cacheGrpPred,
                                    boolean cleanFiles)

Cleanup cache store whether it matches the provided predicate and if matched store was previously initizlized.

Specified by:

cleanupPageStoreIfMatch in interface IgnitePageStoreManager

Parameters:

cacheGrpPred - Predicate to match by id cache group stores to clean.

cleanFiles - True to delete all persisted files related to particular store.

stop0

public void stop0(boolean cancel)

Overrides:

stop0 in class GridCacheSharedManagerAdapter

Parameters:

cancel - Cancel flag.

onKernalStop0

public void onKernalStop0(boolean cancel)

Overrides:

onKernalStop0 in class GridCacheSharedManagerAdapter

Parameters:

cancel - Cancel flag.

onActivate

public void onActivate(GridKernalContext kctx)
                throws IgniteCheckedException

Called when cluster performing activation.

Specified by:

onActivate in interface IgniteChangeGlobalStateSupport

Parameters:

kctx - Kernal context.

Throws:

IgniteCheckedException - If failed.

onDeActivate

public void onDeActivate(GridKernalContext kctx)

Called when cluster performing deactivation.

Specified by:

onDeActivate in interface IgniteChangeGlobalStateSupport

Parameters:

kctx - Kernal context.

beginRecover

public void beginRecover()

Invoked before starting checkpoint recover.

Specified by:

beginRecover in interface IgnitePageStoreManager

finishRecover

public void finishRecover()
                   throws IgniteCheckedException

Invoked after checkpoint recover is finished.

Specified by:

finishRecover in interface IgnitePageStoreManager

Throws:

IgniteCheckedException

initialize

public void initialize(int cacheId,
                       int partitions,
                       String cacheName,
                       PageMetrics pageMetrics)
                throws IgniteCheckedException

Initializes disk store structures.

Specified by:

initialize in interface IgnitePageStoreManager

Parameters:

cacheId - Cache id.

partitions - Partitions count.

cacheName - Cache name.

pageMetrics - Page metrics.

Throws:

IgniteCheckedException - If failed.

initializeForCache

public void initializeForCache(CacheGroupDescriptor grpDesc,
                               CacheConfiguration<?,?> ccfg)
                        throws IgniteCheckedException

Callback called when a cache is starting.

Specified by:

initializeForCache in interface IgnitePageStoreManager

Parameters:

grpDesc - Cache group descriptor.

ccfg - Cache configuration.

Throws:

IgniteCheckedException - If failed to handle cache start callback.

initializeForMetastorage

public void initializeForMetastorage()
                              throws IgniteCheckedException

Initializes disk cache store structures.

Specified by:

initializeForMetastorage in interface IgnitePageStoreManager

Throws:

IgniteCheckedException

shutdownForCacheGroup

public void shutdownForCacheGroup(CacheGroupContext grp,
                                  boolean destroy)
                           throws IgniteCheckedException

Callback called when a cache is stopping. After this callback is invoked, no data associated with the given cache will be stored on disk.

Specified by:

shutdownForCacheGroup in interface IgnitePageStoreManager

Parameters:

grp - Cache group being stopped.

destroy - Flag indicating if the cache is being destroyed and data should be cleaned.

Throws:

IgniteCheckedException - If failed to handle cache destroy callback.

truncate

public void truncate(int grpId,
                     int partId,
                     int tag)
              throws IgniteCheckedException

Callback called when a partition for the given cache is evicted from the local node. After this callback is invoked, no data associated with the partition will be stored on disk.

Specified by:

truncate in interface IgnitePageStoreManager

Parameters:

grpId - Cache group ID of the evicted partition.

partId - Partition ID.

tag - Partition tag (growing 1-based partition file version).

Throws:

IgniteCheckedException - If failed to handle partition destroy callback.

read

public void read(int grpId,
                 long pageId,
                 ByteBuffer pageBuf,
                 boolean keepCrc)
          throws IgniteCheckedException

Reads a page for the given cache ID. Cache ID may be 0 if the page is a meta page.

Specified by:

read in interface PageReadWriteManager

Parameters:

grpId - Cache group ID.

pageId - PageID to read.

pageBuf - Page buffer to write to.

keepCrc - Keep CRC flag.

Throws:

IgniteCheckedException - If failed to read the page.

exists

public boolean exists(int grpId,
                      int partId)
               throws IgniteCheckedException

Checks if partition store exists.

Specified by:

exists in interface IgnitePageStoreManager

Parameters:

grpId - Cache group ID.

partId - Partition ID.

Returns:

True if partition store exists.

Throws:

IgniteCheckedException - If failed.

readHeader

public void readHeader(int grpId,
                       int partId,
                       ByteBuffer buf)
                throws IgniteCheckedException

Reads a header of a page store.

Specified by:

readHeader in interface IgnitePageStoreManager

Parameters:

grpId - Cache group ID.

partId - Partition ID.

buf - Buffer to write to.

Throws:

IgniteCheckedException - If failed.

write

public PageStore write(int grpId,
                       long pageId,
                       ByteBuffer pageBuf,
                       int tag,
                       boolean calculateCrc)
                throws IgniteCheckedException

Writes the page for the given cache ID. Cache ID may be 0 if the page is a meta page.

Specified by:

write in interface IgnitePageStoreManager

Specified by:

write in interface PageReadWriteManager

Parameters:

grpId - Cache group ID.

pageId - Page ID.

pageBuf - Page buffer to write.

Throws:

IgniteCheckedException - If failed to write page.

pageOffset

public long pageOffset(int grpId,
                       long pageId)
                throws IgniteCheckedException

Gets page offset within the page store file.

Specified by:

pageOffset in interface IgnitePageStoreManager

Parameters:

grpId - Cache group ID.

pageId - Page ID.

Returns:

Page offset.

Throws:

IgniteCheckedException - If failed.

getPath

public Path getPath(boolean isSharedGroup,
                    String cacheOrGroupName,
                    int partId)

getPageStoreFactory

public FileVersionCheckingFactory getPageStoreFactory(int grpId,
                                                      boolean encrypted)

Parameters:

grpId - Cache group id.

encrypted - true if cache group encryption enabled.

Returns:

Factory to create page stores.

getPageStoreFactory

public FileVersionCheckingFactory getPageStoreFactory(int grpId,
                                                      EncryptionCacheKeyProvider encrKeyProvider)

Parameters:

grpId - Cache group id.

encrKeyProvider - Encryption keys provider for encrypted IO. If null, no encryption is used.

Returns:

Factory to create page stores with certain encryption keys provider.

encryptedFileIoFactory

public EncryptedFileIOFactory encryptedFileIoFactory(FileIOFactory plainFileIOFactory,
                                                     int cacheGrpId,
                                                     EncryptionCacheKeyProvider encrKeyProvider)

Parameters:

plainFileIOFactory - Not-encrypting file io factory.

cacheGrpId - Cache group id.

encrKeyProvider - Encryption keys provider for encrypted IO. If null, no encryption is used.

Returns:

Encrypted file IO factory.

encryptedFileIoFactory

public EncryptedFileIOFactory encryptedFileIoFactory(FileIOFactory plainFileIOFactory,
                                                     int cacheGrpId)

Returns:

Encrypted file IO factory with stored internal encryption keys.

getPartitionFile

@NotNull
public static @NotNull File getPartitionFile(File workDir,
                                                      String cacheDirName,
                                                      int partId)

Parameters:

workDir - Cache work directory.

cacheDirName - Cache directory name.

partId - Partition id.

Returns:

Partition file.

getPartitionFileName

public static String getPartitionFileName(int partId)

Parameters:

partId - Partition id.

Returns:

File name.

checkAndInitCacheWorkDir

public static boolean checkAndInitCacheWorkDir(File cacheWorkDir,
                                               IgniteLogger log)
                                        throws IgniteCheckedException

Parameters:

cacheWorkDir - Cache work directory.

Throws:

IgniteCheckedException

sync

public void sync(int grpId,
                 int partId)
          throws IgniteCheckedException

Makes sure that all previous writes to the store has been written to disk.

Specified by:

sync in interface IgnitePageStoreManager

Parameters:

grpId - Cache group ID to sync.

partId - Partition ID to sync.

Throws:

IgniteCheckedException - If IO error occurred while running sync.

ensure

public void ensure(int grpId,
                   int partId)
            throws IgniteCheckedException

Specified by:

ensure in interface IgnitePageStoreManager

Parameters:

grpId - Cache group ID.

partId - Partition ID.

Throws:

IgniteCheckedException - If failed.

allocatePage

public long allocatePage(int grpId,
                         int partId,
                         byte flags)
                  throws IgniteCheckedException

Allocates a page for the given page space.

Specified by:

allocatePage in interface IgnitePageStoreManager

Specified by:

allocatePage in interface PageReadWriteManager

Parameters:

grpId - Cache group ID.

partId - Partition ID. Used only if flags is not equal to PageIdAllocator.FLAG_IDX.

flags - Page allocation flags.

Returns:

Allocated page ID.

Throws:

IgniteCheckedException - If IO exception occurred while allocating a page ID.

pages

public int pages(int grpId,
                 int partId)
          throws IgniteCheckedException

Gets total number of allocated pages for the given space.

Specified by:

pages in interface IgnitePageStoreManager

Parameters:

grpId - Cache group ID.

partId - Partition ID.

Returns:

Number of allocated pages.

Throws:

IgniteCheckedException - If failed.

cacheDirectories

public static List<File> cacheDirectories(File dir,
                                          Predicate<String> names)

Parameters:

dir - Directory to check.

names - Cache group names to filter.

Returns:

Files that match cache or cache group pattern.

cacheDirectory

public static File cacheDirectory(File dir,
                                  int grpId)

Parameters:

dir - Directory to check.

grpId - Cache group id

Returns:

Files that match cache or cache group pattern.

partId

public static int partId(String partFileName)

Parameters:

partFileName - Partition file name.

Returns:

Partition id.

cachePartitionFiles

public static List<File> cachePartitionFiles(File cacheDir)

Parameters:

cacheDir - Cache directory to check.

Returns:

List of cache partitions in given directory.

cacheGroupName

public static String cacheGroupName(File dir)

Parameters:

dir - Cache directory on disk.

Returns:

Cache or cache group name.

cacheDataFiles

public static File[] cacheDataFiles(File root)

Parameters:

root - Root directory.

Returns:

Array of cache data files.

hasIndexStore

public boolean hasIndexStore(int grpId)

Specified by:

hasIndexStore in interface IgnitePageStoreManager

Parameters:

grpId - Cache group ID.

Returns:

True if index store for given cache group existed before node started.

pagesAllocated

public long pagesAllocated(int grpId)

Calculates number of pages currently allocated for given cache group.

Specified by:

pagesAllocated in interface IgnitePageStoreManager

Parameters:

grpId - cache group id.

Returns:

number of pages.

workDir

public File workDir()

Returns:

Store work dir. Includes consistent-id based folder

cacheWorkDir

public File cacheWorkDir(CacheConfiguration<?,?> ccfg)

Parameters:

ccfg - Cache configuration.

Returns:

Store dir for given cache.

cacheWorkDir

public File cacheWorkDir(boolean isSharedGroup,
                         String cacheOrGroupName)

Parameters:

isSharedGroup - True if cache is sharing the same `underlying` cache.

cacheOrGroupName - Cache name.

Returns:

Store directory for given cache.

cacheWorkDir

public static File cacheWorkDir(File storeWorkDir,
                                String cacheDirName)

Parameters:

cacheDirName - Cache directory name.

Returns:

Store directory for given cache.

cacheDirName

public static String cacheDirName(boolean isSharedGroup,
                                  String cacheOrGroupName)

Parameters:

isSharedGroup - True if cache is sharing the same `underlying` cache.

cacheOrGroupName - Cache name.

Returns:

The full cache directory name.

cacheDirName

public static String cacheDirName(CacheConfiguration<?,?> ccfg)

Parameters:

ccfg - Cache configuration.

Returns:

The full cache directory name.

cacheDirName

public String cacheDirName(int grpId)
                    throws IgniteCheckedException

Parameters:

grpId - Group id.

Returns:

Name of cache group directory.

Throws:

IgniteCheckedException - If cache group doesn't exist.

getStores

public Collection<PageStore> getStores(int grpId)
                                throws IgniteCheckedException

Specified by:

getStores in interface PageStoreCollection

Parameters:

grpId - Cache group ID.

Returns:

Collection of related page stores.

Throws:

IgniteCheckedException - If failed.

getStore

public PageStore getStore(int grpId,
                          int partId)
                   throws IgniteCheckedException

Specified by:

getStore in interface PageStoreCollection

Parameters:

grpId - Cache group ID.

partId - Partition ID.

Returns:

Page store for the corresponding parameters.

Throws:

IgniteCheckedException - If cache or partition with the given ID was not created. Note: visible for testing.

setPageStoreFileIOFactories

public void setPageStoreFileIOFactories(FileIOFactory pageStoreFileIoFactory,
                                        FileIOFactory pageStoreV1FileIoFactory)

Parameters:

pageStoreFileIoFactory - File IO factory to override default, may be used for blocked read-write.

pageStoreV1FileIoFactory - File IO factory for reading V1 page store and for fast touching page files (non blocking).

getPageStoreFileIoFactory

public FileIOFactory getPageStoreFileIoFactory()

Returns:

File IO factory currently selected for page store.

pageSize

public int pageSize()

Returns:

Durable memory page size in bytes.