org.apache.ignite.internal.processors.cache.persistence.pagemem

Class PageMemoryImpl

java.lang.Object

org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl

All Implemented Interfaces:

PageIdAllocator, PageMemory, PageSupport, PageMemoryEx, LifecycleAware

public class PageMemoryImpl
extends Object
implements PageMemoryEx

Page header structure is described by the following diagram. When page is not allocated (in a free list):

 +--------+------------------------------------------------------+
 |8 bytes |         PAGE_SIZE + PAGE_OVERHEAD - 8 bytes          |
 +--------+------------------------------------------------------+
 |Next ptr|                      Page data                       |
 +--------+------------------------------------------------------+
 

When page is allocated and is in use:

 +------------------+--------+--------+----+----+--------+--------+----------------------+
 |     8 bytes      |8 bytes |8 bytes |4 b |4 b |8 bytes |8 bytes |       PAGE_SIZE      |
 +------------------+--------+--------+----+----+--------+--------+----------------------+
 | Marker/Timestamp |Rel ptr |Page ID |C ID|PIN | LOCK   |TMP BUF |       Page data      |
 +------------------+--------+--------+----+----+--------+--------+----------------------+
 

Note that first 8 bytes of page header are used either for page marker or for next relative pointer depending on whether the page is in use or not.

Field Summary

Fields

Modifier and Type	Field and Description
static int	PAGE_CACHE_ID_OFFSET Page cache ID offset.
static int	PAGE_ID_OFFSET Page ID offset
static int	PAGE_LOCK_OFFSET Page lock offset.
static long	PAGE_MARKER
static int	PAGE_OVERHEAD 8b Marker/timestamp 8b Relative pointer 8b Page ID 4b Cache ID 4b Pin count 8b Lock 8b Temporary buffer
static int	PAGE_PIN_CNT_OFFSET Page pin counter offset.
static int	PAGE_TMP_BUF_OFFSET Page lock offset.
static int	RANDOM_PAGES_EVICT_NUM Number of random pages that will be picked for eviction.
static int	RELATIVE_PTR_OFFSET Page relative pointer.

Fields inherited from interface org.apache.ignite.internal.pagemem.PageIdAllocator

FLAG_DATA, FLAG_IDX, INDEX_PARTITION, MAX_PARTITION_ID

Constructor Summary

Constructors

Constructor and Description
PageMemoryImpl(DirectMemoryProvider directMemoryProvider, long[] sizes, GridCacheSharedContext<?,?> ctx, int pageSize, GridInClosure3X<FullPageId,ByteBuffer,Integer> flushDirtyPage, GridInClosure3X<Long,FullPageId,PageMemoryEx> changeTracker, CheckpointLockStateChecker stateChecker, DataRegionMetricsImpl memMetrics, boolean throttleEnabled)

Method Summary

Modifier and Type	Method and Description
long	acquiredPages()
long	acquirePage(int cacheId, long pageId) Gets the page absolute pointer associated with the given page ID.
long	acquirePage(int cacheId, long pageId, boolean restore)
int	activePagesCount() Gets the number of active pages across all segments.
long	allocatePage(int cacheId, int partId, byte flags) Allocates a page from the space for the given partition ID and the given flags.
GridMultiCollectionWrapper<FullPageId>	beginCheckpoint() Gets a collection of dirty page IDs since the last checkpoint.
int	checkpointBufferPagesCount() Number of pages used in checkpoint buffer.
int	checkpointBufferPagesSize() Number of used pages in checkpoint buffer.
IgniteInternalFuture<Void>	clearAsync(GridPredicate3<Integer,Long,Integer> pred, boolean cleanDirty) Asynchronously clears pages satisfying the given predicate.
void	finishCheckpoint() Finishes checkpoint operation.
boolean	freePage(int cacheId, long pageId) The given page is free now.
Integer	getForCheckpoint(FullPageId fullId, ByteBuffer outBuf, CheckpointMetricsTracker tracker) Gets page byte buffer for the checkpoint procedure.
boolean	hasTempCopy(long absPtr) Checks if a page has temp copy buffer.
int	invalidate(int grpId, int partId) Marks partition as invalid / outdated.
boolean	isDirty(int cacheId, long pageId, long page)
long	loadedPages()
long	metaPageId(int grpId) Gets or allocates metadata page for specified grpId.
void	onCacheGroupDestroyed(int grpId) Clears internal metadata of destroyed cache group.
ByteBuffer	pageBuffer(long pageAddr)
int	pageSize()
long	partitionMetaPageId(int grpId, int partId) Gets or allocates partition metadata page for specified grpId and partId.
long	readLock(int cacheId, long pageId, long page)
long	readLockForce(int cacheId, long pageId, long page) Obtains read lock without checking page tag.
void	readUnlock(int cacheId, long pageId, long page) Releases locked page.
void	releasePage(int cacheId, long pageId, long page)
boolean	safeToUpdate() Heuristic method which allows a thread to check if it safe to start memory struture modifications in regard with checkpointing.
static int	segmentIndex(int grpId, long pageId, int segments)
void	start() Starts grid component, called on grid start.
void	stop() Stops grid component, called on grid shutdown.
int	systemPageSize()
long	tryWriteLock(int cacheId, long pageId, long page)
long	writeLock(int cacheId, long pageId, long page)
long	writeLock(int grpId, long pageId, long page, boolean restore)
void	writeUnlock(int cacheId, long pageId, long page, Boolean walPlc, boolean dirtyFlag) Releases locked page.
void	writeUnlock(int grpId, long pageId, long page, Boolean walPlc, boolean dirtyFlag, boolean restore)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

PAGE_MARKER

public static final long PAGE_MARKER

See Also:

Constant Field Values

RELATIVE_PTR_OFFSET

public static final int RELATIVE_PTR_OFFSET

Page relative pointer. Does not change once a page is allocated.

See Also:

Constant Field Values

PAGE_ID_OFFSET

public static final int PAGE_ID_OFFSET

Page ID offset

See Also:

Constant Field Values

PAGE_CACHE_ID_OFFSET

public static final int PAGE_CACHE_ID_OFFSET

Page cache ID offset.

See Also:

Constant Field Values

PAGE_PIN_CNT_OFFSET

public static final int PAGE_PIN_CNT_OFFSET

Page pin counter offset.

See Also:

Constant Field Values

PAGE_LOCK_OFFSET

public static final int PAGE_LOCK_OFFSET

Page lock offset.

See Also:

Constant Field Values

PAGE_TMP_BUF_OFFSET

public static final int PAGE_TMP_BUF_OFFSET

Page lock offset.

See Also:

Constant Field Values

PAGE_OVERHEAD

public static final int PAGE_OVERHEAD

8b Marker/timestamp 8b Relative pointer 8b Page ID 4b Cache ID 4b Pin count 8b Lock 8b Temporary buffer

See Also:

Constant Field Values

RANDOM_PAGES_EVICT_NUM

public static final int RANDOM_PAGES_EVICT_NUM

Number of random pages that will be picked for eviction.

See Also:

Constant Field Values

Constructor Detail

PageMemoryImpl

public PageMemoryImpl(DirectMemoryProvider directMemoryProvider,
                      long[] sizes,
                      GridCacheSharedContext<?,?> ctx,
                      int pageSize,
                      GridInClosure3X<FullPageId,ByteBuffer,Integer> flushDirtyPage,
                      @Nullable
                      GridInClosure3X<Long,FullPageId,PageMemoryEx> changeTracker,
                      CheckpointLockStateChecker stateChecker,
                      DataRegionMetricsImpl memMetrics,
                      boolean throttleEnabled)

Parameters:

directMemoryProvider - Memory allocator to use.

ctx - Cache shared context.

pageSize - Page size.

flushDirtyPage - Callback invoked when a dirty page is evicted.

changeTracker - Callback invoked to track changes in pages.

throttleEnabled - Write throttle enabled flag.

Method Detail

start

public void start()
           throws IgniteException

Starts grid component, called on grid start.

Specified by:

start in interface LifecycleAware

Throws:

IgniteException - If failed.

stop

public void stop()
          throws IgniteException

Stops grid component, called on grid shutdown.

Specified by:

stop in interface LifecycleAware

Throws:

IgniteException - If failed.

releasePage

public void releasePage(int cacheId,
                        long pageId,
                        long page)

Specified by:

releasePage in interface PageSupport

Parameters:

cacheId - Cache group ID.

pageId - Page ID to release.

page - Page pointer.

readLock

public long readLock(int cacheId,
                     long pageId,
                     long page)

Specified by:

readLock in interface PageSupport

Parameters:

cacheId - Cache group ID.

pageId - Page ID.

page - Page pointer.

Returns:

Pointer for reading the page.

readUnlock

public void readUnlock(int cacheId,
                       long pageId,
                       long page)

Releases locked page.

Specified by:

readUnlock in interface PageSupport

Parameters:

cacheId - Cache group ID.

pageId - Page ID.

page - Page pointer.

writeLock

public long writeLock(int cacheId,
                      long pageId,
                      long page)

Specified by:

writeLock in interface PageSupport

Parameters:

cacheId - Cache group ID.

pageId - Page ID.

page - Page pointer.

Returns:

Address of a buffer with contents of the given page or 0L if attempt to take the write lock failed.

writeLock

public long writeLock(int grpId,
                      long pageId,
                      long page,
                      boolean restore)

Specified by:

writeLock in interface PageMemoryEx

Parameters:

grpId - Group ID.

pageId - Page ID.

page - Page pointer.

restore - Determines if the page is locked for restore.

Returns:

ByteBuffer for modifying the page.

tryWriteLock

public long tryWriteLock(int cacheId,
                         long pageId,
                         long page)

Specified by:

tryWriteLock in interface PageSupport

Parameters:

cacheId - Cache group ID.

pageId - Page ID.

page - Page pointer.

Returns:

Address of a buffer with contents of the given page or 0L if attempt to take the write lock failed.

writeUnlock

public void writeUnlock(int cacheId,
                        long pageId,
                        long page,
                        Boolean walPlc,
                        boolean dirtyFlag)

Releases locked page.

Specified by:

writeUnlock in interface PageSupport

Parameters:

cacheId - Cache group ID.

pageId - Page ID.

page - Page pointer.

walPlc - True if page should be recorded to WAL, false if the page must not be recorded and null for the default behavior.

dirtyFlag - Determines whether the page was modified since the last checkpoint.

writeUnlock

public void writeUnlock(int grpId,
                        long pageId,
                        long page,
                        Boolean walPlc,
                        boolean dirtyFlag,
                        boolean restore)

Specified by:

writeUnlock in interface PageMemoryEx

Parameters:

grpId - Group ID.

pageId - Page ID.

page - Page pointer.

walPlc - True if page should be recorded to WAL, false if the page must not be recorded and null for the default behavior.

dirtyFlag - Determines whether the page was modified since the last checkpoint.

restore - Determines if the page is locked for restore.

isDirty

public boolean isDirty(int cacheId,
                       long pageId,
                       long page)

Specified by:

isDirty in interface PageSupport

Parameters:

cacheId - Cache group ID.

pageId - Page ID.

page - Page pointer.

Returns:

True if the page is dirty.

allocatePage

public long allocatePage(int cacheId,
                         int partId,
                         byte flags)
                  throws IgniteCheckedException

Allocates a page from the space for the given partition ID and the given flags.

Specified by:

allocatePage in interface PageIdAllocator

partId - Partition ID.

Returns:

Allocated page ID.

Throws:

IgniteCheckedException

pageBuffer

public ByteBuffer pageBuffer(long pageAddr)

Specified by:

pageBuffer in interface PageMemory

Parameters:

pageAddr - Page address.

Returns:

Page byte buffer.

freePage

public boolean freePage(int cacheId,
                        long pageId)
                 throws IgniteCheckedException

The given page is free now.

Specified by:

freePage in interface PageIdAllocator

Parameters:

cacheId - Cache ID.

pageId - Page ID.

Throws:

IgniteCheckedException

metaPageId

public long metaPageId(int grpId)
                throws IgniteCheckedException

Gets or allocates metadata page for specified grpId.

Specified by:

metaPageId in interface PageMemoryEx

Parameters:

grpId - Group ID.

Returns:

Meta page for grpId.

Throws:

IgniteCheckedException - If failed.

partitionMetaPageId

public long partitionMetaPageId(int grpId,
                                int partId)
                         throws IgniteCheckedException

Gets or allocates partition metadata page for specified grpId and partId.

Specified by:

partitionMetaPageId in interface PageMemoryEx

Parameters:

grpId - Group ID.

partId - Partition ID.

Returns:

Meta page for grpId and partId.

Throws:

IgniteCheckedException - If failed.

acquirePage

public long acquirePage(int cacheId,
                        long pageId)
                 throws IgniteCheckedException

Gets the page absolute pointer associated with the given page ID. Each page obtained with this method must be released by calling PageSupport.releasePage(int, long, long). This method will allocate page with given ID if it doesn't exist.

Specified by:

acquirePage in interface PageSupport

Parameters:

cacheId - Cache group ID.

pageId - Page ID.

Returns:

Page pointer.

Throws:

IgniteCheckedException - If failed.

acquirePage

public long acquirePage(int cacheId,
                        long pageId,
                        boolean restore)
                 throws IgniteCheckedException

Specified by:

acquirePage in interface PageMemoryEx

Parameters:

cacheId - Cache group ID.

pageId - Page id.

restore - Get page for restore

Returns:

Page.

Throws:

IgniteCheckedException - If failed.

See Also:

Will not read page from file if it is not present in memory

pageSize

public int pageSize()

Specified by:

pageSize in interface PageMemory

Returns:

Page size in bytes.

systemPageSize

public int systemPageSize()

Specified by:

systemPageSize in interface PageMemory

Returns:

Page size with system overhead, in bytes.

safeToUpdate

public boolean safeToUpdate()

Heuristic method which allows a thread to check if it safe to start memory struture modifications in regard with checkpointing.

Specified by:

safeToUpdate in interface PageMemoryEx

Returns:

False if there are too many dirty pages and a thread should wait for a checkpoint to begin.

beginCheckpoint

public GridMultiCollectionWrapper<FullPageId> beginCheckpoint()
                                                       throws IgniteException

Gets a collection of dirty page IDs since the last checkpoint. If a dirty page is being written after the checkpointing operation begun, the modifications will be written to a temporary buffer which will be flushed to the main memory after the checkpointing finished. This method must be called when no concurrent operations on pages are performed.

Specified by:

beginCheckpoint in interface PageMemoryEx

Returns:

Collection of dirty page IDs.

Throws:

IgniteException - If checkpoint has been already started and was not finished.

finishCheckpoint

public void finishCheckpoint()

Finishes checkpoint operation.

Specified by:

finishCheckpoint in interface PageMemoryEx

getForCheckpoint

public Integer getForCheckpoint(FullPageId fullId,
                                ByteBuffer outBuf,
                                CheckpointMetricsTracker tracker)

Gets page byte buffer for the checkpoint procedure.

Specified by:

getForCheckpoint in interface PageMemoryEx

Parameters:

fullId - Page ID to get byte buffer for. The page ID must be present in the collection returned by the PageMemoryEx.beginCheckpoint() method call.

outBuf - Temporary buffer to write changes into.

tracker - Checkpoint metrics tracker.

Returns:

Partition tag if data was read, null otherwise (data already saved to storage).

invalidate

public int invalidate(int grpId,
                      int partId)

Marks partition as invalid / outdated.

Specified by:

invalidate in interface PageMemoryEx

Parameters:

grpId - Group ID.

partId - Partition ID.

Returns:

New partition tag (growing 1-based partition file version).

onCacheGroupDestroyed

public void onCacheGroupDestroyed(int grpId)

Clears internal metadata of destroyed cache group.

Specified by:

onCacheGroupDestroyed in interface PageMemoryEx

Parameters:

grpId - Cache group ID.

clearAsync

public IgniteInternalFuture<Void> clearAsync(GridPredicate3<Integer,Long,Integer> pred,
                                             boolean cleanDirty)

Asynchronously clears pages satisfying the given predicate.

Specified by:

clearAsync in interface PageMemoryEx

Parameters:

pred - Predicate for cache group id, pageId and partition tag.

cleanDirty - Flag indicating that dirty pages collection should be cleaned.

Returns:

Future that will be completed when all pages are cleared.

loadedPages

public long loadedPages()

Specified by:

loadedPages in interface PageMemory

Returns:

Total number of loaded pages in memory.

acquiredPages

public long acquiredPages()

Returns:

Total number of acquired pages.

readLockForce

public long readLockForce(int cacheId,
                          long pageId,
                          long page)

Obtains read lock without checking page tag.

Specified by:

readLockForce in interface PageSupport

Parameters:

cacheId - Cache group ID.

pageId - Page ID.

page - Page pointer.

Returns:

Pointer for reading the page.

hasTempCopy

public boolean hasTempCopy(long absPtr)

Checks if a page has temp copy buffer.

Parameters:

absPtr - Absolute pointer.

Returns:

True if a page has temp buffer.

activePagesCount

public int activePagesCount()

Gets the number of active pages across all segments. Used for test purposes only.

Returns:

Number of active pages.

checkpointBufferPagesCount

public int checkpointBufferPagesCount()

Number of pages used in checkpoint buffer.

Specified by:

checkpointBufferPagesCount in interface PageMemory

checkpointBufferPagesSize

public int checkpointBufferPagesSize()

Number of used pages in checkpoint buffer.

segmentIndex

public static int segmentIndex(int grpId,
                               long pageId,
                               int segments)

Parameters:

pageId - Page ID.

Returns:

Segment index.