org.apache.ignite.internal.processors.cache.persistence.pagemem

## Class PageMemoryImpl

- java.lang.Object
- - org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl

All Implemented Interfaces:
  PageIdAllocator, PageMemory, PageSupport, PageMemoryEx

    public class PageMemoryImpl
    extends Object
    implements PageMemoryEx

Page header structure is described by the following diagram. When page is not allocated (in a free list):

     +--------+------------------------------------------------------+
     |8 bytes |         PAGE_SIZE + PAGE_OVERHEAD - 8 bytes          |
     +--------+------------------------------------------------------+
     |Next ptr|                      Page data                       |
     +--------+------------------------------------------------------+

When page is allocated and is in use:

     +------------------+--------+--------+----+----+--------+--------+----------------------+
     |     8 bytes      |8 bytes |8 bytes |4 b |4 b |8 bytes |8 bytes |       PAGE_SIZE      |
     +------------------+--------+--------+----+----+--------+--------+----------------------+
     | Marker/Timestamp |Rel ptr |Page ID |C ID|PIN | LOCK   |TMP BUF |       Page data      |
     +------------------+--------+--------+----+----+--------+--------+----------------------+

Note that first 8 bytes of page header are used either for page marker or for next relative pointer depending on whether the page is in use or not.

### Nested Class Summary

Nested Classes
`static class` `PageMemoryImpl.ThrottlingPolicy` Throttling enabled and its type enum.

### Field Summary

Fields
`static String` `CHECKPOINT_POOL_OVERFLOW_ERROR_MSG` Checkpoint pool overflow error message.
`static boolean` `DFLT_DELAYED_REPLACED_PAGE_WRITE`
`static boolean` `DFLT_LOADED_PAGES_BACKWARD_SHIFT_MAP`
`static int` `PAGE_LOCK_OFFSET` Page lock offset.
`static int` `PAGE_OVERHEAD` 8b Marker/timestamp 8b Relative pointer 8b Page ID 4b Cache group ID 4b Pin count 8b Lock 8b Temporary buffer
`static long` `RELATIVE_PTR_MASK` Full relative pointer mask.
`static int` `TRY_AGAIN_TAG` Try again tag.

### Fields inherited from interface org.apache.ignite.internal.pagemem.PageIdAllocator

`FLAG_AUX, FLAG_DATA, FLAG_IDX, INDEX_PARTITION, MAX_PARTITION_ID, META_PAGE_ID`

### Constructor Summary

Constructors
Constructor and Description

`PageMemoryImpl(DirectMemoryProvider directMemoryProvider, long[] sizes, GridCacheSharedContext<?,?> ctx, PageReadWriteManager pmPageMgr, int pageSize, PageStoreWriter flushDirtyPage, boolean trackable, CheckpointLockStateChecker stateChecker, DataRegionMetricsImpl dataRegionMetrics, DataRegionConfiguration dataRegionCfg, @Nullable PageMemoryImpl.ThrottlingPolicy throttlingPlc, IgniteOutClosure<CheckpointProgress> cpProgressProvider)`

### Method Summary

All Methods Static Methods Instance Methods Concrete Methods
`long` `acquiredPages()`
`long` `acquirePage(int grpId, long pageId)` Gets the page absolute pointer associated with the given page ID.
`long` `acquirePage(int grpId, long pageId, AtomicBoolean pageAllocated)`
`long` `acquirePage(int grpId, long pageId, IoStatisticsHolder statHolder)` Gets the page absolute pointer associated with the given page ID.
`long` `acquirePage(int grpId, long pageId, IoStatisticsHolder statHolder, boolean restore)`
`int` `activePagesCount()` Gets the number of active pages across all segments.
`long` `allocatePage(int grpId, int partId, byte flags)` Allocates a page from the space for the given partition ID and the given flags.
`GridMultiCollectionWrapper<FullPageId>` `beginCheckpoint(IgniteInternalFuture allowToReplace)` Gets a collection of dirty page IDs since the last checkpoint.
`int` `checkpointBufferPagesCount()` Number of pages used in checkpoint buffer.
`int` `checkpointBufferPagesSize()` Number of used pages in checkpoint buffer.
`void` `checkpointWritePage(FullPageId fullId, ByteBuffer buf, PageStoreWriter pageStoreWriter, CheckpointMetricsTracker metricsTracker)` Prepare page for write during checkpoint.
`IgniteInternalFuture<Void>` `clearAsync(LoadedPagesMap.KeyPredicate pred, boolean cleanDirty)` Asynchronously clears pages satisfying the given predicate.
`Collection<FullPageId>` `dirtyPages()` Gets a collection of all pages currently marked as dirty.
`void` `finishCheckpoint()` Finishes checkpoint operation.
`boolean` `freePage(int grpId, long pageId)` The given page is free now.
`boolean` `hasLoadedPage(FullPageId fullPageId)`
`boolean` `hasTempCopy(long absPtr)` Checks if a page has temp copy buffer.
`int` `invalidate(int grpId, int partId)` Marks partition as invalid / outdated.
`boolean` `isCpBufferOverflowThresholdExceeded()` Checks if the Checkpoint Buffer is currently close to exhaustion.
`boolean` `isDirty(int grpId, long pageId, long page)`
`long` `loadedPages()`
`DataRegionMetricsImpl` `metrics()` Metrics of the data region this memory is associated with.
`void` `onCacheGroupDestroyed(int grpId)` Clears internal metadata of destroyed cache group.
`ByteBuffer` `pageBuffer(long pageAddr)`
`PageReadWriteManager` `pageManager()`
`int` `pageSize()`
`long` `partitionMetaPageId(int grpId, int partId)` Gets partition metadata page for specified grpId and partId.
`FullPageId` `pullPageFromCpBuffer()` Get arbitrary page from cp buffer.
`long` `readLock(int grpId, long pageId, long page)`
`long` `readLock(long absPtr, long pageId, boolean force, boolean touch)`
`long` `readLockForce(int grpId, long pageId, long page)` Obtains read lock without checking page tag.
`void` `readUnlock(int grpId, long pageId, long page)` Releases locked page.
`int` `realPageSize(int grpId)`
`void` `releasePage(int grpId, long pageId, long page)`
`boolean` `safeToUpdate()` Heuristic method which allows a thread to check if it safe to start memory struture modifications in regard with checkpointing.
`static int` `segmentIndex(int grpId, long pageId, int segments)`
`void` `start()` Start page memory.
`void` `stop(boolean deallocate)` Stop page memory.
`int` `systemPageSize()`
`long` `totalPages()` Total pages can be placed to memory.
`long` `tryWriteLock(int grpId, long pageId, long page)`
`long` `writeLock(int grpId, long pageId, long page)`
`long` `writeLock(int grpId, long pageId, long page, boolean restore)`
`void` `writeUnlock(int grpId, long pageId, long page, Boolean walPlc, boolean dirtyFlag)` Releases locked page.
`void` `writeUnlock(int grpId, long pageId, long page, Boolean walPlc, boolean dirtyFlag, boolean restore)`

### Methods inherited from class java.lang.Object

`clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

### Field Detail

#### RELATIVE_PTR_MASK

    public static final long RELATIVE_PTR_MASK

Full relative pointer mask.

See Also:
  Constant Field Values

#### PAGE_LOCK_OFFSET

    public static final int PAGE_LOCK_OFFSET

Page lock offset.

See Also:
  Constant Field Values

#### PAGE_OVERHEAD

    public static final int PAGE_OVERHEAD

8b Marker/timestamp 8b Relative pointer 8b Page ID 4b Cache group ID 4b Pin count 8b Lock 8b Temporary buffer

See Also:
  Constant Field Values

#### TRY_AGAIN_TAG

    public static final int TRY_AGAIN_TAG

Try again tag.

See Also:
  Constant Field Values

#### DFLT_DELAYED_REPLACED_PAGE_WRITE

    public static final boolean DFLT_DELAYED_REPLACED_PAGE_WRITE

See Also:
  `IgniteSystemProperties.IGNITE_DELAYED_REPLACED_PAGE_WRITE`, Constant Field Values

#### DFLT_LOADED_PAGES_BACKWARD_SHIFT_MAP

    public static final boolean DFLT_LOADED_PAGES_BACKWARD_SHIFT_MAP

See Also:
  `IgniteSystemProperties.IGNITE_LOADED_PAGES_BACKWARD_SHIFT_MAP`, Constant Field Values

#### CHECKPOINT_POOL_OVERFLOW_ERROR_MSG

    public static final String CHECKPOINT_POOL_OVERFLOW_ERROR_MSG

Checkpoint pool overflow error message.

See Also:
  Constant Field Values

### Constructor Detail

#### PageMemoryImpl

    public PageMemoryImpl(DirectMemoryProvider directMemoryProvider,
                          long[] sizes,
                          GridCacheSharedContext<?,?> ctx,
                          PageReadWriteManager pmPageMgr,
                          int pageSize,
                          PageStoreWriter flushDirtyPage,
                          boolean trackable,
                          CheckpointLockStateChecker stateChecker,
                          DataRegionMetricsImpl dataRegionMetrics,
                          DataRegionConfiguration dataRegionCfg,
                          @Nullable
                          @Nullable PageMemoryImpl.ThrottlingPolicy throttlingPlc,
                          IgniteOutClosure<CheckpointProgress> cpProgressProvider)

Parameters:
  `directMemoryProvider` - Memory allocator to use.
  `sizes` - segments sizes, last is checkpoint pool size.
  `ctx` - Cache shared context.
  `pmPageMgr` - Page store manager.
  `pageSize` - Page size.
  `flushDirtyPage` - write callback invoked when a dirty page is removed for replacement.
  `trackable` - Obsolete flag. Keeped for compatibility with existing PDS.
  `stateChecker` - Checkpoint lock state provider. Used to ensure lock is held by thread, which modify pages.
  `dataRegionMetrics` - Memory metrics to track dirty pages count and page replace rate.
  `dataRegionCfg` - Data region configuration.
  `throttlingPlc` - Write throttle enabled and its type. Null equal to none.
  `cpProgressProvider` - checkpoint progress, base for throttling. Null disables throttling.

### Method Detail

#### start

    public void start()
               throws IgniteException

Start page memory.

Specified by:
  `start` in interface `PageMemory`
Throws:
  `IgniteException`

#### stop

    public void stop(boolean deallocate)
              throws IgniteException

Stop page memory.

Specified by:
  `stop` in interface `PageMemory`
Parameters:
  `deallocate` - `True` to deallocate memory, `false` to allow memory reuse on subsequent `PageMemory.start()`
Throws:
  `IgniteException`

#### releasePage

    public void releasePage(int grpId,
                            long pageId,
                            long page)

Specified by:
  `releasePage` in interface `PageSupport`
Parameters:
  `grpId` - Cache group ID.
  `pageId` - Page ID to release.
  `page` - Page pointer.

#### readLock

    public long readLock(int grpId,
                         long pageId,
                         long page)

Specified by:
  `readLock` in interface `PageSupport`
Parameters:
  `grpId` - Cache group ID.
  `pageId` - Page ID.
  `page` - Page pointer.
Returns:
  Pointer for reading the page.

#### readUnlock

    public void readUnlock(int grpId,
                           long pageId,
                           long page)

Releases locked page.

Specified by:
  `readUnlock` in interface `PageSupport`
Parameters:
  `grpId` - Cache group ID.
  `pageId` - Page ID.
  `page` - Page pointer.

#### writeLock

    public long writeLock(int grpId,
                          long pageId,
                          long page)

Specified by:
  `writeLock` in interface `PageSupport`
Parameters:
  `grpId` - Cache group ID.
  `pageId` - Page ID.
  `page` - Page pointer.
Returns:
  Address of a buffer with contents of the given page or `0L` if attempt to take the write lock failed.

#### writeLock

    public long writeLock(int grpId,
                          long pageId,
                          long page,
                          boolean restore)

Specified by:
  `writeLock` in interface `PageMemoryEx`
Parameters:
  `grpId` - Group ID.
  `pageId` - Page ID.
  `page` - Page pointer.
  `restore` - Determines if the page is locked for restore memory (crash recovery).
Returns:
  ByteBuffer for modifying the page.

#### tryWriteLock

    public long tryWriteLock(int grpId,
                             long pageId,
                             long page)

Specified by:
  `tryWriteLock` in interface `PageSupport`
Parameters:
  `grpId` - Cache group ID.
  `pageId` - Page ID.
  `page` - Page pointer.
Returns:
  Address of a buffer with contents of the given page or `0L` if attempt to take the write lock failed.

#### writeUnlock

    public void writeUnlock(int grpId,
                            long pageId,
                            long page,
                            Boolean walPlc,
                            boolean dirtyFlag)

Releases locked page.

Specified by:
  `writeUnlock` in interface `PageSupport`
Parameters:
  `grpId` - Cache group ID.
  `pageId` - Page ID.
  `page` - Page pointer.
  `walPlc` - `True` if page should be recorded to WAL, `false` if the page must not be recorded and `null` for the default behavior.
  `dirtyFlag` - Determines whether the page was modified since the last checkpoint.

#### writeUnlock

    public void writeUnlock(int grpId,
                            long pageId,
                            long page,
                            Boolean walPlc,
                            boolean dirtyFlag,
                            boolean restore)

Specified by:
  `writeUnlock` in interface `PageMemoryEx`
Parameters:
  `grpId` - Group ID.
  `pageId` - Page ID.
  `page` - Page pointer.
  `walPlc` - `True` if page should be recorded to WAL, `false` if the page must not be recorded and `null` for the default behavior.
  `dirtyFlag` - Determines whether the page was modified since the last checkpoint.
  `restore` - Determines if the page is locked for restore.

#### isDirty

    public boolean isDirty(int grpId,
                           long pageId,
                           long page)

Specified by:
  `isDirty` in interface `PageSupport`
Parameters:
  `grpId` - Cache group ID.
  `pageId` - Page ID.
  `page` - Page pointer.
Returns:
  `True` if the page is dirty.

#### allocatePage

    public long allocatePage(int grpId,
                             int partId,
                             byte flags)
                      throws IgniteCheckedException

Allocates a page from the space for the given partition ID and the given flags.

Specified by:
  `allocatePage` in interface `PageIdAllocator`
Parameters:
  `grpId` - Cache Group ID.
  `partId` - Partition ID.
Returns:
  Allocated page ID.
Throws:
  `IgniteCheckedException`

#### pageBuffer

    public ByteBuffer pageBuffer(long pageAddr)

Specified by:
  `pageBuffer` in interface `PageMemory`
Parameters:
  `pageAddr` - Page address.
Returns:
  Page byte buffer.

#### freePage

    public boolean freePage(int grpId,
                            long pageId)

The given page is free now.

Specified by:
  `freePage` in interface `PageIdAllocator`
Parameters:
  `grpId` - Cache Group ID.
  `pageId` - Page ID.

#### partitionMetaPageId

    public long partitionMetaPageId(int grpId,
                                    int partId)

Gets partition metadata page for specified grpId and partId.

Specified by:
  `partitionMetaPageId` in interface `PageMemoryEx`
Parameters:
  `grpId` - Group ID.
  `partId` - Partition ID.
Returns:
  Meta page for grpId and partId.

#### acquirePage

    public long acquirePage(int grpId,
                            long pageId)
                     throws IgniteCheckedException

Gets the page absolute pointer associated with the given page ID. Each page obtained with this method must be released by calling `PageSupport.releasePage(int, long, long)`. This method will allocate page with given ID if it doesn't exist.

Specified by:
  `acquirePage` in interface `PageSupport`
Parameters:
  `grpId` - Cache group ID.
  `pageId` - Page ID.
Returns:
  Page pointer.
Throws:
  `IgniteCheckedException` - If failed.

#### acquirePage

    public long acquirePage(int grpId,
                            long pageId,
                            IoStatisticsHolder statHolder)
                     throws IgniteCheckedException

Gets the page absolute pointer associated with the given page ID. Each page obtained with this method must be released by calling `PageSupport.releasePage(int, long, long)`. This method will allocate page with given ID if it doesn't exist.

Specified by:
  `acquirePage` in interface `PageSupport`
Parameters:
  `grpId` - Cache group ID.
  `pageId` - Page ID.
  `statHolder` - Statistics holder to track IO operations.
Returns:
  Page pointer.
Throws:
  `IgniteCheckedException` - If failed.

#### acquirePage

    public long acquirePage(int grpId,
                            long pageId,
                            AtomicBoolean pageAllocated)
                     throws IgniteCheckedException

Specified by:
  `acquirePage` in interface `PageMemoryEx`
Parameters:
  `grpId` - Cache group ID.
  `pageId` - Page ID.
  `pageAllocated` - Flag is set if new page was allocated in offheap memory.
Returns:
  Page.
Throws:
  `IgniteCheckedException`
See Also:
  `Sets additional flag indicating that page was not found in memory and had to be allocated.`

#### acquirePage

    public long acquirePage(int grpId,
                            long pageId,
                            IoStatisticsHolder statHolder,
                            boolean restore)
                     throws IgniteCheckedException

Specified by:
  `acquirePage` in interface `PageMemoryEx`
Parameters:
  `grpId` - Cache group ID.
  `pageId` - Page id.
  `restore` - Get page for restore
Returns:
  Page.
Throws:
  `IgniteCheckedException` - If failed.
  `StorageException` - If page reading failed from storage.
See Also:
  `Will read page from file if it is not present in memory`

#### pageSize

    public int pageSize()

Specified by:
  `pageSize` in interface `PageMemory`
Returns:
  Page size in bytes.

#### systemPageSize

    public int systemPageSize()

Specified by:
  `systemPageSize` in interface `PageMemory`
Returns:
  Page size with system overhead, in bytes.

#### realPageSize

    public int realPageSize(int grpId)

Specified by:
  `realPageSize` in interface `PageMemory`
Parameters:
  `grpId` - Group id.
Returns:
  Page size without encryption overhead.

#### safeToUpdate

    public boolean safeToUpdate()

Heuristic method which allows a thread to check if it safe to start memory struture modifications in regard with checkpointing. May return false-negative result during or after partition eviction.

Specified by:
  `safeToUpdate` in interface `PageMemoryEx`
Returns:
  `False` if there are too many dirty pages and a thread should wait for a checkpoint to begin.

#### totalPages

    public long totalPages()

Description copied from interface: `PageMemoryEx`
Total pages can be placed to memory.

Specified by:
  `totalPages` in interface `PageMemoryEx`
Returns:
  Total pages can be placed in all segments.

#### beginCheckpoint

    public GridMultiCollectionWrapper<FullPageId> beginCheckpoint(IgniteInternalFuture allowToReplace)
                                                           throws IgniteException

Gets a collection of dirty page IDs since the last checkpoint. If a dirty page is being written after the checkpointing operation begun, the modifications will be written to a temporary buffer which will be flushed to the main memory after the checkpointing finished. This method must be called when no concurrent operations on pages are performed.

Specified by:
  `beginCheckpoint` in interface `PageMemoryEx`
Parameters:
  `allowToReplace` - The sign which allows to replace pages from a checkpoint by page replacer.
Returns:
  Collection of dirty page IDs.
Throws:
  `IgniteException` - If checkpoint has been already started and was not finished.

#### finishCheckpoint

    public void finishCheckpoint()

Finishes checkpoint operation.

Specified by:
  `finishCheckpoint` in interface `PageMemoryEx`

#### pageManager

    public PageReadWriteManager pageManager()

Specified by:
  `pageManager` in interface `PageMemoryEx`

#### checkpointWritePage

    public void checkpointWritePage(FullPageId fullId,
                                    ByteBuffer buf,
                                    PageStoreWriter pageStoreWriter,
                                    CheckpointMetricsTracker metricsTracker)
                             throws IgniteCheckedException

Prepare page for write during checkpoint. `PageStoreWriter` will be called when the page will be ready to write.

Specified by:
  `checkpointWritePage` in interface `PageMemoryEx`
Parameters:
  `fullId` - Page ID to get byte buffer for. The page ID must be present in the collection returned by the `PageMemoryEx.beginCheckpoint(IgniteInternalFuture)` method call.
  `buf` - Temporary buffer to write changes into.
  `pageStoreWriter` - Checkpoint page write context.
  `metricsTracker` - Checkpoint metrics tracker.
Throws:
  `IgniteCheckedException` - If failed to obtain page data.

#### invalidate

    public int invalidate(int grpId,
                          int partId)

Marks partition as invalid / outdated.

Specified by:
  `invalidate` in interface `PageMemoryEx`
Parameters:
  `grpId` - Group ID.
  `partId` - Partition ID.
Returns:
  New partition generation (growing 1-based partition file version).

#### onCacheGroupDestroyed

    public void onCacheGroupDestroyed(int grpId)

Clears internal metadata of destroyed cache group.

Specified by:
  `onCacheGroupDestroyed` in interface `PageMemoryEx`
Parameters:
  `grpId` - Cache group ID.

#### clearAsync

    public IgniteInternalFuture<Void> clearAsync(LoadedPagesMap.KeyPredicate pred,
                                                 boolean cleanDirty)

Asynchronously clears pages satisfying the given predicate.

Specified by:
  `clearAsync` in interface `PageMemoryEx`
Parameters:
  `pred` - Predicate for cache group id, pageId.
  `cleanDirty` - Flag indicating that dirty pages collection should be cleaned.
Returns:
  Future that will be completed when all pages are cleared.

#### loadedPages

    public long loadedPages()

Specified by:
  `loadedPages` in interface `PageMemory`
Returns:
  Total number of loaded pages in memory.

#### acquiredPages

    public long acquiredPages()

Returns:
  Total number of acquired pages.

#### hasLoadedPage

    public boolean hasLoadedPage(FullPageId fullPageId)

Parameters:
  `fullPageId` - Full page ID to check.
Returns:
  `true` if the page is contained in the loaded pages table, `false` otherwise.

#### readLock

    public long readLock(long absPtr,
                         long pageId,
                         boolean force,
                         boolean touch)

Specified by:
  `readLock` in interface `PageMemoryEx`
Parameters:
  `absPtr` - Absolute pointer to read lock.
  `pageId` - Page ID.
  `force` - Force flag.
  `touch` - Update page timestamp.
Returns:
  Pointer to the page read buffer.

#### readLockForce

    public long readLockForce(int grpId,
                              long pageId,
                              long page)

Obtains read lock without checking page tag.

Specified by:
  `readLockForce` in interface `PageSupport`
Parameters:
  `grpId` - Cache group ID.
  `pageId` - Page ID.
  `page` - Page pointer.
Returns:
  Pointer for reading the page.

#### hasTempCopy

    public boolean hasTempCopy(long absPtr)

Checks if a page has temp copy buffer.

Parameters:
  `absPtr` - Absolute pointer.
Returns:
  `True` if a page has temp buffer.

#### activePagesCount

    public int activePagesCount()

Gets the number of active pages across all segments. Used for test purposes only.

Returns:
  Number of active pages.

#### checkpointBufferPagesCount

    public int checkpointBufferPagesCount()

Number of pages used in checkpoint buffer.

Specified by:
  `checkpointBufferPagesCount` in interface `PageMemory`

#### checkpointBufferPagesSize

    public int checkpointBufferPagesSize()

Number of used pages in checkpoint buffer.

#### segmentIndex

    public static int segmentIndex(int grpId,
                                   long pageId,
                                   int segments)

Parameters:
  `pageId` - Page ID.
Returns:
  Segment index.

#### isCpBufferOverflowThresholdExceeded

    public boolean isCpBufferOverflowThresholdExceeded()

Checks if the Checkpoint Buffer is currently close to exhaustion.

Specified by:
  `isCpBufferOverflowThresholdExceeded` in interface `PageMemoryEx`
Returns:
  `true` if measures like throttling to protect Checkpoint Buffer should be applied, and `false` otherwise.

#### metrics

    public DataRegionMetricsImpl metrics()

Metrics of the data region this memory is associated with.

Specified by:
  `metrics` in interface `PageMemory`

#### pullPageFromCpBuffer

    public FullPageId pullPageFromCpBuffer()

Get arbitrary page from cp buffer.

Specified by:
  `pullPageFromCpBuffer` in interface `PageMemoryEx`

#### dirtyPages

    @TestOnly
    public Collection<FullPageId> dirtyPages()

Gets a collection of all pages currently marked as dirty. Will create a collection copy.

Returns:
  Collection of all page IDs marked as dirty.

${current.year} Copyright © Apache Software Foundation

Follow @ApacheIgnite

Ignite Database and Caching Platform | : ver. 2.16.0
Release Date | : December 15 2023