Package org.apache.ignite.internal.processors.cache.persistence.pagemem

## Class PageMemoryImpl

- java.lang.Object
- - org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl

All Implemented Interfaces:
  `PageIdAllocator`, `PageMemory`, `PageSupport`, `PageMemoryEx`

    public class PageMemoryImpl
    extends Object
    implements PageMemoryEx

Page header structure is described by the following diagram. When page is not allocated (in a free list):

     +--------+------------------------------------------------------+
     |8 bytes |         PAGE_SIZE + PAGE_OVERHEAD - 8 bytes          |
     +--------+------------------------------------------------------+
     |Next ptr|                      Page data                       |
     +--------+------------------------------------------------------+

When page is allocated and is in use:

     +------------------+--------+--------+----+----+--------+--------+----------------------+
     |     8 bytes      |8 bytes |8 bytes |4 b |4 b |8 bytes |8 bytes |       PAGE_SIZE      |
     +------------------+--------+--------+----+----+--------+--------+----------------------+
     | Marker/Timestamp |Rel ptr |Page ID |C ID|PIN | LOCK   |TMP BUF |       Page data      |
     +------------------+--------+--------+----+----+--------+--------+----------------------+

Note that first 8 bytes of page header are used either for page marker or for next relative pointer depending on whether the page is in use or not.

### Nested Class Summary

Nested Classes
`static class` `PageMemoryImpl.ThrottlingPolicy` Throttling enabled and its type enum.

### Field Summary

Fields
`static String` `CHECKPOINT_POOL_OVERFLOW_ERROR_MSG` Checkpoint pool overflow error message.
`static boolean` `DFLT_DELAYED_REPLACED_PAGE_WRITE`
`static boolean` `DFLT_LOADED_PAGES_BACKWARD_SHIFT_MAP`
`static int` `PAGE_LOCK_OFFSET` Page lock offset.
`static int` `PAGE_OVERHEAD` 8b Marker/timestamp 8b Relative pointer 8b Page ID 4b Cache group ID 4b Pin count 8b Lock 8b Temporary buffer
`static long` `RELATIVE_PTR_MASK` Full relative pointer mask.
`static int` `TRY_AGAIN_TAG` Try again tag.

### Fields inherited from interface org.apache.ignite.internal.pagemem.PageIdAllocator

`FLAG_AUX, FLAG_DATA, FLAG_IDX, INDEX_PARTITION, MAX_PARTITION_ID, META_PAGE_ID`

### Constructor Summary

Constructors

`PageMemoryImpl​(DirectMemoryProvider directMemoryProvider, long[] sizes, GridCacheSharedContext<?,​?> ctx, PageReadWriteManager pmPageMgr, int pageSize, PageStoreWriter flushDirtyPage, boolean trackable, CheckpointLockStateChecker stateChecker, DataRegionMetricsImpl dataRegionMetrics, DataRegionConfiguration dataRegionCfg, @Nullable PageMemoryImpl.ThrottlingPolicy throttlingPlc, IgniteOutClosure<CheckpointProgress> cpProgressProvider)`

### Method Summary

All Methods Static Methods Instance Methods Concrete Methods
`long` `acquiredPages()`
`long` `acquirePage​(int grpId, long pageId)` Gets the page absolute pointer associated with the given page ID.
`long` `acquirePage​(int grpId, long pageId, AtomicBoolean pageAllocated)`
`long` `acquirePage​(int grpId, long pageId, IoStatisticsHolder statHolder)` Gets the page absolute pointer associated with the given page ID.
`long` `acquirePage​(int grpId, long pageId, IoStatisticsHolder statHolder, boolean restore)`
`int` `activePagesCount()` Gets the number of active pages across all segments.
`long` `allocatePage​(int grpId, int partId, byte flags)` Allocates a page from the space for the given partition ID and the given flags.
`GridMultiCollectionWrapper<FullPageId>` `beginCheckpoint​(IgniteInternalFuture allowToReplace)` Gets a collection of dirty page IDs since the last checkpoint.
`int` `checkpointBufferPagesCount()` Number of pages used in checkpoint buffer.
`int` `checkpointBufferPagesSize()` Number of used pages in checkpoint buffer.
`void` `checkpointWritePage​(FullPageId fullId, ByteBuffer buf, PageStoreWriter pageStoreWriter, CheckpointMetricsTracker metricsTracker)` Prepare page for write during checkpoint.
`IgniteInternalFuture<Void>` `clearAsync​(LoadedPagesMap.KeyPredicate pred, boolean cleanDirty)` Asynchronously clears pages satisfying the given predicate.
`Collection<FullPageId>` `dirtyPages()` Gets a collection of all pages currently marked as dirty.
`void` `finishCheckpoint()` Finishes checkpoint operation.
`boolean` `freePage​(int grpId, long pageId)` The given page is free now.
`boolean` `hasLoadedPage​(FullPageId fullPageId)`
`boolean` `hasTempCopy​(long absPtr)` Checks if a page has temp copy buffer.
`int` `invalidate​(int grpId, int partId)` Marks partition as invalid / outdated.
`boolean` `isCpBufferOverflowThresholdExceeded()` Checks if the Checkpoint Buffer is currently close to exhaustion.
`boolean` `isDirty​(int grpId, long pageId, long page)`
`long` `loadedPages()`
`DataRegionMetricsImpl` `metrics()` Metrics of the data region this memory is associated with.
`void` `onCacheGroupDestroyed​(int grpId)` Clears internal metadata of destroyed cache group.
`ByteBuffer` `pageBuffer​(long pageAddr)`
`PageReadWriteManager` `pageManager()`
`int` `pageSize()`
`long` `partitionMetaPageId​(int grpId, int partId)` Gets partition metadata page for specified grpId and partId.
`FullPageId` `pullPageFromCpBuffer()` Get arbitrary page from cp buffer.
`long` `readLock​(int grpId, long pageId, long page)`
`long` `readLock​(long absPtr, long pageId, boolean force, boolean touch)`
`long` `readLockForce​(int grpId, long pageId, long page)` Obtains read lock without checking page tag.
`void` `readUnlock​(int grpId, long pageId, long page)` Releases locked page.
`int` `realPageSize​(int grpId)`
`void` `releasePage​(int grpId, long pageId, long page)`
`boolean` `safeToUpdate()` Heuristic method which allows a thread to check if it safe to start memory struture modifications in regard with checkpointing.
`static int` `segmentIndex​(int grpId, long pageId, int segments)`
`void` `start()` Start page memory.
`void` `stop​(boolean deallocate)` Stop page memory.
`int` `systemPageSize()`
`long` `totalPages()` Total pages can be placed to memory.
`long` `tryWriteLock​(int grpId, long pageId, long page)`
`long` `writeLock​(int grpId, long pageId, long page)`
`long` `writeLock​(int grpId, long pageId, long page, boolean restore)`
`void` `writeUnlock​(int grpId, long pageId, long page, Boolean walPlc, boolean dirtyFlag)` Releases locked page.
`void` `writeUnlock​(int grpId, long pageId, long page, Boolean walPlc, boolean dirtyFlag, boolean restore)`

### Methods inherited from class java.lang.Object

`clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

### Field Detail

#### RELATIVE_PTR_MASK

    public static final long RELATIVE_PTR_MASK

Full relative pointer mask.

See Also:
  Constant Field Values

#### PAGE_LOCK_OFFSET

    public static final int PAGE_LOCK_OFFSET

Page lock offset.

See Also:
  Constant Field Values

#### PAGE_OVERHEAD

    public static final int PAGE_OVERHEAD

8b Marker/timestamp 8b Relative pointer 8b Page ID 4b Cache group ID 4b Pin count 8b Lock 8b Temporary buffer

See Also:
  Constant Field Values

#### TRY_AGAIN_TAG

    public static final int TRY_AGAIN_TAG

Try again tag.

See Also:
  Constant Field Values

#### DFLT_DELAYED_REPLACED_PAGE_WRITE

    public static final boolean DFLT_DELAYED_REPLACED_PAGE_WRITE

See Also:
  `IgniteSystemProperties.IGNITE_DELAYED_REPLACED_PAGE_WRITE`, Constant Field Values

#### DFLT_LOADED_PAGES_BACKWARD_SHIFT_MAP

    public static final boolean DFLT_LOADED_PAGES_BACKWARD_SHIFT_MAP

See Also:
  `IgniteSystemProperties.IGNITE_LOADED_PAGES_BACKWARD_SHIFT_MAP`, Constant Field Values

#### CHECKPOINT_POOL_OVERFLOW_ERROR_MSG

    public static final String CHECKPOINT_POOL_OVERFLOW_ERROR_MSG

Checkpoint pool overflow error message.

See Also:
  Constant Field Values

### Constructor Detail

#### PageMemoryImpl

    public PageMemoryImpl​(DirectMemoryProvider directMemoryProvider,
                          long[] sizes,
                          GridCacheSharedContext<?,​?> ctx,
                          PageReadWriteManager pmPageMgr,
                          int pageSize,
                          PageStoreWriter flushDirtyPage,
                          boolean trackable,
                          CheckpointLockStateChecker stateChecker,
                          DataRegionMetricsImpl dataRegionMetrics,
                          DataRegionConfiguration dataRegionCfg,
                          @Nullable
                          @Nullable PageMemoryImpl.ThrottlingPolicy throttlingPlc,
                          IgniteOutClosure<CheckpointProgress> cpProgressProvider)

Parameters:
  `directMemoryProvider` - Memory allocator to use.
  `sizes` - segments sizes, last is checkpoint pool size.
  `ctx` - Cache shared context.
  `pmPageMgr` - Page store manager.
  `pageSize` - Page size.
  `flushDirtyPage` - write callback invoked when a dirty page is removed for replacement.
  `trackable` - Obsolete flag. Keeped for compatibility with existing PDS.
  `stateChecker` - Checkpoint lock state provider. Used to ensure lock is held by thread, which modify pages.
  `dataRegionMetrics` - Memory metrics to track dirty pages count and page replace rate.
  `dataRegionCfg` - Data region configuration.
  `throttlingPlc` - Write throttle enabled and its type. Null equal to none.
  `cpProgressProvider` - checkpoint progress, base for throttling. Null disables throttling.

### Method Detail

#### start

    public void start()
               throws IgniteException

Start page memory.

Specified by:
  `start` in interface `PageMemory`
Throws:
  `IgniteException`

#### stop

    public void stop​(boolean deallocate)
              throws IgniteException

Stop page memory.

Specified by:
  `stop` in interface `PageMemory`
Parameters:
  `deallocate` - `True` to deallocate memory, `false` to allow memory reuse on subsequent `PageMemory.start()`
Throws:
  `IgniteException`

#### releasePage

    public void releasePage​(int grpId,
                            long pageId,
                            long page)

Specified by:
  `releasePage` in interface `PageSupport`
Parameters:
  `grpId` - Cache group ID.
  `pageId` - Page ID to release.
  `page` - Page pointer.

#### readLock

    public long readLock​(int grpId,
                         long pageId,
                         long page)

Specified by:
  `readLock` in interface `PageSupport`
Parameters:
  `grpId` - Cache group ID.
  `pageId` - Page ID.
  `page` - Page pointer.
Returns:
  Pointer for reading the page.

#### readUnlock

    public void readUnlock​(int grpId,
                           long pageId,
                           long page)

Releases locked page.

Specified by:
  `readUnlock` in interface `PageSupport`
Parameters:
  `grpId` - Cache group ID.
  `pageId` - Page ID.
  `page` - Page pointer.

#### writeLock

    public long writeLock​(int grpId,
                          long pageId,
                          long page)

Specified by:
  `writeLock` in interface `PageSupport`
Parameters:
  `grpId` - Cache group ID.
  `pageId` - Page ID.
  `page` - Page pointer.
Returns:
  Address of a buffer with contents of the given page or `0L` if attempt to take the write lock failed.

#### writeLock

    public long writeLock​(int grpId,
                          long pageId,
                          long page,
                          boolean restore)

Specified by:
  `writeLock` in interface `PageMemoryEx`
Parameters:
  `grpId` - Group ID.
  `pageId` - Page ID.
  `page` - Page pointer.
  `restore` - Determines if the page is locked for restore memory (crash recovery).
Returns:
  ByteBuffer for modifying the page.

#### tryWriteLock

    public long tryWriteLock​(int grpId,
                             long pageId,
                             long page)

Specified by:
  `tryWriteLock` in interface `PageSupport`
Parameters:
  `grpId` - Cache group ID.
  `pageId` - Page ID.
  `page` - Page pointer.
Returns:
  Address of a buffer with contents of the given page or `0L` if attempt to take the write lock failed.

#### writeUnlock

    public void writeUnlock​(int grpId,
                            long pageId,
                            long page,
                            Boolean walPlc,
                            boolean dirtyFlag)

Releases locked page.

Specified by:
  `writeUnlock` in interface `PageSupport`
Parameters:
  `grpId` - Cache group ID.
  `pageId` - Page ID.
  `page` - Page pointer.
  `walPlc` - `True` if page should be recorded to WAL, `false` if the page must not be recorded and `null` for the default behavior.
  `dirtyFlag` - Determines whether the page was modified since the last checkpoint.

#### writeUnlock

    public void writeUnlock​(int grpId,
                            long pageId,
                            long page,
                            Boolean walPlc,
                            boolean dirtyFlag,
                            boolean restore)

Specified by:
  `writeUnlock` in interface `PageMemoryEx`
Parameters:
  `grpId` - Group ID.
  `pageId` - Page ID.
  `page` - Page pointer.
  `walPlc` - `True` if page should be recorded to WAL, `false` if the page must not be recorded and `null` for the default behavior.
  `dirtyFlag` - Determines whether the page was modified since the last checkpoint.
  `restore` - Determines if the page is locked for restore.

#### isDirty

    public boolean isDirty​(int grpId,
                           long pageId,
                           long page)

Specified by:
  `isDirty` in interface `PageSupport`
Parameters:
  `grpId` - Cache group ID.
  `pageId` - Page ID.
  `page` - Page pointer.
Returns:
  `True` if the page is dirty.

#### allocatePage

    public long allocatePage​(int grpId,
                             int partId,
                             byte flags)
                      throws IgniteCheckedException

Allocates a page from the space for the given partition ID and the given flags.

Specified by:
  `allocatePage` in interface `PageIdAllocator`
Parameters:
  `grpId` - Cache Group ID.
  `partId` - Partition ID.
Returns:
  Allocated page ID.
Throws:
  `IgniteCheckedException`

#### pageBuffer

    public ByteBuffer pageBuffer​(long pageAddr)

Specified by:
  `pageBuffer` in interface `PageMemory`
Parameters:
  `pageAddr` - Page address.
Returns:
  Page byte buffer.

#### freePage

    public boolean freePage​(int grpId,
                            long pageId)

The given page is free now.

Specified by:
  `freePage` in interface `PageIdAllocator`
Parameters:
  `grpId` - Cache Group ID.
  `pageId` - Page ID.

#### partitionMetaPageId

    public long partitionMetaPageId​(int grpId,
                                    int partId)

Gets partition metadata page for specified grpId and partId.

Specified by:
  `partitionMetaPageId` in interface `PageMemoryEx`
Parameters:
  `grpId` - Group ID.
  `partId` - Partition ID.
Returns:
  Meta page for grpId and partId.

#### acquirePage

    public long acquirePage​(int grpId,
                            long pageId)
                     throws IgniteCheckedException

Gets the page absolute pointer associated with the given page ID. Each page obtained with this method must be released by calling `PageSupport.releasePage(int, long, long)`. This method will allocate page with given ID if it doesn't exist.

Specified by:
  `acquirePage` in interface `PageSupport`
Parameters:
  `grpId` - Cache group ID.
  `pageId` - Page ID.
Returns:
  Page pointer.
Throws:
  `IgniteCheckedException` - If failed.

#### acquirePage

    public long acquirePage​(int grpId,
                            long pageId,
                            IoStatisticsHolder statHolder)
                     throws IgniteCheckedException

Gets the page absolute pointer associated with the given page ID. Each page obtained with this method must be released by calling `PageSupport.releasePage(int, long, long)`. This method will allocate page with given ID if it doesn't exist.

Specified by:
  `acquirePage` in interface `PageSupport`
Parameters:
  `grpId` - Cache group ID.
  `pageId` - Page ID.
  `statHolder` - Statistics holder to track IO operations.
Returns:
  Page pointer.
Throws:
  `IgniteCheckedException` - If failed.

#### acquirePage

    public long acquirePage​(int grpId,
                            long pageId,
                            AtomicBoolean pageAllocated)
                     throws IgniteCheckedException

Specified by:
  `acquirePage` in interface `PageMemoryEx`
Parameters:
  `grpId` - Cache group ID.
  `pageId` - Page ID.
  `pageAllocated` - Flag is set if new page was allocated in offheap memory.
Returns:
  Page.
Throws:
  `IgniteCheckedException`
See Also:
  `Sets additional flag indicating that page was not found in memory and had to be allocated.`

#### acquirePage

    public long acquirePage​(int grpId,
                            long pageId,
                            IoStatisticsHolder statHolder,
                            boolean restore)
                     throws IgniteCheckedException

Specified by:
  `acquirePage` in interface `PageMemoryEx`
Parameters:
  `grpId` - Cache group ID.
  `pageId` - Page id.
  `restore` - Get page for restore
Returns:
  Page.
Throws:
  `IgniteCheckedException` - If failed.
  `StorageException` - If page reading failed from storage.
See Also:
  `Will read page from file if it is not present in memory`

#### pageSize

    public int pageSize()

Specified by:
  `pageSize` in interface `PageMemory`
Returns:
  Page size in bytes.

#### systemPageSize

    public int systemPageSize()

Specified by:
  `systemPageSize` in interface `PageMemory`
Returns:
  Page size with system overhead, in bytes.

#### realPageSize

    public int realPageSize​(int grpId)

Specified by:
  `realPageSize` in interface `PageMemory`
Parameters:
  `grpId` - Group id.
Returns:
  Page size without encryption overhead.

#### safeToUpdate

    public boolean safeToUpdate()

Heuristic method which allows a thread to check if it safe to start memory struture modifications in regard with checkpointing. May return false-negative result during or after partition eviction.

Specified by:
  `safeToUpdate` in interface `PageMemoryEx`
Returns:
  `False` if there are too many dirty pages and a thread should wait for a checkpoint to begin.

#### totalPages

    public long totalPages()

Description copied from interface: `PageMemoryEx`
Total pages can be placed to memory.

Specified by:
  `totalPages` in interface `PageMemoryEx`
Returns:
  Total pages can be placed in all segments.

#### beginCheckpoint

    public GridMultiCollectionWrapper<FullPageId> beginCheckpoint​(IgniteInternalFuture allowToReplace)
                                                           throws IgniteException

Gets a collection of dirty page IDs since the last checkpoint. If a dirty page is being written after the checkpointing operation begun, the modifications will be written to a temporary buffer which will be flushed to the main memory after the checkpointing finished. This method must be called when no concurrent operations on pages are performed.

Specified by:
  `beginCheckpoint` in interface `PageMemoryEx`
Parameters:
  `allowToReplace` - The sign which allows to replace pages from a checkpoint by page replacer.
Returns:
  Collection of dirty page IDs.
Throws:
  `IgniteException` - If checkpoint has been already started and was not finished.

#### finishCheckpoint

    public void finishCheckpoint()

Finishes checkpoint operation.

Specified by:
  `finishCheckpoint` in interface `PageMemoryEx`

#### pageManager

    public PageReadWriteManager pageManager()

Specified by:
  `pageManager` in interface `PageMemoryEx`

#### checkpointWritePage

    public void checkpointWritePage​(FullPageId fullId,
                                    ByteBuffer buf,
                                    PageStoreWriter pageStoreWriter,
                                    CheckpointMetricsTracker metricsTracker)
                             throws IgniteCheckedException

Prepare page for write during checkpoint. `PageStoreWriter` will be called when the page will be ready to write.

Specified by:
  `checkpointWritePage` in interface `PageMemoryEx`
Parameters:
  `fullId` - Page ID to get byte buffer for. The page ID must be present in the collection returned by the `PageMemoryEx.beginCheckpoint(IgniteInternalFuture)` method call.
  `buf` - Temporary buffer to write changes into.
  `pageStoreWriter` - Checkpoint page write context.
  `metricsTracker` - Checkpoint metrics tracker.
Throws:
  `IgniteCheckedException` - If failed to obtain page data.

#### invalidate

    public int invalidate​(int grpId,
                          int partId)

Marks partition as invalid / outdated.

Specified by:
  `invalidate` in interface `PageMemoryEx`
Parameters:
  `grpId` - Group ID.
  `partId` - Partition ID.
Returns:
  New partition generation (growing 1-based partition file version).

#### onCacheGroupDestroyed

    public void onCacheGroupDestroyed​(int grpId)

Clears internal metadata of destroyed cache group.

Specified by:
  `onCacheGroupDestroyed` in interface `PageMemoryEx`
Parameters:
  `grpId` - Cache group ID.

#### clearAsync

    public IgniteInternalFuture<Void> clearAsync​(LoadedPagesMap.KeyPredicate pred,
                                                 boolean cleanDirty)

Asynchronously clears pages satisfying the given predicate.

Specified by:
  `clearAsync` in interface `PageMemoryEx`
Parameters:
  `pred` - Predicate for cache group id, pageId.
  `cleanDirty` - Flag indicating that dirty pages collection should be cleaned.
Returns:
  Future that will be completed when all pages are cleared.

#### loadedPages

    public long loadedPages()

Specified by:
  `loadedPages` in interface `PageMemory`
Returns:
  Total number of loaded pages in memory.

#### acquiredPages

    public long acquiredPages()

Returns:
  Total number of acquired pages.

#### hasLoadedPage

    public boolean hasLoadedPage​(FullPageId fullPageId)

Parameters:
  `fullPageId` - Full page ID to check.
Returns:
  `true` if the page is contained in the loaded pages table, `false` otherwise.

#### readLock

    public long readLock​(long absPtr,
                         long pageId,
                         boolean force,
                         boolean touch)

Specified by:
  `readLock` in interface `PageMemoryEx`
Parameters:
  `absPtr` - Absolute pointer to read lock.
  `pageId` - Page ID.
  `force` - Force flag.
  `touch` - Update page timestamp.
Returns:
  Pointer to the page read buffer.

#### readLockForce

    public long readLockForce​(int grpId,
                              long pageId,
                              long page)

Obtains read lock without checking page tag.

Specified by:
  `readLockForce` in interface `PageSupport`
Parameters:
  `grpId` - Cache group ID.
  `pageId` - Page ID.
  `page` - Page pointer.
Returns:
  Pointer for reading the page.

#### hasTempCopy

    public boolean hasTempCopy​(long absPtr)

Checks if a page has temp copy buffer.

Parameters:
  `absPtr` - Absolute pointer.
Returns:
  `True` if a page has temp buffer.

#### activePagesCount

    public int activePagesCount()

Gets the number of active pages across all segments. Used for test purposes only.

Returns:
  Number of active pages.

#### checkpointBufferPagesCount

    public int checkpointBufferPagesCount()

Number of pages used in checkpoint buffer.

Specified by:
  `checkpointBufferPagesCount` in interface `PageMemory`

#### checkpointBufferPagesSize

    public int checkpointBufferPagesSize()

Number of used pages in checkpoint buffer.

#### segmentIndex

    public static int segmentIndex​(int grpId,
                                   long pageId,
                                   int segments)

Parameters:
  `pageId` - Page ID.
Returns:
  Segment index.

#### isCpBufferOverflowThresholdExceeded

    public boolean isCpBufferOverflowThresholdExceeded()

Checks if the Checkpoint Buffer is currently close to exhaustion.

Specified by:
  `isCpBufferOverflowThresholdExceeded` in interface `PageMemoryEx`
Returns:
  `true` if measures like throttling to protect Checkpoint Buffer should be applied, and `false` otherwise.

#### metrics

    public DataRegionMetricsImpl metrics()

Metrics of the data region this memory is associated with.

Specified by:
  `metrics` in interface `PageMemory`

#### pullPageFromCpBuffer

    public FullPageId pullPageFromCpBuffer()

Get arbitrary page from cp buffer.

Specified by:
  `pullPageFromCpBuffer` in interface `PageMemoryEx`

#### dirtyPages

    @TestOnly
    public Collection<FullPageId> dirtyPages()

Gets a collection of all pages currently marked as dirty. Will create a collection copy.

Returns:
  Collection of all page IDs marked as dirty.