Package org.apache.ignite.internal.processors.cache.persistence.pagemem

Class PageMemoryImpl

  • All Implemented Interfaces:
    PageIdAllocator, PageMemory, PageSupport, PageMemoryEx
    public class PageMemoryImpl
extends Object
implements PageMemoryEx
    Page header structure is described by the following diagram. When page is not allocated (in a free list): 
     +--------+------------------------------------------------------+
 |8 bytes |         PAGE_SIZE + PAGE_OVERHEAD - 8 bytes          |
 +--------+------------------------------------------------------+
 |Next ptr|                      Page data                       |
 +--------+------------------------------------------------------+

    When page is allocated and is in use: 

     +------------------+--------+--------+----+----+--------+--------+----------------------+
 |     8 bytes      |8 bytes |8 bytes |4 b |4 b |8 bytes |8 bytes |       PAGE_SIZE      |
 +------------------+--------+--------+----+----+--------+--------+----------------------+
 | Marker/Timestamp |Rel ptr |Page ID |C ID|PIN | LOCK   |TMP BUF |       Page data      |
 +------------------+--------+--------+----+----+--------+--------+----------------------+
    Note that first 8 bytes of page header are used either for page marker or for next relative pointer depending on whether the page is in use or not.

    • Constructor Detail

      • PageMemoryImpl

        public PageMemoryImpl​(DirectMemoryProvider directMemoryProvider,
                      long[] sizes,
                      GridCacheSharedContext<?,​?> ctx,
                      PageReadWriteManager pmPageMgr,
                      int pageSize,
                      PageStoreWriter flushDirtyPage,
                      boolean trackable,
                      CheckpointLockStateChecker stateChecker,
                      DataRegionMetricsImpl dataRegionMetrics,
                      DataRegionConfiguration dataRegionCfg,
                      @Nullable
                      @Nullable PageMemoryImpl.ThrottlingPolicy throttlingPlc,
                      IgniteOutClosure<CheckpointProgress> cpProgressProvider)
        Parameters:
        directMemoryProvider - Memory allocator to use.
        sizes - segments sizes, last is checkpoint pool size.
        ctx - Cache shared context.
        pmPageMgr - Page store manager.
        pageSize - Page size.
        flushDirtyPage - write callback invoked when a dirty page is removed for replacement.
        trackable - Obsolete flag. Keeped for compatibility with existing PDS.
        stateChecker - Checkpoint lock state provider. Used to ensure lock is held by thread, which modify pages.
        dataRegionMetrics - Memory metrics to track dirty pages count and page replace rate.
        dataRegionCfg - Data region configuration.
        throttlingPlc - Write throttle enabled and its type. Null equal to none.
        cpProgressProvider - checkpoint progress, base for throttling. Null disables throttling.

    • Method Detail

      • releasePage

        public void releasePage​(int grpId,
                        long pageId,
                        long page)
        Specified by:
        releasePage in interface PageSupport
        Parameters:
        grpId - Cache group ID.
        pageId - Page ID to release.
        page - Page pointer.

      • readLock

        public long readLock​(int grpId,
                     long pageId,
                     long page)
        Specified by:
        readLock in interface PageSupport
        Parameters:
        grpId - Cache group ID.
        pageId - Page ID.
        page - Page pointer.
        Returns:
        Pointer for reading the page.

      • readUnlock

        public void readUnlock​(int grpId,
                       long pageId,
                       long page)
        Releases locked page.
        Specified by:
        readUnlock in interface PageSupport
        Parameters:
        grpId - Cache group ID.
        pageId - Page ID.
        page - Page pointer.

      • writeLock

        public long writeLock​(int grpId,
                      long pageId,
                      long page)
        Specified by:
        writeLock in interface PageSupport
        Parameters:
        grpId - Cache group ID.
        pageId - Page ID.
        page - Page pointer.
        Returns:
        Address of a buffer with contents of the given page or 0L if attempt to take the write lock failed.

      • writeLock

        public long writeLock​(int grpId,
                      long pageId,
                      long page,
                      boolean restore)
        Specified by:
        writeLock in interface PageMemoryEx
        Parameters:
        grpId - Group ID.
        pageId - Page ID.
        page - Page pointer.
        restore - Determines if the page is locked for restore memory (crash recovery).
        Returns:
        ByteBuffer for modifying the page.

      • tryWriteLock

        public long tryWriteLock​(int grpId,
                         long pageId,
                         long page)
        Specified by:
        tryWriteLock in interface PageSupport
        Parameters:
        grpId - Cache group ID.
        pageId - Page ID.
        page - Page pointer.
        Returns:
        Address of a buffer with contents of the given page or 0L if attempt to take the write lock failed.

      • writeUnlock

        public void writeUnlock​(int grpId,
                        long pageId,
                        long page,
                        Boolean walPlc,
                        boolean dirtyFlag)
        Releases locked page.
        Specified by:
        writeUnlock in interface PageSupport
        Parameters:
        grpId - Cache group ID.
        pageId - Page ID.
        page - Page pointer.
        walPlc - True if page should be recorded to WAL, false if the page must not be recorded and null for the default behavior.
        dirtyFlag - Determines whether the page was modified since the last checkpoint.

      • writeUnlock

        public void writeUnlock​(int grpId,
                        long pageId,
                        long page,
                        Boolean walPlc,
                        boolean dirtyFlag,
                        boolean restore)
        Specified by:
        writeUnlock in interface PageMemoryEx
        Parameters:
        grpId - Group ID.
        pageId - Page ID.
        page - Page pointer.
        walPlc - True if page should be recorded to WAL, false if the page must not be recorded and null for the default behavior.
        dirtyFlag - Determines whether the page was modified since the last checkpoint.
        restore - Determines if the page is locked for restore.

      • isDirty

        public boolean isDirty​(int grpId,
                       long pageId,
                       long page)
        Specified by:
        isDirty in interface PageSupport
        Parameters:
        grpId - Cache group ID.
        pageId - Page ID.
        page - Page pointer.
        Returns:
        True if the page is dirty.

      • allocatePage

        public long allocatePage​(int grpId,
                         int partId,
                         byte flags)
                  throws IgniteCheckedException
        Allocates a page from the space for the given partition ID and the given flags.
        Specified by:
        allocatePage in interface PageIdAllocator
        Parameters:
        grpId - Cache Group ID.
        partId - Partition ID.
        Returns:
        Allocated page ID.
        Throws:
        IgniteCheckedException

      • pageBuffer

        public ByteBuffer pageBuffer​(long pageAddr)
        Specified by:
        pageBuffer in interface PageMemory
        Parameters:
        pageAddr - Page address.
        Returns:
        Page byte buffer.

      • freePage

        public boolean freePage​(int grpId,
                        long pageId)
        The given page is free now.
        Specified by:
        freePage in interface PageIdAllocator
        Parameters:
        grpId - Cache Group ID.
        pageId - Page ID.

      • partitionMetaPageId

        public long partitionMetaPageId​(int grpId,
                                int partId)
        Gets partition metadata page for specified grpId and partId.
        Specified by:
        partitionMetaPageId in interface PageMemoryEx
        Parameters:
        grpId - Group ID.
        partId - Partition ID.
        Returns:
        Meta page for grpId and partId.

      • pageSize

        public int pageSize()
        Specified by:
        pageSize in interface PageMemory
        Returns:
        Page size in bytes.

      • systemPageSize

        public int systemPageSize()
        Specified by:
        systemPageSize in interface PageMemory
        Returns:
        Page size with system overhead, in bytes.

      • realPageSize

        public int realPageSize​(int grpId)
        Specified by:
        realPageSize in interface PageMemory
        Parameters:
        grpId - Group id.
        Returns:
        Page size without encryption overhead.

      • safeToUpdate

        public boolean safeToUpdate()
        Heuristic method which allows a thread to check if it safe to start memory struture modifications in regard with checkpointing. May return false-negative result during or after partition eviction.
        Specified by:
        safeToUpdate in interface PageMemoryEx
        Returns:
        False if there are too many dirty pages and a thread should wait for a checkpoint to begin.

      • totalPages

        public long totalPages()
        Description copied from interface: PageMemoryEx
        Total pages can be placed to memory.
        Specified by:
        totalPages in interface PageMemoryEx
        Returns:
        Total pages can be placed in all segments.

      • beginCheckpoint

        public GridMultiCollectionWrapper<FullPageId> beginCheckpoint​(IgniteInternalFuture allowToReplace)
                                                       throws IgniteException
        Gets a collection of dirty page IDs since the last checkpoint. If a dirty page is being written after the checkpointing operation begun, the modifications will be written to a temporary buffer which will be flushed to the main memory after the checkpointing finished. This method must be called when no concurrent operations on pages are performed.
        Specified by:
        beginCheckpoint in interface PageMemoryEx
        Parameters:
        allowToReplace - The sign which allows to replace pages from a checkpoint by page replacer.
        Returns:
        Collection of dirty page IDs.
        Throws:
        IgniteException - If checkpoint has been already started and was not finished.

      • finishCheckpoint

        public void finishCheckpoint()
        Finishes checkpoint operation.
        Specified by:
        finishCheckpoint in interface PageMemoryEx

      • invalidate

        public int invalidate​(int grpId,
                      int partId)
        Marks partition as invalid / outdated.
        Specified by:
        invalidate in interface PageMemoryEx
        Parameters:
        grpId - Group ID.
        partId - Partition ID.
        Returns:
        New partition generation (growing 1-based partition file version).

      • onCacheGroupDestroyed

        public void onCacheGroupDestroyed​(int grpId)
        Clears internal metadata of destroyed cache group.
        Specified by:
        onCacheGroupDestroyed in interface PageMemoryEx
        Parameters:
        grpId - Cache group ID.

      • clearAsync

        public IgniteInternalFuture<Void> clearAsync​(LoadedPagesMap.KeyPredicate pred,
                                             boolean cleanDirty)
        Asynchronously clears pages satisfying the given predicate.
        Specified by:
        clearAsync in interface PageMemoryEx
        Parameters:
        pred - Predicate for cache group id, pageId.
        cleanDirty - Flag indicating that dirty pages collection should be cleaned.
        Returns:
        Future that will be completed when all pages are cleared.

      • loadedPages

        public long loadedPages()
        Specified by:
        loadedPages in interface PageMemory
        Returns:
        Total number of loaded pages in memory.

      • acquiredPages

        public long acquiredPages()
        Returns:
        Total number of acquired pages.

      • hasLoadedPage

        public boolean hasLoadedPage​(FullPageId fullPageId)
        Parameters:
        fullPageId - Full page ID to check.
        Returns:
        true if the page is contained in the loaded pages table, false otherwise.

      • readLock

        public long readLock​(long absPtr,
                     long pageId,
                     boolean force,
                     boolean touch)
        Specified by:
        readLock in interface PageMemoryEx
        Parameters:
        absPtr - Absolute pointer to read lock.
        pageId - Page ID.
        force - Force flag.
        touch - Update page timestamp.
        Returns:
        Pointer to the page read buffer.

      • readLockForce

        public long readLockForce​(int grpId,
                          long pageId,
                          long page)
        Obtains read lock without checking page tag.
        Specified by:
        readLockForce in interface PageSupport
        Parameters:
        grpId - Cache group ID.
        pageId - Page ID.
        page - Page pointer.
        Returns:
        Pointer for reading the page.

      • hasTempCopy

        public boolean hasTempCopy​(long absPtr)
        Checks if a page has temp copy buffer.
        Parameters:
        absPtr - Absolute pointer.
        Returns:
        True if a page has temp buffer.

      • activePagesCount

        public int activePagesCount()
        Gets the number of active pages across all segments. Used for test purposes only.
        Returns:
        Number of active pages.

      • checkpointBufferPagesCount

        public int checkpointBufferPagesCount()
        Number of pages used in checkpoint buffer.
        Specified by:
        checkpointBufferPagesCount in interface PageMemory

      • checkpointBufferPagesSize

        public int checkpointBufferPagesSize()
        Number of used pages in checkpoint buffer.

      • segmentIndex

        public static int segmentIndex​(int grpId,
                               long pageId,
                               int segments)
        Parameters:
        pageId - Page ID.
        Returns:
        Segment index.

      • isCpBufferOverflowThresholdExceeded

        public boolean isCpBufferOverflowThresholdExceeded()
        Checks if the Checkpoint Buffer is currently close to exhaustion.
        Specified by:
        isCpBufferOverflowThresholdExceeded in interface PageMemoryEx
        Returns:
        true if measures like throttling to protect Checkpoint Buffer should be applied, and false otherwise.

      • dirtyPages

        @TestOnly
public Collection<FullPageId> dirtyPages()
        Gets a collection of all pages currently marked as dirty. Will create a collection copy.
        Returns:
        Collection of all page IDs marked as dirty.