Package org.apache.ignite.internal.processors.cache.persistence.pagemem

Interface PagesWriteThrottlePolicy

  • All Known Implementing Classes:
    PagesWriteSpeedBasedThrottle, PagesWriteThrottle
    public interface PagesWriteThrottlePolicy
    Throttling policy, encapsulates logic of delaying write operations.

    There are two resources that get (or might get) consumed when writing:

    • Checkpoint Buffer where a page is placed when, being under checkpoint, it gets written
    • Clean pages which get dirtied when writes occur
    Both resources are limited in size. Both are freed when checkpoint finishes. This means that, if writers write too fast, they can consume any of these two resources before we have a chance to finish a checkpoint. If this happens, the cluster fails or stalls.

    Write throttling solves this problem by slowing down the writers to a rate at which they do not exhaust any of the two resources.

    An alternative to just slowing down is to wait in a loop till the resource we're after gets freed, and only then allow the write to happen. The problem with this approach is that we cannot wait in a loop/sleep under a write lock, so the logic would be a lot more complicated. Maybe in the future we'll follow this path, but for now, a simpler approach of just throttling is used (see below).

    If we just slow writers down by throttling their writes, AND we have enough Checkpoint Buffer and pages in segments to take some load bursts, we are fine. Under such assumptions, it does not matter whether we throttle a writer thread before acquiring write lock or after it gets released; in the current implementation, this happens after write lock gets released (because it was considered simpler to implement).

    The actual throttling happens when a page gets marked dirty by calling onMarkDirty(boolean).

    There are two additional methods for interfacing with other parts of the system:

    • wakeupThrottledThreads() which wakes up the threads currently being throttled; in the current implementation, it is called when Checkpoint Buffer utilization falls below 1/2.
    • isCpBufferOverflowThresholdExceeded() which is called by a checkpointer to see whether the Checkpoint Buffer is in a danger zone and, if yes, it starts to prioritize writing pages from the Checkpoint Buffer over pages from the normal checkpoint sequence.

    • Method Detail

      • onMarkDirty

        void onMarkDirty​(boolean isPageInCheckpoint)
        Callback to apply throttling delay.
        Parameters:
        isPageInCheckpoint - flag indicating if current page is in scope of current checkpoint.

      • wakeupThrottledThreads

        void wakeupThrottledThreads()
        Callback to wakeup throttled threads. Invoked when the Checkpoint Buffer use drops below a certain threshold.

      • onBeginCheckpoint

        void onBeginCheckpoint()
        Callback to notify throttling policy checkpoint was started.

      • onFinishCheckpoint

        void onFinishCheckpoint()
        Callback to notify throttling policy checkpoint was finished.

      • isCpBufferOverflowThresholdExceeded

        boolean isCpBufferOverflowThresholdExceeded()
        Whether Checkpoint Buffer is currently close to exhaustion.
        Returns:
        true if measures like throttling to protect Checkpoint Buffer should be applied, and false otherwise.