org.h2.mvstore

## Class FileStore<C extends Chunk<C>>

- java.lang.Object
- - org.h2.mvstore.FileStore<C>

Direct Known Subclasses:
  AppendOnlyMultiFileStore, RandomAccessStore

    public abstract class FileStore<C extends Chunk<C>>
    extends java.lang.Object

Class FileStore is a base class to allow for different store implementations. FileStore concept revolves around notion of a "chunk", which is a piece of data written into the store at once.

### Nested Class Summary

Nested Classes
`class` `FileStore.PageSerializationManager`

### Field Summary

Fields
`protected C` `lastChunk` The newest chunk.
`static java.lang.String` `META_ID_KEY` The key for the entry within "layout" map, which contains id of "meta" map.
`static int` `PIPE_LENGTH`
`protected java.util.concurrent.atomic.AtomicLong` `readBytes` The number of read bytes.
`protected java.util.concurrent.atomic.AtomicLong` `readCount` The number of read operations.
`protected boolean` `recoveryMode`
`protected java.util.concurrent.locks.ReentrantLock` `saveChunkLock`
`protected java.util.HashMap<java.lang.String,java.lang.Object>` `storeHeader`
`protected java.util.concurrent.atomic.AtomicLong` `writeBytes` The number of written bytes.
`protected java.util.concurrent.atomic.AtomicLong` `writeCount` The number of write operations.

### Constructor Summary

Constructors
`protected` `FileStore(java.util.Map<java.lang.String,java.lang.Object> config)`

### Method Summary

All Methods Instance Methods Abstract Methods Concrete Methods
`void` `accountForRemovedPage(long pos, long version, boolean pinned, int pageNo)` Remove a page.
`protected abstract void` `adjustStoreToLastChunk()` Make persistent changes after lastChunk was reset
`protected abstract void` `allocateChunkSpace(C chunk, WriteBuffer buff)` Allocate logical space and assign position of the buffer within the store.
`abstract void` `backup(java.util.zip.ZipOutputStream out)`
`void` `bind(MVStore mvStore)`
`void` `clear()`
`void` `close()`
`boolean` `compact(int targetFillRate, int write)` Try to increase the fill rate by re-writing partially full chunks.
`protected abstract void` `compactStore(int thresholdFillRate, long maxCompactTime, int maxWriteSize, MVStore mvStore)` Compact store file, that is, compact blocks that have a low fill rate, and move chunks next to each other.
`void` `compactStore(long maxCompactTime)`
`protected abstract C` `createChunk(int id)`
`protected abstract C` `createChunk(java.util.Map<java.lang.String,java.lang.String> map)`
`abstract C` `createChunk(java.lang.String s)` Build a Chunk from the given string.
`boolean` `deregisterMapRoot(int mapId)` Performs final stage of map removal - delete root location info from the layout map.
`protected C` `discoverChunk(long block)` Discover a valid chunk, searching file backwards from the given block
`protected abstract void` `doHousekeeping(MVStore mvStore)`
`void` `dropUnusedChunks()`
`void` `executeFilestoreOperation(java.lang.Runnable operation)`
`protected boolean` `findLastChunkWithCompleteValidChunkSet(java.util.Comparator<C> chunkComparator, java.util.Map<java.lang.Long,C> validChunksByLocation, boolean afterFullScan)`
`protected abstract void` `freeChunkSpace(java.lang.Iterable<C> chunks)` Mark the space occupied by specified chunks as free.
`int` `getAutoCommitDelay()` Get the auto-commit delay.
`protected int` `getAutoCompactFillRate()`
`int` `getCacheHitRatio()`
`int` `getCacheSize()` Get the maximum cache size, in MB.
`int` `getCacheSizeUsed()` Get the amount of memory used for caching, in MB.
`protected java.util.concurrent.ConcurrentMap<java.lang.Integer,C>` `getChunks()`
`int` `getChunksFillRate()` Get the current fill rate (percentage of used space in the file).
`protected java.lang.Iterable<C>` `getChunksFromLayoutMap()`
`long` `getCreationTime()` The time the store was created, in milliseconds since 1970.
`int` `getDefaultRetentionTime()` Get the default retention time for this store in milliseconds.
`java.lang.String` `getFileName()` Get the file name.
`abstract int` `getFillRate()`
`java.util.Map<java.lang.String,java.lang.String>` `getLayoutMap()` Get this store's layout map.
`long` `getMaxPageSize()`
`int` `getMetaMapId(java.util.function.IntSupplier nextIdSupplier)`
`protected MVStore` `getMvStore()`
`long` `getReadBytes()` Get the number of read bytes since this store was opened.
`long` `getReadCount()` Get the number of read operations since this store was opened.
`int` `getRetentionTime()`
`protected java.util.Collection<C>` `getRewriteCandidates()`
`long` `getRootPos(int mapId)` Get "position" of the root page for the specified map
`java.util.Map<java.lang.String,java.lang.Object>` `getStoreHeader()` Get the store header.
`int` `getTocCacheHitRatio()`
`WriteBuffer` `getWriteBuffer()` Get a buffer for writing.
`long` `getWriteCount()` Get the number of write operations since this store was opened.
`boolean` `hasChangesSince(long lastStoredVersion)` Check whether there are any unsaved changes since specified version.
`protected boolean` `hasPersistentData()`
`protected void` `init(java.lang.String fileName, boolean readOnly)`
`protected void` `initializeCommonHeaderAttributes(long time)`
`protected abstract void` `initializeStoreHeader(long time)`
`protected boolean` `isIdle()`
`boolean` `isKnownVersion(long version)` Check whether all data can be read from this version.
`boolean` `isReadOnly()`
`boolean` `isRegularMap(MVMap<?,?> map)`
`boolean` `isSpaceReused()`
`long` `lastChunkVersion()`
`protected abstract void` `markUsed(long pos, int length)` Mark the space as in use.
`abstract FileStore<C>` `open(java.lang.String fileName, boolean readOnly)`
`abstract void` `open(java.lang.String fileName, boolean readOnly, char[] encryptionKey)`
`void` `populateInfo(java.util.function.BiConsumer<java.lang.String,java.lang.String> consumer)`
`protected void` `processCommonHeaderAttributes()`
`protected C` `readChunkFooter(long block)` Try to read a chunk footer.
`protected C` `readChunkHeaderAndFooter(long block, int expectedId)` Read a chunk header and footer, and verify the stored data is consistent.
`protected C` `readChunkHeaderOptionally(long block)`
`protected C` `readChunkHeaderOptionally(long block, int expectedId)`
`abstract java.nio.ByteBuffer` `readFully(C chunk, long pos, int len)` Read data from the store.
`protected java.nio.ByteBuffer` `readFully(java.nio.channels.FileChannel file, long pos, int len)`
`protected abstract void` `readStoreHeader(boolean recoveryMode)`
`protected void` `registerDeadChunk(C chunk)`
`void` `releaseWriteBuffer(WriteBuffer buff)` Release a buffer for writing.
`protected boolean` `rewriteChunks(int writeLimit, int targetFillRate)`
`void` `rollbackTo(long version)`
`void` `saveChunkMetadataChanges(C chunk)` Store chunk's serialized metadata as an entry in a layout map.
`void` `setAutoCommitDelay(int millis)` Set the maximum delay in milliseconds to auto-commit changes.
`void` `setCacheSize(int mb)` Set the read cache size in MB.
`protected void` `setLastChunk(C last)`
`void` `setRetentionTime(int ms)` How long to retain old, persisted chunks, in milliseconds.
`void` `setReuseSpace(boolean reuseSpace)`
`protected void` `setSize(long size)`
`abstract boolean` `shouldSaveNow(int unsavedMemory, int autoCommitMemory)` Decision about autocommit is delegated to store
`protected abstract void` `shrinkStoreIfPossible(int minPercent)` Shrink store if possible, and if at least a given percentage can be saved.
`long` `size()` Get the file size.
`MVMap<java.lang.String,java.lang.String>` `start()`
`void` `stop(long allowedCompactionTime)`
`protected void` `store()`
`void` `sync()`
`protected abstract boolean` `validateFileLength(java.lang.String msg)`
`protected abstract void` `writeChunk(C chunk, WriteBuffer buffer)` Write buffer associated with chunk into store at chunk's allocated position
`protected void` `writeCleanShutdown()`
`protected abstract void` `writeCleanShutdownMark()` Performs final preparation before store is closed normally
`protected abstract void` `writeFully(C chunk, long pos, java.nio.ByteBuffer src)` Write to the file.

### Methods inherited from class java.lang.Object

`clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

### Field Detail

#### META_ID_KEY

    public static final java.lang.String META_ID_KEY

The key for the entry within "layout" map, which contains id of "meta" map. Entry value (hex encoded) is usually equal to 1, unless it's a legacy (upgraded) database and id 1 has been taken already by another map.

See Also:
  Constant Field Values

#### readCount

    protected final java.util.concurrent.atomic.AtomicLong readCount

The number of read operations.

#### readBytes

    protected final java.util.concurrent.atomic.AtomicLong readBytes

The number of read bytes.

#### writeCount

    protected final java.util.concurrent.atomic.AtomicLong writeCount

The number of write operations.

#### writeBytes

    protected final java.util.concurrent.atomic.AtomicLong writeBytes

The number of written bytes.

#### lastChunk

    protected volatile C extends Chunk<C> lastChunk

The newest chunk. If nothing was stored yet, this field is not set.

#### saveChunkLock

    protected final java.util.concurrent.locks.ReentrantLock saveChunkLock

#### storeHeader

    protected final java.util.HashMap<java.lang.String,java.lang.Object> storeHeader

#### recoveryMode

    protected final boolean recoveryMode

#### PIPE_LENGTH

    public static final int PIPE_LENGTH

See Also:
  Constant Field Values

### Constructor Detail

#### FileStore

    protected FileStore(java.util.Map<java.lang.String,java.lang.Object> config)

### Method Detail

#### open

    public abstract void open(java.lang.String fileName,
                              boolean readOnly,
                              char[] encryptionKey)

#### open

    public abstract FileStore<C> open(java.lang.String fileName,
                                      boolean readOnly)

#### init

    protected final void init(java.lang.String fileName,
                              boolean readOnly)

#### bind

    public final void bind(MVStore mvStore)

#### stop

    public final void stop(long allowedCompactionTime)

#### close

    public void close()

#### getMetaMapId

    public final int getMetaMapId(java.util.function.IntSupplier nextIdSupplier)

#### getLayoutMap

    public final java.util.Map<java.lang.String,java.lang.String> getLayoutMap()

Get this store's layout map. This data is for informational purposes only. The data is subject to change in future versions.

The data in this map should not be modified (changing system data may corrupt the store).

The layout map contains the following entries:

     chunk.{chunkId} = {chunk metadata}
     root.{mapId} = {root position}

Returns:
  the metadata map

#### isRegularMap

    public final boolean isRegularMap(MVMap<?,?> map)

#### getRootPos

    public final long getRootPos(int mapId)

Get "position" of the root page for the specified map

Parameters:
  `mapId` - to get root position for
Returns:
  opaque "position" value, that should be used to read the page

#### deregisterMapRoot

    public final boolean deregisterMapRoot(int mapId)

Performs final stage of map removal - delete root location info from the layout map. Specified map is supposedly closed, is anonymous and has no outstanding usage by now.

Parameters:
  `mapId` - to deregister
Returns:
  true if root was removed, false if it is not there

#### hasChangesSince

    public final boolean hasChangesSince(long lastStoredVersion)

Check whether there are any unsaved changes since specified version.

Parameters:
  `lastStoredVersion` - version to take as a base for changes
Returns:
  if there are any changes

#### lastChunkVersion

    public final long lastChunkVersion()

#### getMaxPageSize

    public final long getMaxPageSize()

#### getRetentionTime

    public final int getRetentionTime()

#### setRetentionTime

    public final void setRetentionTime(int ms)

How long to retain old, persisted chunks, in milliseconds. Chunks that are older may be overwritten once they contain no live data.

The default value is 45000 (45 seconds) when using the default file store. It is assumed that a file system and hard disk will flush all write buffers within this time. Using a lower value might be dangerous, unless the file system and hard disk flush the buffers earlier. To manually flush the buffers, use `MVStore.getFile().force(true)`, however please note that according to various tests this does not always work as expected depending on the operating system and hardware.

The retention time needs to be long enough to allow reading old chunks while traversing over the entries of a map.

This setting is not persisted.

Parameters:
  `ms` - how many milliseconds to retain old chunks (0 to overwrite them as early as possible)

#### shouldSaveNow

    public abstract boolean shouldSaveNow(int unsavedMemory,
                                          int autoCommitMemory)

Decision about autocommit is delegated to store

Parameters:
  `unsavedMemory` - amount of unsaved memory, so far
  `autoCommitMemory` - configured limit on amount of unsaved memory
Returns:
  true if commit should happen now

#### getAutoCommitDelay

    public final int getAutoCommitDelay()

Get the auto-commit delay.

Returns:
  the delay in milliseconds, or 0 if auto-commit is disabled.

#### setAutoCommitDelay

    public final void setAutoCommitDelay(int millis)

Set the maximum delay in milliseconds to auto-commit changes.

To disable auto-commit, set the value to 0. In this case, changes are only committed when explicitly calling commit.

The default is 1000, meaning all changes are committed after at most one second.

Parameters:
  `millis` - the maximum delay

#### isKnownVersion

    public final boolean isKnownVersion(long version)

Check whether all data can be read from this version. This requires that all chunks referenced by this version are still available (not overwritten).

Parameters:
  `version` - the version
Returns:
  true if all data can be read

#### rollbackTo

    public final void rollbackTo(long version)

#### initializeCommonHeaderAttributes

    protected final void initializeCommonHeaderAttributes(long time)

#### processCommonHeaderAttributes

    protected final void processCommonHeaderAttributes()

#### hasPersistentData

    protected final boolean hasPersistentData()

#### isIdle

    protected final boolean isIdle()

#### setLastChunk

    protected final void setLastChunk(C last)

#### registerDeadChunk

    protected final void registerDeadChunk(C chunk)

#### dropUnusedChunks

    public final void dropUnusedChunks()

#### writeFully

    protected abstract void writeFully(C chunk,
                                       long pos,
                                       java.nio.ByteBuffer src)

Write to the file.

Parameters:
  `chunk` - to write
  `pos` - the write position
  `src` - the source buffer

#### readFully

    public abstract java.nio.ByteBuffer readFully(C chunk,
                                                  long pos,
                                                  int len)

Read data from the store.

Parameters:
  `chunk` - that owns data to be read
  `pos` - the read "position"
  `len` - the number of bytes to read
Returns:
  the byte buffer with data requested

#### readFully

    protected final java.nio.ByteBuffer readFully(java.nio.channels.FileChannel file,
                                                  long pos,
                                                  int len)

#### allocateChunkSpace

    protected abstract void allocateChunkSpace(C chunk,
                                               WriteBuffer buff)

Allocate logical space and assign position of the buffer within the store.

Parameters:
  `chunk` - to allocate space for
  `buff` - to allocate space for

#### writeChunk

    protected abstract void writeChunk(C chunk,
                                       WriteBuffer buffer)

Write buffer associated with chunk into store at chunk's allocated position

Parameters:
  `chunk` - chunk to write
  `buffer` - to write

#### writeCleanShutdownMark

    protected abstract void writeCleanShutdownMark()

Performs final preparation before store is closed normally

#### adjustStoreToLastChunk

    protected abstract void adjustStoreToLastChunk()

Make persistent changes after lastChunk was reset

#### getStoreHeader

    public java.util.Map<java.lang.String,java.lang.Object> getStoreHeader()

Get the store header. This data is for informational purposes only. The data is subject to change in future versions. The data should not be modified (doing so may corrupt the store).

Returns:
  the store header

#### createChunk

    protected abstract C createChunk(int id)

#### createChunk

    public abstract C createChunk(java.lang.String s)

Build a Chunk from the given string.

Parameters:
  `s` - the string
Returns:
  the Chunk created

#### createChunk

    protected abstract C createChunk(java.util.Map<java.lang.String,java.lang.String> map)

#### writeCleanShutdown

    protected void writeCleanShutdown()

#### saveChunkMetadataChanges

    public void saveChunkMetadataChanges(C chunk)

Store chunk's serialized metadata as an entry in a layout map. Key for this entry would be "chunk.<id>"

Parameters:
  `chunk` - to save

#### freeChunkSpace

    protected abstract void freeChunkSpace(java.lang.Iterable<C> chunks)

Mark the space occupied by specified chunks as free.

Parameters:
  `chunks` - chunks to be processed

#### validateFileLength

    protected abstract boolean validateFileLength(java.lang.String msg)

#### compact

    public boolean compact(int targetFillRate,
                           int write)

Try to increase the fill rate by re-writing partially full chunks. Chunks with a low number of live items are re-written.

If the current fill rate is higher than the target fill rate, nothing is done.

Please note this method will not necessarily reduce the file size, as empty chunks are not overwritten.

Only data of open maps can be moved. For maps that are not open, the old chunk is still referenced. Therefore, it is recommended to open all maps before calling this method.

Parameters:
  `targetFillRate` - the minimum percentage of live entries
  `write` - the minimum number of bytes to write
Returns:
  if any chunk was re-written

#### compactStore

    public void compactStore(long maxCompactTime)

#### compactStore

    protected abstract void compactStore(int thresholdFillRate,
                                         long maxCompactTime,
                                         int maxWriteSize,
                                         MVStore mvStore)

Compact store file, that is, compact blocks that have a low fill rate, and move chunks next to each other. This will typically shrink the file. Changes are flushed to the file, and old chunks are overwritten.

Parameters:
  `thresholdFillRate` - do not compact if store fill rate above this value (0-100)
  `maxCompactTime` - the maximum time in milliseconds to compact
  `maxWriteSize` - the maximum amount of data to be written as part of this call
  `mvStore` - that owns this FileStore

#### doHousekeeping

    protected abstract void doHousekeeping(MVStore mvStore)
                                    throws java.lang.InterruptedException

Throws:
  `java.lang.InterruptedException`

#### start

    public MVMap<java.lang.String,java.lang.String> start()

#### initializeStoreHeader

    protected abstract void initializeStoreHeader(long time)

#### readStoreHeader

    protected abstract void readStoreHeader(boolean recoveryMode)

#### discoverChunk

    protected final C discoverChunk(long block)

Discover a valid chunk, searching file backwards from the given block

Parameters:
  `block` - to start search from (found chunk footer should be no further than block-1)
Returns:
  valid chunk or null if none found

#### findLastChunkWithCompleteValidChunkSet

    protected final boolean findLastChunkWithCompleteValidChunkSet(java.util.Comparator<C> chunkComparator,
                                                                   java.util.Map<java.lang.Long,C> validChunksByLocation,
                                                                   boolean afterFullScan)

#### getChunksFromLayoutMap

    protected java.lang.Iterable<C> getChunksFromLayoutMap()

#### readChunkHeaderAndFooter

    protected final C readChunkHeaderAndFooter(long block,
                                               int expectedId)

Read a chunk header and footer, and verify the stored data is consistent.

Parameters:
  `block` - the block
  `expectedId` - of the chunk
Returns:
  the chunk, or null if the header or footer don't match or are not consistent

#### readChunkHeaderOptionally

    protected final C readChunkHeaderOptionally(long block,
                                                int expectedId)

#### readChunkHeaderOptionally

    protected final C readChunkHeaderOptionally(long block)

#### readChunkFooter

    protected final C readChunkFooter(long block)

Try to read a chunk footer.

Parameters:
  `block` - the index of the next block after the chunk
Returns:
  the chunk, or null if not successful

#### getWriteBuffer

    public WriteBuffer getWriteBuffer()

Get a buffer for writing. This caller must synchronize on the store before calling the method and until after using the buffer.

Returns:
  the buffer

#### releaseWriteBuffer

    public void releaseWriteBuffer(WriteBuffer buff)

Release a buffer for writing. This caller must synchronize on the store before calling the method and until after using the buffer.

Parameters:
  `buff` - the buffer than can be re-used

#### getCreationTime

    public long getCreationTime()

The time the store was created, in milliseconds since 1970.

Returns:
  creation time

#### getAutoCompactFillRate

    protected final int getAutoCompactFillRate()

#### sync

    public void sync()

#### getFillRate

    public abstract int getFillRate()

#### shrinkStoreIfPossible

    protected abstract void shrinkStoreIfPossible(int minPercent)

Shrink store if possible, and if at least a given percentage can be saved.

Parameters:
  `minPercent` - the minimum percentage to save

#### size

    public long size()

Get the file size.

Returns:
  the file size

#### setSize

    protected final void setSize(long size)

#### getWriteCount

    public long getWriteCount()

Get the number of write operations since this store was opened. For file based stores, this is the number of file write operations.

Returns:
  the number of write operations

#### getReadCount

    public long getReadCount()

Get the number of read operations since this store was opened. For file based stores, this is the number of file read operations.

Returns:
  the number of read operations

#### getReadBytes

    public long getReadBytes()

Get the number of read bytes since this store was opened.

Returns:
  the number of write operations

#### isReadOnly

    public boolean isReadOnly()

#### getDefaultRetentionTime

    public int getDefaultRetentionTime()

Get the default retention time for this store in milliseconds.

Returns:
  the retention time

#### clear

    public void clear()

#### getFileName

    public java.lang.String getFileName()

Get the file name.

Returns:
  the file name

#### getMvStore

    protected final MVStore getMvStore()

#### markUsed

    protected abstract void markUsed(long pos,
                                     int length)

Mark the space as in use.

Parameters:
  `pos` - the position in bytes
  `length` - the number of bytes

#### backup

    public abstract void backup(java.util.zip.ZipOutputStream out)
                         throws java.io.IOException

Throws:
  `java.io.IOException`

#### getChunks

    protected final java.util.concurrent.ConcurrentMap<java.lang.Integer,C> getChunks()

#### getRewriteCandidates

    protected java.util.Collection<C> getRewriteCandidates()

#### isSpaceReused

    public boolean isSpaceReused()

#### setReuseSpace

    public void setReuseSpace(boolean reuseSpace)

#### store

    protected final void store()

#### getChunksFillRate

    public int getChunksFillRate()

Get the current fill rate (percentage of used space in the file). Unlike the fill rate of the store, here we only account for chunk data; the fill rate here is how much of the chunk data is live (still referenced). Young chunks are considered live.

Returns:
  the fill rate, in percent (100 is completely full)

#### getCacheSize

    public int getCacheSize()

Get the maximum cache size, in MB. Note that this does not include the page chunk references cache, which is 25% of the size of the page cache.

Returns:
  the cache size

#### getCacheSizeUsed

    public int getCacheSizeUsed()

Get the amount of memory used for caching, in MB. Note that this does not include the page chunk references cache, which is 25% of the size of the page cache.

Returns:
  the amount of memory used for caching

#### setCacheSize

    public void setCacheSize(int mb)

Set the read cache size in MB.

Parameters:
  `mb` - the cache size in MB.

#### populateInfo

    public void populateInfo(java.util.function.BiConsumer<java.lang.String,java.lang.String> consumer)

#### getCacheHitRatio

    public int getCacheHitRatio()

#### getTocCacheHitRatio

    public int getTocCacheHitRatio()

#### rewriteChunks

    protected boolean rewriteChunks(int writeLimit,
                                    int targetFillRate)

#### executeFilestoreOperation

    public void executeFilestoreOperation(java.lang.Runnable operation)

#### accountForRemovedPage

    public void accountForRemovedPage(long pos,
                                      long version,
                                      boolean pinned,
                                      int pageNo)

Remove a page.

Parameters:
  `pos` - the position of the page
  `version` - at which page was removed
  `pinned` - whether page is considered pinned
  `pageNo` - sequential page number within chunk