# Class AbstractRemoteChunkProvider<T extends AbstractArrowResultChunk>

java.lang.Object
com.databricks.jdbc.api.impl.arrow.AbstractRemoteChunkProvider<T>

Type Parameters:
  `T` - The specific type of AbstractArrowResultChunk this provider manages

All Implemented Interfaces:
  `ChunkProvider`

Direct Known Subclasses:
  `RemoteChunkProvider`, `RemoteChunkProviderV2`

public abstract class AbstractRemoteChunkProvider<T extends AbstractArrowResultChunk> extends Object implements ChunkProvider
Abstract base implementation of both `ChunkProvider` and `ChunkDownloadManager` interfaces.

The provider maintains a concurrent map of chunks and implements a sliding window approach or memory management, releasing consumed chunks and downloading new ones as needed. It ensures that the number of chunks in memory never exceeds the configured parallel download limit.

## Field Summary

Fields
Modifier and Type
Field
Description
`protected long`
`allowedChunksInMemory`
`protected long`
`chunkCount`
`protected final ConcurrentMap<Long,T>`
`chunkIndexToChunksMap`
`protected final int`
`chunkReadyTimeoutSeconds`
`protected final CompressionCodec`
`compressionCodec`
`protected long`
`currentChunkIndex`
`protected final IDatabricksHttpClient`
`httpClient`
`protected boolean`
`isClosed`
`protected final ChunkLinkDownloadService<T>`
`linkDownloadService`
`protected final int`
`maxParallelChunkDownloadsPerQuery`
Maximum number of parallel chunk downloads allowed per query.
`protected long`
`nextChunkToDownload`
`protected long`
`rowCount`
`protected final IDatabricksSession`
`session`
`protected final StatementId`
`statementId`
`protected long`
`totalChunksInMemory`

## Constructor Summary

Constructors
Modifier
Constructor
Description
`protected`
`AbstractRemoteChunkProvider(IDatabricksStatementInternal parentStatement, TFetchResultsResp resultsResp, IDatabricksSession session, IDatabricksHttpClient httpClient, int maxParallelChunkDownloadsPerQuery, CompressionCodec compressionCodec)`
`protected`
`AbstractRemoteChunkProvider(StatementId statementId, ResultManifest resultManifest, ResultData resultData, IDatabricksSession session, IDatabricksHttpClient httpClient, int maxParallelChunkDownloadsPerQuery, CompressionCodec compressionCodec)`

## Method Summary

Modifier and Type
Method
Description
`final void`
`close()`
Closes the chunk provider and releases any resources associated with it.
`protected abstract T`
`createChunk(StatementId statementId, long chunkIndex, TSparkArrowResultLink resultLink)`
Creates chunk `AbstractRemoteChunkProvider` based on the `TSparkArrowResultLink`.
`protected abstract T`
`createChunk(StatementId statementId, long chunkIndex, com.databricks.sdk.service.sql.BaseChunkInfo chunkInfo)`
Creates chunk `AbstractRemoteChunkProvider` based on the `BaseChunkInfo`.
`protected void`
`doClose()`
Subclasses should override this method to perform their specific cleanup.
`void`
`downloadNextChunks()`
Downloads the next set of chunks based on the current memory constraints and chunk availability.
`long`
`getAllowedChunksInMemory()`
`T`
`getChunk()`
Retrieves the current chunk of data after a successful call to `ChunkProvider.next()`.
`long`
`getChunkCount()`
`CompressionCodec`
`getCompressionCodec()`
`long`
`getRowCount()`
`boolean`
`hasNextChunk()`
Checks if there are more chunks available to iterate over.
`boolean`
`isClosed()`
`boolean`
`next()`
Advances to the next available chunk.

### Methods inherited from class java.lang.Object

`clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

## Field Details

### session

protected final IDatabricksSession session

### statementId

protected final StatementId statementId

### httpClient

protected final IDatabricksHttpClient httpClient

### compressionCodec

protected final CompressionCodec compressionCodec

### chunkIndexToChunksMap

protected final ConcurrentMap<Long,T extends AbstractArrowResultChunk> chunkIndexToChunksMap

### chunkCount

protected long chunkCount

### rowCount

protected long rowCount

### currentChunkIndex

protected long currentChunkIndex

### nextChunkToDownload

protected long nextChunkToDownload

### totalChunksInMemory

protected long totalChunksInMemory

### allowedChunksInMemory

protected long allowedChunksInMemory

### isClosed

protected boolean isClosed

### maxParallelChunkDownloadsPerQuery

protected final int maxParallelChunkDownloadsPerQuery
Maximum number of parallel chunk downloads allowed per query.

### linkDownloadService

protected final ChunkLinkDownloadService<T extends AbstractArrowResultChunk> linkDownloadService

### chunkReadyTimeoutSeconds

protected final int chunkReadyTimeoutSeconds

## Constructor Details

### AbstractRemoteChunkProvider

protected AbstractRemoteChunkProvider(StatementId statementId, ResultManifest resultManifest, ResultData resultData, IDatabricksSession session, IDatabricksHttpClient httpClient, int maxParallelChunkDownloadsPerQuery, CompressionCodec compressionCodec) throws DatabricksSQLException

Throws:
  `DatabricksSQLException`

### AbstractRemoteChunkProvider

protected AbstractRemoteChunkProvider(IDatabricksStatementInternal parentStatement, TFetchResultsResp resultsResp, IDatabricksSession session, IDatabricksHttpClient httpClient, int maxParallelChunkDownloadsPerQuery, CompressionCodec compressionCodec) throws DatabricksSQLException

Throws:
  `DatabricksSQLException`

## Method Details

### createChunk

protected abstract T createChunk(StatementId statementId, long chunkIndex, com.databricks.sdk.service.sql.BaseChunkInfo chunkInfo) throws DatabricksSQLException
Creates chunk `AbstractRemoteChunkProvider` based on the `BaseChunkInfo`. Used in SQL Execution API flow.

Throws:
  `DatabricksSQLException`

### createChunk

protected abstract T createChunk(StatementId statementId, long chunkIndex, TSparkArrowResultLink resultLink) throws DatabricksSQLException
Creates chunk `AbstractRemoteChunkProvider` based on the `TSparkArrowResultLink`. Used in Thrift CLI flow.

Throws:
  `DatabricksSQLException`

### getCompressionCodec

public CompressionCodec getCompressionCodec()

### hasNextChunk

public boolean hasNextChunk()
Checks if there are more chunks available to iterate over.

Specified by:
  `hasNextChunk` in interface `ChunkProvider`
Returns:
  `true` if there are additional chunks to be retrieved; `false` otherwise.

### getRowCount

public long getRowCount()

Specified by:
  `getRowCount` in interface `ChunkProvider`

### getChunkCount

public long getChunkCount()

Specified by:
  `getChunkCount` in interface `ChunkProvider`

### getChunk

public T getChunk() throws DatabricksSQLException
Retrieves the current chunk of data after a successful call to `ChunkProvider.next()`.

Retrieves and waits for the current chunk to be ready.

Specified by:
  `getChunk` in interface `ChunkProvider`
Returns:
  The current `AbstractArrowResultChunk` containing the data.
Throws:
  `DatabricksSQLException` - if an error occurs while fetching the chunk.

### next

public boolean next() throws DatabricksSQLException
Advances to the next available chunk. This method should be called before calling `ChunkProvider.getChunk()` to retrieve the data from the next chunk.

Specified by:
  `next` in interface `ChunkProvider`
Returns:
  `true` if the next chunk was successfully moved to; `false` if there are no more chunks.
Throws:
  `DatabricksSQLException`

### close

public final void close()
Closes the chunk provider and releases any resources associated with it. After calling this method, the chunk provider should not be used again.

This method is final to ensure proper cleanup order. Subclasses should override doClose() instead.

Specified by:
  `close` in interface `ChunkProvider`

### isClosed

public boolean isClosed()

Specified by:
  `isClosed` in interface `ChunkProvider`

### getAllowedChunksInMemory

public long getAllowedChunksInMemory()

### doClose

protected void doClose()
Subclasses should override this method to perform their specific cleanup.

### downloadNextChunks

void downloadNextChunks() throws DatabricksSQLException
Downloads the next set of chunks based on the current memory constraints and chunk availability. This method manages the download process by respecting:

- The total number of chunks allowed in memory
- Whether the provider has been closed
- The availability of more chunks to download

Throws:
  `DatabricksSQLException` - If there's an error during the chunk download process