Package com.databricks.jdbc.api.impl.arrow

Class StreamingInlineArrowResult

java.lang.Object

com.databricks.jdbc.api.impl.arrow.StreamingInlineArrowResult

All Implemented Interfaces:

IExecutionResult

public class StreamingInlineArrowResult
extends Object
implements IExecutionResult

High-throughput streaming implementation for inline Arrow results.

Uses ThriftStreamingProvider for proactive batch prefetching, achieving throughput comparable to eager loading while maintaining the memory benefits of lazy loading.

Key features:

• Background prefetch thread fetches Arrow batches ahead of consumption
• Sliding window limits memory usage to a configurable number of batches
• Non-blocking iteration when prefetch keeps up with consumption
• Automatic native memory cleanup via type-safe release actions
• Type-safe: Uses generic ThriftStreamingProvider<ArrowResultChunk>

This implementation replaces LazyThriftInlineArrowResult for improved throughput.

Constructor Summary

Constructors

Constructor	Description
StreamingInlineArrowResult(TFetchResultsResp initialResponse, IDatabricksStatementInternal statement, IDatabricksSession session)	Creates a new StreamingInlineArrowResult.

Method Summary

Modifier and Type	Method	Description
void	close()	Closes this result and releases associated resources.
List<String>	getArrowMetadata()	Gets the Arrow metadata for the current chunk.
int	getBatchesInMemory()	Gets the number of batches currently in memory.
long	getChunkCount()	Gets the chunk count.
long	getCurrentRow()	Gets the current row index (0-based).
Object	getObject(int columnIndex)	Gets the value at the specified column index for the current row.
long	getRowCount()	Gets the number of rows in the current batch.
long	getTotalRowsFetched()	Gets the total number of rows fetched from the server so far.
boolean	hasNext()	Checks if there are more rows available without advancing the cursor.
boolean	isCompletelyFetched()	Checks if all data has been fetched from the server.
boolean	next()	Moves the cursor to the next row.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

StreamingInlineArrowResult

public StreamingInlineArrowResult(TFetchResultsResp initialResponse,
                                  IDatabricksStatementInternal statement,
                                  IDatabricksSession session)
                           throws DatabricksSQLException

Creates a new StreamingInlineArrowResult.

Configuration values (maxBatchesInMemory, timeout) are read from the session's connection context.

Parameters:

initialResponse - The initial Thrift response containing the first Arrow batch

statement - The statement that generated this result

session - The session for fetching additional batches

Throws:

DatabricksSQLException - if initialization fails

Method Detail

getObject

public Object getObject(int columnIndex)
                 throws DatabricksSQLException

Gets the value at the specified column index for the current row.

Specified by:

getObject in interface IExecutionResult

Parameters:

columnIndex - the zero-based column index

Returns:

the value at the specified column

Throws:

DatabricksSQLException - if access fails

getCurrentRow

public long getCurrentRow()

Gets the current row index (0-based). Returns -1 if before the first row.

Specified by:

getCurrentRow in interface IExecutionResult

Returns:

the current row index

next

public boolean next()
             throws DatabricksSQLException

Moves the cursor to the next row. Fetches additional batches from server if needed.

Specified by:

next in interface IExecutionResult

Returns:

true if there is a next row, false if at the end

Throws:

DatabricksSQLException - if an error occurs

hasNext

public boolean hasNext()

Checks if there are more rows available without advancing the cursor.

Specified by:

hasNext in interface IExecutionResult

Returns:

true if there are more rows, false otherwise

close

public void close()

Closes this result and releases associated resources.

Specified by:

close in interface IExecutionResult

getRowCount

public long getRowCount()

Gets the number of rows in the current batch.

Specified by:

getRowCount in interface IExecutionResult

Returns:

the number of rows in the current batch

getChunkCount

public long getChunkCount()

Gets the chunk count. Always returns 0 for streaming results.

Specified by:

getChunkCount in interface IExecutionResult

Returns:

0

getArrowMetadata

public List<String> getArrowMetadata()
                              throws DatabricksSQLException

Gets the Arrow metadata for the current chunk.

Returns:

list of arrow metadata strings, or null if no chunk is loaded

Throws:

DatabricksSQLException - if an error occurs

getTotalRowsFetched

public long getTotalRowsFetched()

Gets the total number of rows fetched from the server so far.

Returns:

the total rows fetched

isCompletelyFetched

public boolean isCompletelyFetched()

Checks if all data has been fetched from the server.

Returns:

true if end of stream reached

getBatchesInMemory

public int getBatchesInMemory()

Gets the number of batches currently in memory.

Returns:

the batch count in memory