net.snowflake.client.jdbc.cloud.storage

Interface SnowflakeStorageClient

All Known Implementing Classes:

SnowflakeS3Client

public interface SnowflakeStorageClient

Interface for storage client provider implementations

Author:

lgiakoumakis

Method Summary

Modifier and Type	Method and Description
void	download(SFSession connection, String command, String localLocation, String destFileName, int parallelism, String remoteStorageLocation, String stageFilePath, String stageRegion) Download a file from remote storage.
int	getEncryptionKeySize()
int	getMaxRetries()
StorageObjectMetadata	getObjectMetadata(String remoteStorageLocation, String prefix) Returns the metadata properties for a remote storage object
int	getRetryBackoffMaxExponent() Returns the max exponent for multiplying backoff with the power of 2, the value of 4 will give us 16secs as the max number of time to sleep before retry
int	getRetryBackoffMin()
void	handleStorageException(Exception ex, int retryCount, String operation, SFSession connection, String command) Handles exceptions thrown by the remote storage provider
boolean	isEncrypting()
StorageObjectSummaryCollection	listObjects(String remoteStorageLocation, String prefix) For a set of remote storage objects under a remote location and a given prefix/path returns their properties wrapped in ObjectSummary objects
void	renew(Map stageCredentials) Re-creates the encapsulated storage client with a fresh access token
void	shutdown() shuts down the client
void	upload(SFSession connection, String command, int parallelism, boolean uploadFromStream, String remoteStorageLocation, File srcFile, String destFileName, InputStream inputStream, FileBackedOutputStream fileBackedOutputStream, StorageObjectMetadata meta, String stageRegion) Upload a file (-stream) to remote storage

Method Detail

getMaxRetries

int getMaxRetries()

Returns:

Returns the Max number of retry attempts

getRetryBackoffMaxExponent

int getRetryBackoffMaxExponent()

Returns the max exponent for multiplying backoff with the power of 2, the value of 4 will give us 16secs as the max number of time to sleep before retry

Returns:

Returns the exponent

getRetryBackoffMin

int getRetryBackoffMin()

Returns:

Returns the min number of milliseconds to sleep before retry

isEncrypting

boolean isEncrypting()

Returns:

Returns true if encryption is enabled

getEncryptionKeySize

int getEncryptionKeySize()

Returns:

Returns the size of the encryption key

renew

void renew(Map stageCredentials)
    throws SnowflakeSQLException

Re-creates the encapsulated storage client with a fresh access token

Parameters:

stageCredentials - a Map (as returned by GS) which contains the new credential properties

Throws:

SnowflakeSQLException - if any error occurs

shutdown

void shutdown()

shuts down the client

listObjects

StorageObjectSummaryCollection listObjects(String remoteStorageLocation,
                                           String prefix)
                                    throws StorageProviderException

For a set of remote storage objects under a remote location and a given prefix/path returns their properties wrapped in ObjectSummary objects

Parameters:

remoteStorageLocation - location, i.e. bucket for S3

prefix - the prefix to list

Returns:

a collection of storage summary objects

Throws:

StorageProviderException - if any error occurs

getObjectMetadata

StorageObjectMetadata getObjectMetadata(String remoteStorageLocation,
                                        String prefix)
                                 throws StorageProviderException

Returns the metadata properties for a remote storage object

Parameters:

remoteStorageLocation - location, i.e. bucket for S3

prefix - the prefix/path of the object to retrieve

Returns:

storage metadata object

Throws:

StorageProviderException - if any error occurs

download

void download(SFSession connection,
              String command,
              String localLocation,
              String destFileName,
              int parallelism,
              String remoteStorageLocation,
              String stageFilePath,
              String stageRegion)
       throws SnowflakeSQLException

Download a file from remote storage.

Parameters:

connection - connection object

command - command to download file

localLocation - local file path

destFileName - destination file name

parallelism - number of threads for parallel downloading

remoteStorageLocation - remote storage location, i.e. bucket for S3

stageFilePath - stage file path

stageRegion - region name where the stage persists

Throws:

SnowflakeSQLException - if any error occurs

upload

void upload(SFSession connection,
            String command,
            int parallelism,
            boolean uploadFromStream,
            String remoteStorageLocation,
            File srcFile,
            String destFileName,
            InputStream inputStream,
            FileBackedOutputStream fileBackedOutputStream,
            StorageObjectMetadata meta,
            String stageRegion)
     throws SnowflakeSQLException

Upload a file (-stream) to remote storage

Parameters:

connection - connection object

command - upload command

parallelism - number of threads do parallel uploading

uploadFromStream - true if upload source is stream

remoteStorageLocation - s3 bucket name

srcFile - source file if not uploading from a stream

destFileName - file name on remote storage after upload

inputStream - stream used for uploading if fileBackedOutputStream is null

fileBackedOutputStream - stream used for uploading if not null

meta - object meta data

stageRegion - region name where the stage persists

Throws:

SnowflakeSQLException - if upload failed even after retry

handleStorageException

void handleStorageException(Exception ex,
                            int retryCount,
                            String operation,
                            SFSession connection,
                            String command)
                     throws SnowflakeSQLException

Handles exceptions thrown by the remote storage provider

Parameters:

ex - the exception to handle

retryCount - current number of retries, incremented by the caller before each call

operation - string that indicates the function/operation that was taking place, when the exception was raised, for example "upload"

connection - the current SFSession object used by the client

command - the command attempted at the time of the exception

Throws:

SnowflakeSQLException - if any error occurs