public abstract class ArrowReader extends Object implements DictionaryProvider, AutoCloseable
DictionaryProvider.MapDictionaryProvider
Modifier and Type | Field and Description |
---|---|
protected BufferAllocator |
allocator |
protected Map<Long,Dictionary> |
dictionaries |
Modifier | Constructor and Description |
---|---|
protected |
ArrowReader(BufferAllocator allocator) |
protected |
ArrowReader(BufferAllocator allocator,
CompressionCodec.Factory compressionFactory) |
Modifier and Type | Method and Description |
---|---|
abstract long |
bytesRead()
Return the number of bytes read from the ReadChannel.
|
void |
close()
Close resources, including vector schema root and dictionary vectors, and the
underlying read source.
|
void |
close(boolean closeReadSource)
Close resources, including vector schema root and dictionary vectors.
|
protected abstract void |
closeReadSource()
Close the underlying read source.
|
protected void |
ensureInitialized()
Initialize if not done previously.
|
Set<Long> |
getDictionaryIds()
Get all dictionary IDs.
|
Map<Long,Dictionary> |
getDictionaryVectors()
Returns any dictionaries that were loaded along with ArrowRecordBatches.
|
VectorSchemaRoot |
getVectorSchemaRoot()
Returns the vector schema root.
|
protected void |
initialize()
Reads the schema and initializes the vectors.
|
protected void |
loadDictionary(ArrowDictionaryBatch dictionaryBatch)
Load an ArrowDictionaryBatch to the readers dictionary vectors.
|
abstract boolean |
loadNextBatch()
Load the next ArrowRecordBatch to the vector schema root if available.
|
protected void |
loadRecordBatch(ArrowRecordBatch batch)
Load an ArrowRecordBatch to the readers VectorSchemaRoot.
|
Dictionary |
lookup(long id)
Lookup a dictionary that has been loaded using the dictionary id.
|
protected void |
prepareLoadNextBatch()
Ensure the reader has been initialized and reset the VectorSchemaRoot row count to 0.
|
protected abstract Schema |
readSchema()
Read the Schema from the source, will be invoked at the beginning the initialization.
|
protected final BufferAllocator allocator
protected Map<Long,Dictionary> dictionaries
protected ArrowReader(BufferAllocator allocator)
protected ArrowReader(BufferAllocator allocator, CompressionCodec.Factory compressionFactory)
public VectorSchemaRoot getVectorSchemaRoot() throws IOException
IOException
- if reading of schema failspublic Map<Long,Dictionary> getDictionaryVectors() throws IOException
IOException
- if reading of schema failspublic Dictionary lookup(long id)
lookup
in interface DictionaryProvider
id
- Unique identifier for a dictionarypublic Set<Long> getDictionaryIds()
DictionaryProvider
getDictionaryIds
in interface DictionaryProvider
public abstract boolean loadNextBatch() throws IOException
IOException
- on errorpublic abstract long bytesRead()
public void close() throws IOException
close
in interface AutoCloseable
IOException
- on errorpublic void close(boolean closeReadSource) throws IOException
closeReadSource
- Flag to control if closing the underlying read sourceIOException
- on errorprotected abstract void closeReadSource() throws IOException
IOException
- on errorprotected abstract Schema readSchema() throws IOException
IOException
- on errorprotected void ensureInitialized() throws IOException
IOException
- on errorprotected void initialize() throws IOException
IOException
protected void prepareLoadNextBatch() throws IOException
IOException
- on errorprotected void loadRecordBatch(ArrowRecordBatch batch)
batch
- the record batch to loadprotected void loadDictionary(ArrowDictionaryBatch dictionaryBatch)
dictionaryBatch
- dictionary batch to loadCopyright © 2024 The Apache Software Foundation. All rights reserved.