ParquetWriter<T>

A typed Parquet writer, wrapping the base Parquet writer.

Extends

• ParquetWriter

Type Parameters

Type Parameter
T extends ParquetRecordLike

Implements

• AsyncDisposable

Constructors

new ParquetWriter()

new ParquetWriter<T>(
   schema, 
   envelopeWriter, 
opts?): ParquetWriter<T>

Create a new buffered parquet writer for a given envelope writer

Parameters

Parameter	Type
schema	ParquetSchema
envelopeWriter	ParquetEnvelopeWriter
opts?	WriterOptions

Returns

ParquetWriter<T>

Inherited from

BaseParquetWriter.constructor

Properties

closed

closed: boolean;

Inherited from

BaseParquetWriter.closed

---

envelopeWriter

envelopeWriter: null | ParquetEnvelopeWriter;

Inherited from

BaseParquetWriter.envelopeWriter

---

rowBuffer

rowBuffer: RecordBuffer;

Inherited from

BaseParquetWriter.rowBuffer

---

rowGroupSize

rowGroupSize: number;

Inherited from

BaseParquetWriter.rowGroupSize

---

schema

schema: ParquetSchema<T>;

Overrides

BaseParquetWriter.schema

---

userMetadata

userMetadata: Record<string, string>;

Inherited from

BaseParquetWriter.userMetadata

---

SchemaDefinitionCache

protected readonly static SchemaDefinitionCache: ParquetSchemaDefinitionCache;

Methods

[asyncDispose]()

asyncDispose: Promise<void>

Returns

Promise<void>

Implementation of

AsyncDisposable.[asyncDispose]

---

appendRow()

appendRow(row): Promise<void>

Append a row to the buffer. If the buffer is full, the data will be written to disk.

Parameters

Parameter	Type
row	T

Returns

Promise<void>

Overrides

BaseParquetWriter.appendRow

---

close()

close(): Promise<void>

Flush all buffered data to disk, close the file, and release resources.

Returns

Promise<void>

Overrides

BaseParquetWriter.close

---

dispose()

dispose(): Promise<void>

Returns

Promise<void>

---

setMetadata()

setMetadata(key, value): void

Set a metadata key-value pair on the writer.

Parameters

Parameter	Type
key	string
value	string

Returns

void

Overrides

BaseParquetWriter.setMetadata

---

setPageSize()

setPageSize(cnt): void

Set the parquet data page size. The data page size controls the maximum number of column values that are written to disk as a consecutive array

Parameters

Parameter	Type
cnt	number

Returns

void

Inherited from

BaseParquetWriter.setPageSize

---

setRowGroupSize()

setRowGroupSize(cnt): void

Set the parquet row group size. This values controls the maximum number of rows that are buffered in memory at any given time as well as the number of rows that are co-located on disk. A higher value is generally better for read-time I/O performance at the tradeoff of write-time memory usage.

Parameters

Parameter	Type
cnt	number

Returns

void

Inherited from

BaseParquetWriter.setRowGroupSize

---

openFile()

static openFile<T>(
   schemaLike, 
   sourcePath, 
opts?): Promise<ParquetWriter<T>>

Convenience method to create a new buffered parquet writer that writes to the specified file

Type Parameters

Type Parameter
T extends ParquetRecordLike

Parameters

Parameter	Type
schemaLike	ParquetSchema<T> | ParquetSchemaDefinition<T>
sourcePath	string | URL | Buffer
opts?	WriterOptions

Returns

Promise<ParquetWriter<T>>

Overrides

BaseParquetWriter.openFile

---

openStream()

static openStream<T>(
   schemaLike, 
   outputStream, 
opts): Promise<ParquetWriter<T>>

Convenience method to create a new buffered parquet writer that writes to the specified stream

Type Parameters

Type Parameter
T extends ParquetRecordLike

Parameters

Parameter	Type
schemaLike	ParquetSchema<T> | ParquetSchemaDefinition<T>
outputStream	WriteStreamMinimal
opts	WriterOptions

Returns

Promise<ParquetWriter<T>>

Overrides

BaseParquetWriter.openStream