Apache Arrow Go 18.8.0 Release

Published 08 Sep 2026
By The Apache Arrow PMC (pmc)

The Apache Arrow team is pleased to announce the v18.8.0 release of Apache Arrow Go. This minor release covers 283 commits from 16 distinct contributors.

Contributors

$ git shortlog -sn v18.7.0..v18.8.0
Minh Vu
Matt Topol
Colton Loftus
Digvijay
Tom Frank
Derek Perkins
Lucas Valente
Madan kumar
Neelesh Salian
Nick Ripley
Pedro Matias
Raúl Cumplido
Ruihao Chen
Tobias Pütz
Tomáš Karela Procházka
Willem Jan

Highlights

Arrow

Features

Added VariantGet for efficient path extraction from variant arrays #1206
Added list_element, cumulative_sum, cumulative_sum_checked, and dictionary_encode compute functions #1138, #1139, #1140
Added ValueAsAny for accessing Arrow array values as native Go values #1170
Strengthened IPC file validation for malformed block framing and lengths #1097

Bug Fixes

Fixed a memory leak when calling Resize(0) on non-empty builders #995
Preserved exact precision when casting strings to Decimal128 and Decimal256 scalars #1107

Performance Improvements

This is a performance-heavy release: 32 of the merged pull requests are dedicated optimizations. Unless noted otherwise, the figures below are the PR-reported benchmarks on an Apple M1 Pro, comparing the new path against the scalar baseline it replaces.

Expanded SIMD acceleration for compute arithmetic, comparisons, and filtering on ARM64 and AMD64. Arithmetic kernels run 4.1-4.4x faster on int64 and float64 adds, int64 array/array comparisons drop from 313.6us to 29.3us per op (10.7x), narrow 8- and 16-bit integer comparisons reach 12.8-15.1x, and null-free 32-bit fixed-width filters compress up to 11.8x faster on 1M-element inputs #1265, #1266, #1284, #1286, #1287, #1288
Building take indices from fragmented selection vectors is no longer quadratic. For a 1M-element alternating filter it drops from 4.31s to 515us, and a serial FilterRecordBatch over the same filter falls from 2.06s to 831us #1287
Added SIMD-accelerated packing of boolean slices and validity data into Arrow bitmaps: BooleanBuilder packing of 65,536 values improves roughly 4-5x (23.4us to 3.9us), batched validity packing is up to 4.4x faster, and ARM64 NEON bitmap operations gain 2.2-2.9x #1283, #1185, #1264, #1090
Batched run-end encoded appends turn a millisecond-scale operation into a microsecond-scale one for null and empty-value runs, cutting 1.16MB across 55 allocations per op down to 1.7KB across 19 #1214
Concatenation avoids temporary buffers and copies bitmaps directly: 65,536 int64 values spread over 8,192 chunks go from 571.7us to 214.5us (-62%) with allocations falling from 8,199 to 6 per op #1194, #1224
Serial and small-batch execution skips goroutine and span overhead: single-span kernel dispatch nearly halves latency at length 1 (342.0ns to 189.0ns), serial record filtering over 128 columns goes from 709.8us to 420.9us, and serial Take plus IPC encoder scratch reuse cut per-op allocations across the board #1233, #1245, #1226, #1221
CSV writing no longer materializes rows first, cutting a 1,000-row by 16-column write from 4.78ms to 0.84ms (-82%) and allocated bytes by 81% #1190

Parquet

Features

Added read and write support for GeoArrow extension types using Parquet geometry and geography logical types #960, #969
Added codec-aware dictionary cost fallback and per-column writer controls #1036

Bug Fixes

Correctly populated Bloom filters from dictionary-encoded values #1164
Rejected decimal values that overflow Parquet INT32 or INT64 columns instead of silently narrowing them #1122, #1161
Fixed reading fixed-size lists whose parent values are null #1099

Performance Improvements

Added SIMD BYTE_STREAM_SPLIT encoding on ARM64 and AMD64: 16.9x faster for 4-byte values (113,877ns to 6,745ns per op) and 7.6x for 8-byte values #1260
Boolean decoding now targets Arrow bitmaps directly. All-true RLE boolean pages of 1M values decode in 2.45us instead of 2.65ms, and dense boolean column reads through pqarrow drop 90% in time and 65% in heap use #1246, #1182
Batched encoding and decoding across the core codecs: plain BYTE_ARRAY encoding is 45-62% faster, RLE level encoding is 79% faster for all-defined levels, and DELTA_BINARY_PACKED decoding is 34-66% faster #1193, #1177, #1169
Dictionary encoding allocates far less: numeric insertion of 65,535 float32 values goes from 64,905 allocations per op to 13, and typed byte-array memo insertion cuts time 24% and allocations 50% (Apple M3 Max). Fixed-width dictionary materialization is also vectorized on AVX2 systems #1178, #1272, #1269
Reduced copying when writing binary and fixed-size binary Arrow values to Parquet: 16-byte FixedSizeBinary writes are 72% faster for required columns with statistics disabled, and 64K String writes go from 1.798ms to 1.260ms #1262, #1263
Compressors are reused across pages: random 64KiB gzip page encoding is 6.1x faster (94.7us to 15.6us) with allocations down from 15 to 1 per op, and Brotli encoding drops from about 22 allocations per op to 1 #1238, #1254
Serial reads skip goroutine orchestration, trimming 18% from single-column row-group reads #1235

Changelog

Selected Changes

A selection of the user-facing changes in this release. See the full changelog link below for all 285 merged pull requests.

Arrow

fix: remove superfluous struct field index compare by @Willem-J-an in #970
fix(ipc): reject truncated message frames by @fallintoplace in #991
fix: Fix arm64 assembly stack frame manipulation by @nsrip-dd in #984
fix(arrow/array): Resize(0) on non-empty builder causes memory leak by @serramatutu in #995
fix(arrow/csv): preserve decimal precision by @fallintoplace in #1022
refactor(arrow/avro): migrate from hamba/avro to twmb/avro by @prochac in #830
feat(flightsql): allow configuring the driver memory allocator by @fallintoplace in #1076
feat(arrow/flight/sql): Add is_update field to ActionCreatePreparedStatementResult by @ennuite in #732
feat(arrow/array): roll back JSON builder state after failed rows by @fallintoplace in #1113
feat(arrow/array): add ValueAsAny for native Go values by @dgvj-work in #1170
feat(extensions): add VariantGet for path extraction from variant arrays by @nssalian in #1206
fix(arrow/extensions): use canonical Variant extension name by @dgvj-work in #1241
fix(arrow/extensions): reject Null Variant typed_value by @dgvj-work in #1243
fix(arrow/ipc): validate file block framing and lengths by @fallintoplace in #1097
fix(arrow/ipc): avoid deadlock on compression errors by @fallintoplace in #1142
fix(arrow/scalar): preserve exact decimal string casts by @fallintoplace in #1107
fix(arrow/scalar): align timestamp timezone parsing by @fallintoplace in #1109
fix(arrow): validate precision in NewDecimalType by @fallintoplace in #1162
fix(arrow): reject duplicate union type codes by @fallintoplace in #1131
fix(arrow/csv): preserve first row during headerless inference by @fallintoplace in #1145
fix(arrow/csv): append null for later parse failures by @fallintoplace in #1149
fix(arrow/array): clamp run ends when concatenating a sliced RunEndEncoded array by @winklemad in #1219
fix(arrow/cdata): validate imported schema topology by @fallintoplace in #1046

Compute

feat(compute): add list_element by @fallintoplace in #1138
feat(compute): add cumulative_sum and cumulative_sum_checked by @fallintoplace in #1139
feat(compute): add dictionary_encode by @fallintoplace in #1140
fix(compute): round negative HalfToOdd ties correctly by @fallintoplace in #1094
fix(compute): handle odd temporal multiples by @fallintoplace in #1119
fix(compute): preserve caller context during execution by @fallintoplace in #1154
fix(compute): discard results after cancellation by @zeroshade in #1278
fix(compute/exprs): avoid empty field reference panic by @fallintoplace in #1130

Parquet

feat(parquet/schema): Wire up logical type functionality for writing geo types by @C-Loftus in #960
feat(parquet/pqarrow): Read support for extension types / geo by @C-Loftus in #969
feat(parquet): add WithDictionaryCostFallback writer property by @tom-eon in #1036
fix(parquet): wrap DELTA_BINARY_PACKED deltas at the physical type width by @tom-eon in #1027
fix(parquet/encoding): read ahead in the streaming value buffer by @joechenrh in #937
fix(parquet/pqarrow): prefer stored schema extension type on read by @twuebi in #1051
fix(parquet/pqarrow): require Variant value field when writing by @dgvj-work in #1242
fix(parquet): guard byte-stream-split encoder release against never-flushed buffer by @tom-eon in #1273
fix(parquet): stop BYTE_STREAM_SPLIT FLBA spaced decode aliasing output by @zeroshade in #1256
fix(parquet): read null fixed-size-list parents by @fallintoplace in #1099
fix(parquet/pqarrow): reject decimal overflow in integer columns by @fallintoplace in #1122
fix(parquet/pqarrow): reject decimal overflow when writing integers by @fallintoplace in #1161
fix(parquet): build bloom filters from dictionary entries by @fallintoplace in #1164
fix(parquet): preserve dictionary index types by @fallintoplace in #1098
fix(parquet): restore Arrow duration units from schema metadata by @fallintoplace in #1095
fix(parquet/compress): honor LZ4 raw destination contract by @fallintoplace in #1147
fix(parquet): row-align V1 pages with offset indexes by @fallintoplace in #1075
fix(parquet/variant): validate metadata offset tables by @fallintoplace in #1063

Performance

perf(arrow/bitutil): add SetBitSwap/ClearBitSwap, use in builder.SetNull by @tom-eon in #1090
perf(bitutil): add ARM64 NEON bitmap operations by @fallintoplace in #1264
perf(arrow/array): pack validity values in batches by @fallintoplace in #1185
perf(arrow/array): SIMD-pack boolean slices into bitmaps by @fallintoplace in #1283
perf(arrow/array): batch run-end encoded appends by @fallintoplace in #1214
perf(arrow/array): avoid temporary buffers when concatenating by @fallintoplace in #1194
perf(arrow/array): concatenate bitmaps directly by @fallintoplace in #1224
perf(compute): add ARM64 NEON arithmetic kernels by @fallintoplace in #1265
perf(compute): add ARM64 NEON comparison kernels by @fallintoplace in #1266
perf(compute): add ARM64 NEON comparisons for 8- and 16-bit integers by @fallintoplace in #1288
perf(compute): add ARM64 NEON compression for null-free 32-bit fixed-width filters by @fallintoplace in #1286
perf(compute): vectorize mixed int32 filters with AVX2 by @fallintoplace in #1284
perf(compute): SIMD-compact fragmented filters into take indices by @fallintoplace in #1287
perf(compute): fast-path single-span execution by @fallintoplace in #1233
perf(arrow/compute): avoid goroutines for serial record filtering by @fallintoplace in #1245
perf(arrow/compute): avoid goroutines for serial take by @fallintoplace in #1226
perf(parquet): SIMD BYTE_STREAM_SPLIT encoding by @fallintoplace in #1260
perf(parquet): vectorize fixed-width dictionary materialization by @fallintoplace in #1269
perf(parquet): optimize numeric dictionary insertion by @fallintoplace in #1178
perf(parquet): use typed memo insertion for byte-array dictionaries by @derekperkins in #1272
perf(parquet): batch plain BYTE_ARRAY encoding by @fallintoplace in #1193
perf(parquet): encode RLE levels in batches by @fallintoplace in #1177
perf(parquet): batch delta binary packed decoding by @fallintoplace in #1169
perf(parquet): decode RLE booleans into bitmaps by @fallintoplace in #1246
perf(parquet/pqarrow): decode booleans directly to bitmaps by @fallintoplace in #1182
perf(parquet/pqarrow): write binary values from Arrow offsets by @fallintoplace in #1262
perf(parquet/pqarrow): write FixedSizeBinary values directly by @fallintoplace in #1263
perf(parquet/pqarrow): use synchronous path for serial reads by @fallintoplace in #1235
perf(parquet/compress): reuse gzip writers by @fallintoplace in #1238
perf(parquet/compress): reuse Brotli writers across encodes by @fallintoplace in #1254
perf(arrow/csv): avoid materializing rows before writing by @fallintoplace in #1190
perf(arrow/ipc): reuse record encoder scratch by @fallintoplace in #1221

New Contributors

@C-Loftus made their first contribution in #960
@tom-eon made their first contribution in #1027
@prochac made their first contribution in #830
@dgvj-work made their first contribution in #1170
@ennuite made their first contribution in #732
@derekperkins made their first contribution in #1272
@winklemad made their first contribution in #1219

Full Changelog: https://github.com/apache/arrow-go/compare/v18.7.0...v18.8.0