<?xml version="1.0" encoding="utf-8"?><feed xmlns="http://www.w3.org/2005/Atom" ><generator uri="https://jekyllrb.com/" version="4.4.1">Jekyll</generator><link href="https://arrow.apache.org/feed.xml" rel="self" type="application/atom+xml" /><link href="https://arrow.apache.org/" rel="alternate" type="text/html" /><updated>2026-11-17T11:25:12-05:00</updated><id>https://arrow.apache.org/feed.xml</id><title type="html">Apache Arrow</title><subtitle>Apache Arrow is the universal columnar format and multi-language toolbox for fast data interchange and in-memory analytics. It specifies a standardized language-independent column-oriented memory format for flat and nested data, organized for efficient analytic operations on modern hardware. It also provides computational libraries and zero-copy streaming messaging and interprocess communication. Languages currently supported include C, C++, C#, Go, Java, JavaScript, MATLAB, Python, R, Ruby, and Rust.</subtitle><entry><title type="html">Apache Arrow Go 18.9.0 Release</title><link href="https://arrow.apache.org/blog/2026/11/16/arrow-go-18.9.0/" rel="alternate" type="text/html" title="Apache Arrow Go 18.9.0 Release" /><published>2026-11-16T00:00:00-05:00</published><updated>2026-11-16T00:00:00-05:00</updated><id>https://arrow.apache.org/blog/2026/11/16/arrow-go-18.9.0</id><content type="html" xml:base="https://arrow.apache.org/blog/2026/11/16/arrow-go-18.9.0/"><![CDATA[<!--

-->
<p>The Apache Arrow team is pleased to announce the v18.9.0 release of Apache Arrow Go.
This minor release covers 152 commits from 27 distinct contributors.</p>
<h2>Contributors</h2>
<pre><code class="language-console">$ git shortlog -sn v18.8.0..v18.9.0
    45	Minh Vu
    37	Harrison Crosse
    31	Matt Topol
     6	singhpratech
     3	Sankalp Thakur
     2	Animesh Kumar
     2	Colton Loftus
     2	Edward Shen
     2	Samuele Resca
     2	Steve O'Brien
     2	Stochastic
     2	Willem Jan
     2	autoCAT
     2	matanper
     1	Arnaud
     1	Arnold Wakim
     1	Ayaan
     1	Jake Thomas
     1	Kyon
     1	Madan Kumar
     1	Nirajan Sah
     1	Ondřej Pavela
     1	Tobias Pütz
     1	hp
     1	peterolkhov
     1	prtkgaur
     1	ump45nose
</code></pre>
<h2>Highlights</h2>
<ul>
<li>Go 1.26 is now the minimum supported Go version <a href="https://github.com/apache/arrow-go/pull/1326">#1326</a></li>
</ul>
<h3>Arrow</h3>
<h4>Features</h4>
<ul>
<li>Added a scalar aggregate framework to the compute package, starting with the <code>count</code> and <code>sum</code> kernels <a href="https://github.com/apache/arrow-go/pull/1336">#1336</a></li>
<li>Added the <code>starts_with</code> compute function, including <code>string_view</code>/<code>binary_view</code> inputs, <code>IgnoreCase</code>, and the Substrait <code>starts_with</code> mapping used by
iceberg-go prefix row filters <a href="https://github.com/apache/arrow-go/pull/1448">#1448</a>, <a href="https://github.com/apache/arrow-go/pull/1467">#1467</a>, <a href="https://github.com/apache/arrow-go/pull/1468">#1468</a></li>
<li>Added <code>SortKey.FloatTotalOrder</code> to sort floats by IEEE 754 totalOrder, distinguishing <code>-0</code>/<code>+0</code> and signed NaNs <a href="https://github.com/apache/arrow-go/pull/1430">#1430</a></li>
<li>Added <code>Decimal32</code>, <code>Decimal64</code>, <code>StringView</code> and <code>BinaryView</code> scalar types <a href="https://github.com/apache/arrow-go/pull/1501">#1501</a>, <a href="https://github.com/apache/arrow-go/pull/1490">#1490</a></li>
<li>Added <code>ipc.WithZstdLevel</code> to choose the zstd compression level of IPC writers <a href="https://github.com/apache/arrow-go/pull/1376">#1376</a></li>
<li>Added <code>Reader.ContinueFrom</code> to resume an IPC reader on a stream that arrives as several blobs, a pattern that stopped working in v18.8.0 <a href="https://github.com/apache/arrow-go/pull/1346">#1346</a></li>
<li>The Flight SQL <code>database/sql</code> driver now accepts <code>grpc+tcp://</code>, <code>grpc+tls://</code> and <code>grpc+unix://</code> DSNs <a href="https://github.com/apache/arrow-go/pull/1420">#1420</a></li>
<li>The JSON and CSV readers expose their input offset to help locate decoding errors <a href="https://github.com/apache/arrow-go/pull/1342">#1342</a></li>
</ul>
<h4>Bug Fixes</h4>
<ul>
<li><code>is_null</code>, <code>is_not_null</code> and <code>not</code> returned wrong masks for sliced arrays, dictionaries with null values, and null, union and run-end encoded arrays, so
filters could silently keep or drop the wrong rows. <code>is_null</code> now also honors <code>NullOptions.NanIsNull</code> <a href="https://github.com/apache/arrow-go/pull/1456">#1456</a>,
<a href="https://github.com/apache/arrow-go/pull/1458">#1458</a>, <a href="https://github.com/apache/arrow-go/pull/1461">#1461</a>, <a href="https://github.com/apache/arrow-go/pull/1465">#1465</a>, <a href="https://github.com/apache/arrow-go/pull/1488">#1488</a></li>
<li><code>NewDatumWithoutOwning</code> now returns a non-owning Datum; previously <code>Release()</code> freed the caller's memory <a href="https://github.com/apache/arrow-go/pull/1377">#1377</a></li>
<li><code>and_kleene</code> and <code>or_kleene</code> returned <code>false</code> instead of null when <code>ExecCtx.ChunkSize</code> was smaller than the input <a href="https://github.com/apache/arrow-go/pull/1306">#1306</a></li>
<li>A cancelled <code>CallFunction</code> no longer returns a result with a nil error, and the default <code>NumParallel</code> follows the cgroup-aware <code>GOMAXPROCS</code> in
containers <a href="https://github.com/apache/arrow-go/pull/1410">#1410</a>, <a href="https://github.com/apache/arrow-go/pull/1384">#1384</a></li>
<li>Fixed generated assembly that clobbered the frame pointer or Go's reserved registers, breaking profilers and frame-pointer unwinders, in the amd64 compute
kernels and the Parquet bit-unpacking and Bloom filter kernels <a href="https://github.com/apache/arrow-go/pull/1432">#1432</a>, <a href="https://github.com/apache/arrow-go/pull/1300">#1300</a>, <a href="https://github.com/apache/arrow-go/pull/1379">#1379</a></li>
<li>Decimal rescaling no longer panics or returns wrong values, infinite decimal strings are rejected, and scales past <code>MaxScale</code> fail clearly <a href="https://github.com/apache/arrow-go/pull/1307">#1307</a>,
<a href="https://github.com/apache/arrow-go/pull/1409">#1409</a>, <a href="https://github.com/apache/arrow-go/pull/1513">#1513</a>, <a href="https://github.com/apache/arrow-go/pull/1514">#1514</a>, <a href="https://github.com/apache/arrow-go/pull/1515">#1515</a></li>
<li>The C Data interface exports extension arrays as their storage and imports zero-length fixed-size lists, such as those returned by DataFusion <a href="https://github.com/apache/arrow-go/pull/1526">#1526</a>,
<a href="https://github.com/apache/arrow-go/pull/1308">#1308</a></li>
<li><code>arrow.TypeEqual</code> now compares <code>LargeListType</code> like <code>ListType</code>, so <code>array.Concatenate</code> no longer rejects identical large list types <a href="https://github.com/apache/arrow-go/pull/1440">#1440</a></li>
<li><code>NewMappedFileReader</code> no longer panics on dictionary fields, and Flight SQL <code>ExecuteIngest</code> sends the schema for empty inputs <a href="https://github.com/apache/arrow-go/pull/1365">#1365</a>,
<a href="https://github.com/apache/arrow-go/pull/1433">#1433</a></li>
<li>The JSON record reader no longer swallows an unexpected EOF on truncated input <a href="https://github.com/apache/arrow-go/pull/1341">#1341</a></li>
</ul>
<h4>Performance Improvements</h4>
<p>Performance remains a major theme: 51 of the merged pull requests are dedicated
optimizations. Unless noted otherwise, the figures below are the PR-reported
benchmarks on an Apple M1 Pro.</p>
<ul>
<li>Bulk array comparisons: comparing an array without a validity bitmap against one with a materialized bitmap is up to 414x faster for booleans (8.8ms to 21.3us
over 1M values) and 15.3x for int64, and all-valid list equality over 65,536 rows drops from 17.8ms to 0.197ms <a href="https://github.com/apache/arrow-go/pull/1327">#1327</a>,
<a href="https://github.com/apache/arrow-go/pull/1330">#1330</a>, <a href="https://github.com/apache/arrow-go/pull/1347">#1347</a></li>
<li>SIMD filtering now covers 8-, 16- and 64-bit types: 1M-element alternating int64 filters go from 3.97ms to 0.45ms (8.7x), int16 from 4.23ms to 225us, and
8-bit alternating masks run 22-24x faster. AVX2 take-index compaction adds up to 4.5x on AMD EPYC <a href="https://github.com/apache/arrow-go/pull/1311">#1311</a>,
<a href="https://github.com/apache/arrow-go/pull/1349">#1349</a>, <a href="https://github.com/apache/arrow-go/pull/1350">#1350</a>, <a href="https://github.com/apache/arrow-go/pull/1348">#1348</a></li>
<li><code>SortIndices</code> radix-sorts single-chunk, null-free integer columns: 65,536 random int64 values sort in 1.31ms instead of 12.3ms <a href="https://github.com/apache/arrow-go/pull/1391">#1391</a></li>
<li>Dictionary-heavy workloads: <code>unique</code> over 256 chunks with changing dictionaries is 36.6x faster (60.0ms to 1.64ms) with 98.5% fewer bytes allocated,
concatenating chunks that share a dictionary drops from 82.4us to 3.7us, and builders reuse unchanged dictionaries, cutting the per-row cost of
small-batch dictionary reads through <code>pqarrow</code> by up to 93% <a href="https://github.com/apache/arrow-go/pull/1395">#1395</a>, <a href="https://github.com/apache/arrow-go/pull/1403">#1403</a>, <a href="https://github.com/apache/arrow-go/pull/1444">#1444</a></li>
<li>Bitmap utilities: <code>AppendBools</code> takes 88% less time (geometric mean), unaligned bitmap operations are up to 11.4x faster, and misaligned bitmap equality and
unaligned transfers improve 6.5x and 5.4x (geometric means) <a href="https://github.com/apache/arrow-go/pull/1402">#1402</a>, <a href="https://github.com/apache/arrow-go/pull/1437">#1437</a>,
<a href="https://github.com/apache/arrow-go/pull/1414">#1414</a>, <a href="https://github.com/apache/arrow-go/pull/1416">#1416</a></li>
<li>IPC zstd compression: pooled decoders cut decompression time by 69-78% for buffers up to 256KiB per column, and pooled encoders take a writer-per-record
workload from 246.4us to 4.5us and from about 2.2MiB to 7.1KiB allocated per op (Apple M2 Pro) <a href="https://github.com/apache/arrow-go/pull/1331">#1331</a>, <a href="https://github.com/apache/arrow-go/pull/1429">#1429</a></li>
<li>Full-file CSV reads stream into builders: 1.69x faster with 71% fewer bytes allocated <a href="https://github.com/apache/arrow-go/pull/1393">#1393</a></li>
</ul>
<h3>Parquet</h3>
<h4>Features</h4>
<ul>
<li>Added opt-in ALP (Adaptive Lossless floating-Point) encoding for FLOAT and DOUBLE columns, interoperable with Arrow C++. Writers enable it with
<code>WithALPEncoding</code>; reading needs no option <a href="https://github.com/apache/arrow-go/pull/1289">#1289</a></li>
<li>Added an optional <code>parquet/encryption</code> package with KMS support, so files encrypted by pyarrow's <code>CryptoFactory</code> (PKMT1 key material, single or double
wrapping) can be read through <code>KmsKeyRetriever</code> <a href="https://github.com/apache/arrow-go/pull/1478">#1478</a></li>
<li>Write geospatial statistics (bounding box and geometry types) for GEOMETRY column chunks, and <code>null_count</code> statistics for GEOMETRY and GEOGRAPHY <a href="https://github.com/apache/arrow-go/pull/1504">#1504</a>,
<a href="https://github.com/apache/arrow-go/pull/1533">#1533</a></li>
</ul>
<h4>Bug Fixes</h4>
<ul>
<li>Encryption interoperability: read encrypted files written by parquet-mr (Spark, Hive, Iceberg on parquet-java), preserve <code>AES_GCM_CTR_V1</code> in plaintext
footers, and return an error instead of panicking on columns without a key <a href="https://github.com/apache/arrow-go/pull/1390">#1390</a>, <a href="https://github.com/apache/arrow-go/pull/1412">#1412</a>, <a href="https://github.com/apache/arrow-go/pull/1406">#1406</a></li>
<li><code>pqarrow</code> wrote wrong data for sliced <code>FixedSizeList</code> columns and for <code>FixedSizeList</code> columns spanning more than one row group <a href="https://github.com/apache/arrow-go/pull/1382">#1382</a></li>
<li>Encoders no longer carry stale state across pages: RLE boolean columns repeated earlier values on later pages, plain boolean and delta encoders wrote
pre-<code>Reset</code> state, and all-null <code>DELTA_BINARY_PACKED</code> pages were padded with zeros <a href="https://github.com/apache/arrow-go/pull/1539">#1539</a>, <a href="https://github.com/apache/arrow-go/pull/1545">#1545</a>,
<a href="https://github.com/apache/arrow-go/pull/1546">#1546</a>, <a href="https://github.com/apache/arrow-go/pull/1400">#1400</a></li>
<li>Statistics: V2 data pages no longer report zero nulls without page statistics, byte array statistics written from Arrow count empty strings, and
<code>UpdateSpaced</code> counts non-null values from the validity bitmap <a href="https://github.com/apache/arrow-go/pull/1543">#1543</a>, <a href="https://github.com/apache/arrow-go/pull/1527">#1527</a>, <a href="https://github.com/apache/arrow-go/pull/1367">#1367</a></li>
<li>Fixed a memory leak in DELTA decoders, and dictionary-encoded boolean columns now return an error instead of panicking <a href="https://github.com/apache/arrow-go/pull/1394">#1394</a>,
<a href="https://github.com/apache/arrow-go/pull/1547">#1547</a></li>
</ul>
<h4>Performance Improvements</h4>
<ul>
<li>Reading dictionary-encoded columns allocates the dictionary up front again, fixing an allocation regression introduced in v18.8.0 <a href="https://github.com/apache/arrow-go/pull/1473">#1473</a></li>
<li>Bloom filter hashing pools its scratch buffers: hashing 64 values drops from 5.52us to 311ns, and dictionary Bloom filter writes take up to 32% less time
(Intel Core Ultra 7 155H) <a href="https://github.com/apache/arrow-go/pull/1378">#1378</a></li>
<li>Level and boolean decoding: nullable boolean bitmap expansion takes 85% less time (geometric mean), 1-bit packed levels decode in 55-60% less time, all-true and
all-false RLE booleans decode 66-98% faster (amd64), and narrow RLE literal levels encode 2.25-2.61x faster <a href="https://github.com/apache/arrow-go/pull/1405">#1405</a>,
<a href="https://github.com/apache/arrow-go/pull/1404">#1404</a>, <a href="https://github.com/apache/arrow-go/pull/1417">#1417</a>, <a href="https://github.com/apache/arrow-go/pull/1419">#1419</a></li>
<li>Dictionary encoding: indices are encoded in RLE batches (65,536 constant indices drop from 187.7us to 26.0us), and FLBA and Int96 dictionary insertion is
29-33% faster with up to 50% fewer allocations <a href="https://github.com/apache/arrow-go/pull/1328">#1328</a>, <a href="https://github.com/apache/arrow-go/pull/1312">#1312</a></li>
<li>DELTA encodings: constant <code>DELTA_BINARY_PACKED</code> blocks decode 2.1x faster, <code>DELTA_LENGTH_BYTE_ARRAY</code> decodes 1.62-2.00x and discards 2.43-2.74x faster,
<code>DELTA_BYTE_ARRAY</code> prefix matching is up to 4.7x faster (AMD EPYC), and its discard path drops from 65,535 allocations to none <a href="https://github.com/apache/arrow-go/pull/1396">#1396</a>,
<a href="https://github.com/apache/arrow-go/pull/1415">#1415</a>, <a href="https://github.com/apache/arrow-go/pull/1388">#1388</a>, <a href="https://github.com/apache/arrow-go/pull/1321">#1321</a></li>
<li>Reads: optional columns use dense decoding for batches without nulls (up to 45% faster per row on AMD EPYC), and serial reads build struct child readers
without goroutines, 37-43% faster (Apple M4 Max) <a href="https://github.com/apache/arrow-go/pull/1447">#1447</a>, <a href="https://github.com/apache/arrow-go/pull/1434">#1434</a></li>
</ul>
<h2>Changelog</h2>
<h3>Selected Changes</h3>
<p>A selection of the user-facing changes in this release. See the full changelog link
below for every change in the release, including automated dependency updates.</p>
<h4>Arrow</h4>
<ul>
<li>feat(arrow/ipc): add Reader.ContinueFrom for streams split across blobs by @zeroshade in <a href="https://github.com/apache/arrow-go/pull/1346">#1346</a></li>
<li>feat(arrow/ipc): add WithZstdLevel option for IPC writer by @Asthenia0412 in <a href="https://github.com/apache/arrow-go/pull/1376">#1376</a></li>
<li>feat(arrow/flight/flightsql/driver): accept grpc+tcp, grpc+tls and grpc+unix schemes by @nirajan1111 in <a href="https://github.com/apache/arrow-go/pull/1420">#1420</a></li>
<li>fix: expose internal offsets in json/csv readers by @Willem-J-an in <a href="https://github.com/apache/arrow-go/pull/1342">#1342</a></li>
<li>feat(arrow/scalar): add StringView and BinaryView scalars by @hcrosse in <a href="https://github.com/apache/arrow-go/pull/1490">#1490</a></li>
<li>feat(arrow/scalar): add Decimal32 and Decimal64 scalars by @hcrosse in <a href="https://github.com/apache/arrow-go/pull/1501">#1501</a></li>
<li>fix(arrow): support zero-length fixed-size lists by @jakthom in <a href="https://github.com/apache/arrow-go/pull/1308">#1308</a></li>
<li>fix(array/record): bldr stop ignore unexpected EOF by @Willem-J-an in <a href="https://github.com/apache/arrow-go/pull/1341">#1341</a></li>
<li>fix(arrow/ipc): open mapped files with dictionary fields by @hpaliwal8 in <a href="https://github.com/apache/arrow-go/pull/1365">#1365</a></li>
<li>fix(arrow): handle dense union slices and empty validation by @skyon0522-ai in <a href="https://github.com/apache/arrow-go/pull/1385">#1385</a></li>
<li>fix(arrow/array): validate bufferless run-end encoded arrays by @zeroshade in <a href="https://github.com/apache/arrow-go/pull/1421">#1421</a></li>
<li>fix(arrow/flight/flightsql): send schema for empty ingest by @arnoldwakim in <a href="https://github.com/apache/arrow-go/pull/1433">#1433</a></li>
<li>fix(arrow): compare LargeListType like ListType in TypeEqual by @paveon in <a href="https://github.com/apache/arrow-go/pull/1440">#1440</a></li>
<li>fix(arrow/bitutil): preserve overlapping bitmap transfers by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1389">#1389</a></li>
<li>fix(arrow/decimal): widen Decimal64 rescale to 64 bits by @winklemad in <a href="https://github.com/apache/arrow-go/pull/1307">#1307</a></li>
<li>fix(arrow/decimal): handle native word widths in big integer conversions by @zeroshade in <a href="https://github.com/apache/arrow-go/pull/1409">#1409</a></li>
<li>fix(arrow/decimal): correct rescaling results and report cast errors by @hcrosse in <a href="https://github.com/apache/arrow-go/pull/1513">#1513</a></li>
<li>fix(arrow/decimal): reject infinite decimal strings and bound extreme exponents by @hcrosse in <a href="https://github.com/apache/arrow-go/pull/1514">#1514</a></li>
<li>fix(arrow/decimal): fail clearly for Decimal128 and Decimal256 scales past MaxScale by @hcrosse in <a href="https://github.com/apache/arrow-go/pull/1515">#1515</a></li>
<li>fix(arrow/cdata): export extension arrays as their storage by @hcrosse in <a href="https://github.com/apache/arrow-go/pull/1526">#1526</a></li>
<li>fix(arrow/cdata): export an empty view sizes buffer as the zero region by @hcrosse in <a href="https://github.com/apache/arrow-go/pull/1544">#1544</a></li>
<li>fix(arrow/array): validate extension arrays by their storage type by @hcrosse in <a href="https://github.com/apache/arrow-go/pull/1487">#1487</a></li>
<li>fix(arrow/array): keep the storage dictionary in NewExtensionArrayWithStorage by @hcrosse in <a href="https://github.com/apache/arrow-go/pull/1540">#1540</a></li>
<li>fix(arrow/csv): preserve braces in list values by @peterolkhov in <a href="https://github.com/apache/arrow-go/pull/1499">#1499</a></li>
<li>fix(arrow/csv): preserve lists holding a single empty string by @zeroshade in <a href="https://github.com/apache/arrow-go/pull/1505">#1505</a></li>
<li>fix(arrow/csv): keep list items equal to the null value distinct from nulls by @zeroshade in <a href="https://github.com/apache/arrow-go/pull/1508">#1508</a></li>
</ul>
<h4>Compute</h4>
<ul>
<li>feat(arrow/compute): scalar aggregate framework with the count and sum kernels by @singhpratech in <a href="https://github.com/apache/arrow-go/pull/1336">#1336</a></li>
<li>feat(arrow/compute): add IEEE 754 total-order float sorting option by @zeroshade in <a href="https://github.com/apache/arrow-go/pull/1430">#1430</a></li>
<li>feat(arrow/compute): add starts_with kernel by @hcrosse in <a href="https://github.com/apache/arrow-go/pull/1448">#1448</a></li>
<li>feat(arrow/compute): support string_view and binary_view in starts_with by @hcrosse in <a href="https://github.com/apache/arrow-go/pull/1467">#1467</a></li>
<li>feat(arrow/compute): support IgnoreCase in starts_with by @hcrosse in <a href="https://github.com/apache/arrow-go/pull/1468">#1468</a></li>
<li>fix(compute): detect temporal rounding overflow by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1127">#1127</a></li>
<li>fix(compute/exec): do not carry a cached null count across ArraySpan.SetSlice by @singhpratech in <a href="https://github.com/apache/arrow-go/pull/1306">#1306</a></li>
<li>fix(arrow/compute): allow casting nullable struct fields to non-nullable when no nulls by @sankalpsthakur in <a href="https://github.com/apache/arrow-go/pull/1363">#1363</a></li>
<li>fix(arrow/compute): fix NewDatumWithoutOwning ownership semantics by @Asthenia0412 in <a href="https://github.com/apache/arrow-go/pull/1377">#1377</a></li>
<li>fix(arrow/compute): respect GOMAXPROCS for default NumParallel by @samueleresca in <a href="https://github.com/apache/arrow-go/pull/1384">#1384</a></li>
<li>fix(arrow/compute): do not return a result from a cancelled call by @zeroshade in <a href="https://github.com/apache/arrow-go/pull/1410">#1410</a></li>
<li>fix(arrow/compute): correct amd64 kernel assembly ABI by @zeroshade in <a href="https://github.com/apache/arrow-go/pull/1432">#1432</a></li>
<li>fix(arrow/compute): honor the input offset in is_not_null by @hcrosse in <a href="https://github.com/apache/arrow-go/pull/1456">#1456</a></li>
<li>fix(arrow/compute): register is_null, is_not_null and is_nan as unary functions by @hcrosse in <a href="https://github.com/apache/arrow-go/pull/1457">#1457</a></li>
<li>fix(arrow/compute): honor the input offset in not by @hcrosse in <a href="https://github.com/apache/arrow-go/pull/1458">#1458</a></li>
<li>fix(arrow/compute): treat null dictionary values as null in is_null and is_not_null by @hcrosse in <a href="https://github.com/apache/arrow-go/pull/1461">#1461</a></li>
<li>fix(arrow/compute): handle null, union and run-end encoded arrays in is_null and is_not_null by @hcrosse in <a href="https://github.com/apache/arrow-go/pull/1465">#1465</a></li>
<li>fix(arrow/compute): honor NullOptions.NanIsNull in is_null by @hcrosse in <a href="https://github.com/apache/arrow-go/pull/1488">#1488</a></li>
<li>fix(arrow/compute/exprs): accept string and binary dictionary columns in field references by @hcrosse in <a href="https://github.com/apache/arrow-go/pull/1449">#1449</a></li>
<li>fix(arrow/compute/exprs): fix offsets, nulls and ownership of nested field references by @zeroshade in <a href="https://github.com/apache/arrow-go/pull/1474">#1474</a></li>
<li>fix(arrow/compute): use int16 kernels for casts to int16 by @hcrosse in <a href="https://github.com/apache/arrow-go/pull/1503">#1503</a></li>
</ul>
<h4>Parquet</h4>
<ul>
<li>feat(parquet): add ALP encoding for FLOAT and DOUBLE columns by @prtkgaur in <a href="https://github.com/apache/arrow-go/pull/1289">#1289</a></li>
<li>feat(parquet/encryption): KMS support for reading PKMT1 key material by @EdwardShen125 in <a href="https://github.com/apache/arrow-go/pull/1478">#1478</a></li>
<li>feat(parquet/file): Write geospatial parquet row group statistics for GEOMETRY column chunks by @C-Loftus in <a href="https://github.com/apache/arrow-go/pull/1504">#1504</a></li>
<li>fix(parquet/file): write <code>null_count</code> statistics for GEOMETRY and GEOGRAPHY column chunks by @C-Loftus in <a href="https://github.com/apache/arrow-go/pull/1533">#1533</a></li>
<li>fix(parquet/utils): make unpack32 NEON profiler-safe by @zeroshade in <a href="https://github.com/apache/arrow-go/pull/1300">#1300</a></li>
<li>fix(parquet/metadata): stop bloom filter asm clobbering the frame pointer by @zeroshade in <a href="https://github.com/apache/arrow-go/pull/1379">#1379</a></li>
<li>fix(parquet/variant): prevent metadata offset cursor overflow by @matanper in <a href="https://github.com/apache/arrow-go/pull/1290">#1290</a></li>
<li>fix(parquet/variant): return &quot;&quot; for an empty long-form string by @CaptainAni187 in <a href="https://github.com/apache/arrow-go/pull/1353">#1353</a></li>
<li>fix(parquet/metadata): count non-null values from the validity bitmap in UpdateSpaced by @CaptainAni187 in <a href="https://github.com/apache/arrow-go/pull/1367">#1367</a></li>
<li>fix(parquet/pqarrow): honor offset and null slots when writing fixed-size lists by @zeroshade in <a href="https://github.com/apache/arrow-go/pull/1382">#1382</a></li>
<li>fix(parquet/file): read encrypted page headers written by parquet-mr by @sankalpsthakur in <a href="https://github.com/apache/arrow-go/pull/1390">#1390</a></li>
<li>fix(parquet/internal/encoding): do not leak delta bit width buffers from decoders by @zeroshade in <a href="https://github.com/apache/arrow-go/pull/1394">#1394</a></li>
<li>fix(parquet/internal/encoding): reset delta bit pack encoder state after flush by @zeroshade in <a href="https://github.com/apache/arrow-go/pull/1400">#1400</a></li>
<li>fix(parquet): return an error instead of panicking on a hidden encrypted column by @ump45nose in <a href="https://github.com/apache/arrow-go/pull/1406">#1406</a></li>
<li>fix(parquet): preserve CTR algorithm in plaintext footers by @autoCAT6 in <a href="https://github.com/apache/arrow-go/pull/1412">#1412</a></li>
<li>fix(parquet/pqarrow): release partial readers when building a field reader fails by @elsteveogrande in <a href="https://github.com/apache/arrow-go/pull/1435">#1435</a></li>
<li>fix(parquet): release writer buffers when a column chunk flush fails by @twuebi in <a href="https://github.com/apache/arrow-go/pull/1506">#1506</a></li>
<li>fix(parquet/metadata): count empty strings in byte array statistics from Arrow by @hcrosse in <a href="https://github.com/apache/arrow-go/pull/1527">#1527</a></li>
<li>fix(parquet/compress): read to the end of a Brotli stream that fills dst by @hcrosse in <a href="https://github.com/apache/arrow-go/pull/1530">#1530</a></li>
<li>fix(parquet/internal/encoding): clear buffered RLE booleans after each flush by @hcrosse in <a href="https://github.com/apache/arrow-go/pull/1539">#1539</a></li>
<li>fix(parquet/file): count V2 page num_nulls from the definition levels by @hcrosse in <a href="https://github.com/apache/arrow-go/pull/1543">#1543</a></li>
<li>fix(parquet/internal/encoding): clear buffered RLE booleans on Reset by @hcrosse in <a href="https://github.com/apache/arrow-go/pull/1545">#1545</a></li>
<li>fix(parquet/internal/encoding): clear encoder state on Reset by @hcrosse in <a href="https://github.com/apache/arrow-go/pull/1546">#1546</a></li>
<li>fix(parquet/file): return an error for dictionary encoded boolean columns by @hcrosse in <a href="https://github.com/apache/arrow-go/pull/1547">#1547</a></li>
</ul>
<h4>Performance</h4>
<ul>
<li>perf(arrow/array): compare optional validity bitmaps in bulk by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1327">#1327</a></li>
<li>perf(arrow/array): compare approximate list values by valid runs by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1330">#1330</a></li>
<li>perf(arrow/array): compare fixed-size binary values in bulk by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1347">#1347</a></li>
<li>perf(arrow/array): copy view buffers directly by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1324">#1324</a></li>
<li>perf(arrow/array): concatenate list offsets directly by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1323">#1323</a></li>
<li>perf(arrow/array): write bulk binary offsets and payloads directly by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1413">#1413</a></li>
<li>perf(arrow/array): reuse unchanged dictionaries across builder arrays by @hcrosse in <a href="https://github.com/apache/arrow-go/pull/1444">#1444</a></li>
<li>perf(arrow/array): reuse dictionary mappings and handle empty chunks in Concatenate by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1403">#1403</a></li>
<li>perf(compute): vectorize mixed 64-bit filters by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1311">#1311</a></li>
<li>perf(compute): vectorize mixed 16-bit filters by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1349">#1349</a></li>
<li>perf(compute): vectorize mixed 8-bit filters by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1350">#1350</a></li>
<li>perf(compute): add AVX2 take-index compaction by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1348">#1348</a></li>
<li>perf(compute): reuse amd64 SIMD comparisons for numeric-to-bool casts by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1351">#1351</a></li>
<li>perf(arrow/compute): read flat take values directly by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1360">#1360</a></li>
<li>perf(arrow/compute): avoid getter wrappers in chunked binary take by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1359">#1359</a></li>
<li>perf(compute): radix-sort null-free integer indices by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1391">#1391</a></li>
<li>perf(arrow/compute): unify changing dictionaries incrementally by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1395">#1395</a></li>
<li>perf(compute): use typed memo insertion for binary hashes by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1315">#1315</a></li>
<li>perf(bitutil): pack boolean bitmap writes by byte by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1402">#1402</a></li>
<li>perf(bitutil): accelerate misaligned bitmap equality by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1414">#1414</a></li>
<li>perf(bitutil): bulk-transfer unaligned bitmap ranges by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1416">#1416</a></li>
<li>perf(bitutil): process unaligned bitmap operations by word by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1437">#1437</a></li>
<li>perf(arrow/csv): stream full-file CSV reads into builders by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1393">#1393</a></li>
<li>perf: zstd implement Pooling + DecodeAll with no new reader. by @samueleresca in <a href="https://github.com/apache/arrow-go/pull/1331">#1331</a></li>
<li>perf(arrow/ipc): reuse zstd encoders across writers by @vdpoora in <a href="https://github.com/apache/arrow-go/pull/1429">#1429</a></li>
<li>perf(parquet/metadata): pool bloom filter hash scratch buffers by @zeroshade in <a href="https://github.com/apache/arrow-go/pull/1378">#1378</a></li>
<li>perf(parquet): use typed memo insertion for fixed-width dictionaries by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1312">#1312</a></li>
<li>perf(parquet): encode dictionary indices in RLE batches by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1328">#1328</a></li>
<li>perf(parquet): reuse DELTA_BYTE_ARRAY discard storage by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1321">#1321</a></li>
<li>perf(parquet): reuse DELTA decoder scratch by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1322">#1322</a></li>
<li>perf(parquet): speed up delta byte array prefix matching by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1388">#1388</a></li>
<li>perf(parquet): fast-path constant delta bitpacked blocks by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1396">#1396</a></li>
<li>perf(parquet): batch DELTA_LENGTH byte-array decoding and discard by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1415">#1415</a></li>
<li>perf(parquet): decode narrow packed levels directly by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1404">#1404</a></li>
<li>perf(parquet): expand nullable boolean bitmaps by byte by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1405">#1405</a></li>
<li>perf(parquet): decode RLE booleans directly into bool slices by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1417">#1417</a></li>
<li>perf(parquet): pack narrow RLE literal levels eight at a time by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1419">#1419</a></li>
<li>perf(parquet/file): use dense decoding for batches without nulls by @hcrosse in <a href="https://github.com/apache/arrow-go/pull/1447">#1447</a></li>
<li>perf(parquet/pqarrow): build struct child readers serially when Parallel is false by @elsteveogrande in <a href="https://github.com/apache/arrow-go/pull/1434">#1434</a></li>
<li>perf(parquet/internal/encoding): preallocate dictionaries bounded by page data by @zeroshade in <a href="https://github.com/apache/arrow-go/pull/1473">#1473</a></li>
</ul>
<h3>New Contributors</h3>
<ul>
<li>@matanper made their first contribution in <a href="https://github.com/apache/arrow-go/pull/1290">#1290</a></li>
<li>@singhpratech made their first contribution in <a href="https://github.com/apache/arrow-go/pull/1303">#1303</a></li>
<li>@jakthom made their first contribution in <a href="https://github.com/apache/arrow-go/pull/1308">#1308</a></li>
<li>@CaptainAni187 made their first contribution in <a href="https://github.com/apache/arrow-go/pull/1353">#1353</a></li>
<li>@samueleresca made their first contribution in <a href="https://github.com/apache/arrow-go/pull/1331">#1331</a></li>
<li>@hpaliwal8 made their first contribution in <a href="https://github.com/apache/arrow-go/pull/1365">#1365</a></li>
<li>@sankalpsthakur made their first contribution in <a href="https://github.com/apache/arrow-go/pull/1363">#1363</a></li>
<li>@Asthenia0412 made their first contribution in <a href="https://github.com/apache/arrow-go/pull/1376">#1376</a></li>
<li>@skyon0522-ai made their first contribution in <a href="https://github.com/apache/arrow-go/pull/1385">#1385</a></li>
<li>@nirajan1111 made their first contribution in <a href="https://github.com/apache/arrow-go/pull/1420">#1420</a></li>
<li>@autoCAT6 made their first contribution in <a href="https://github.com/apache/arrow-go/pull/1412">#1412</a></li>
<li>@vdpoora made their first contribution in <a href="https://github.com/apache/arrow-go/pull/1429">#1429</a></li>
<li>@elsteveogrande made their first contribution in <a href="https://github.com/apache/arrow-go/pull/1435">#1435</a></li>
<li>@EdwardShen125 made their first contribution in <a href="https://github.com/apache/arrow-go/pull/1439">#1439</a></li>
<li>@AynSiddiqui made their first contribution in <a href="https://github.com/apache/arrow-go/pull/1477">#1477</a></li>
<li>@peterolkhov made their first contribution in <a href="https://github.com/apache/arrow-go/pull/1499">#1499</a></li>
<li>@prtkgaur made their first contribution in <a href="https://github.com/apache/arrow-go/pull/1289">#1289</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a href="https://github.com/apache/arrow-go/compare/v18.8.0...v18.9.0">https://github.com/apache/arrow-go/compare/v18.8.0...v18.9.0</a></p>]]></content><author><name>pmc</name></author><category term="release" /><summary type="html"><![CDATA[The Apache Arrow team is pleased to announce the v18.9.0 release of Apache Arrow Go. This minor release covers 152 commits from 27 distinct contributors. Contributors $ git shortlog -sn v18.8.0..v18.9.0 45 Minh Vu 37 Harrison Crosse 31 Matt Topol 6 singhpratech 3 Sankalp Thakur 2 Animesh Kumar 2 Colton Loftus 2 Edward Shen 2 Samuele Resca 2 Steve O'Brien 2 Stochastic 2 Willem Jan 2 autoCAT 2 matanper 1 Arnaud 1 Arnold Wakim 1 Ayaan 1 Jake Thomas 1 Kyon 1 Madan Kumar 1 Nirajan Sah 1 Ondřej Pavela 1 Tobias Pütz 1 hp 1 peterolkhov 1 prtkgaur 1 ump45nose Highlights Go 1.26 is now the minimum supported Go version #1326 Arrow Features Added a scalar aggregate framework to the compute package, starting with the count and sum kernels #1336 Added the starts_with compute function, including string_view/binary_view inputs, IgnoreCase, and the Substrait starts_with mapping used by iceberg-go prefix row filters #1448, #1467, #1468 Added SortKey.FloatTotalOrder to sort floats by IEEE 754 totalOrder, distinguishing -0/+0 and signed NaNs #1430 Added Decimal32, Decimal64, StringView and BinaryView scalar types #1501, #1490 Added ipc.WithZstdLevel to choose the zstd compression level of IPC writers #1376 Added Reader.ContinueFrom to resume an IPC reader on a stream that arrives as several blobs, a pattern that stopped working in v18.8.0 #1346 The Flight SQL database/sql driver now accepts grpc+tcp://, grpc+tls:// and grpc+unix:// DSNs #1420 The JSON and CSV readers expose their input offset to help locate decoding errors #1342 Bug Fixes is_null, is_not_null and not returned wrong masks for sliced arrays, dictionaries with null values, and null, union and run-end encoded arrays, so filters could silently keep or drop the wrong rows. is_null now also honors NullOptions.NanIsNull #1456, #1458, #1461, #1465, #1488 NewDatumWithoutOwning now returns a non-owning Datum; previously Release() freed the caller's memory #1377 and_kleene and or_kleene returned false instead of null when ExecCtx.ChunkSize was smaller than the input #1306 A cancelled CallFunction no longer returns a result with a nil error, and the default NumParallel follows the cgroup-aware GOMAXPROCS in containers #1410, #1384 Fixed generated assembly that clobbered the frame pointer or Go's reserved registers, breaking profilers and frame-pointer unwinders, in the amd64 compute kernels and the Parquet bit-unpacking and Bloom filter kernels #1432, #1300, #1379 Decimal rescaling no longer panics or returns wrong values, infinite decimal strings are rejected, and scales past MaxScale fail clearly #1307, #1409, #1513, #1514, #1515 The C Data interface exports extension arrays as their storage and imports zero-length fixed-size lists, such as those returned by DataFusion #1526, #1308 arrow.TypeEqual now compares LargeListType like ListType, so array.Concatenate no longer rejects identical large list types #1440 NewMappedFileReader no longer panics on dictionary fields, and Flight SQL ExecuteIngest sends the schema for empty inputs #1365, #1433 The JSON record reader no longer swallows an unexpected EOF on truncated input #1341 Performance Improvements Performance remains a major theme: 51 of the merged pull requests are dedicated optimizations. Unless noted otherwise, the figures below are the PR-reported benchmarks on an Apple M1 Pro. Bulk array comparisons: comparing an array without a validity bitmap against one with a materialized bitmap is up to 414x faster for booleans (8.8ms to 21.3us over 1M values) and 15.3x for int64, and all-valid list equality over 65,536 rows drops from 17.8ms to 0.197ms #1327, #1330, #1347 SIMD filtering now covers 8-, 16- and 64-bit types: 1M-element alternating int64 filters go from 3.97ms to 0.45ms (8.7x), int16 from 4.23ms to 225us, and 8-bit alternating masks run 22-24x faster. AVX2 take-index compaction adds up to 4.5x on AMD EPYC #1311, #1349, #1350, #1348 SortIndices radix-sorts single-chunk, null-free integer columns: 65,536 random int64 values sort in 1.31ms instead of 12.3ms #1391 Dictionary-heavy workloads: unique over 256 chunks with changing dictionaries is 36.6x faster (60.0ms to 1.64ms) with 98.5% fewer bytes allocated, concatenating chunks that share a dictionary drops from 82.4us to 3.7us, and builders reuse unchanged dictionaries, cutting the per-row cost of small-batch dictionary reads through pqarrow by up to 93% #1395, #1403, #1444 Bitmap utilities: AppendBools takes 88% less time (geometric mean), unaligned bitmap operations are up to 11.4x faster, and misaligned bitmap equality and unaligned transfers improve 6.5x and 5.4x (geometric means) #1402, #1437, #1414, #1416 IPC zstd compression: pooled decoders cut decompression time by 69-78% for buffers up to 256KiB per column, and pooled encoders take a writer-per-record workload from 246.4us to 4.5us and from about 2.2MiB to 7.1KiB allocated per op (Apple M2 Pro) #1331, #1429 Full-file CSV reads stream into builders: 1.69x faster with 71% fewer bytes allocated #1393 Parquet Features Added opt-in ALP (Adaptive Lossless floating-Point) encoding for FLOAT and DOUBLE columns, interoperable with Arrow C++. Writers enable it with WithALPEncoding; reading needs no option #1289 Added an optional parquet/encryption package with KMS support, so files encrypted by pyarrow's CryptoFactory (PKMT1 key material, single or double wrapping) can be read through KmsKeyRetriever #1478 Write geospatial statistics (bounding box and geometry types) for GEOMETRY column chunks, and null_count statistics for GEOMETRY and GEOGRAPHY #1504, #1533 Bug Fixes Encryption interoperability: read encrypted files written by parquet-mr (Spark, Hive, Iceberg on parquet-java), preserve AES_GCM_CTR_V1 in plaintext footers, and return an error instead of panicking on columns without a key #1390, #1412, #1406 pqarrow wrote wrong data for sliced FixedSizeList columns and for FixedSizeList columns spanning more than one row group #1382 Encoders no longer carry stale state across pages: RLE boolean columns repeated earlier values on later pages, plain boolean and delta encoders wrote pre-Reset state, and all-null DELTA_BINARY_PACKED pages were padded with zeros #1539, #1545, #1546, #1400 Statistics: V2 data pages no longer report zero nulls without page statistics, byte array statistics written from Arrow count empty strings, and UpdateSpaced counts non-null values from the validity bitmap #1543, #1527, #1367 Fixed a memory leak in DELTA decoders, and dictionary-encoded boolean columns now return an error instead of panicking #1394, #1547 Performance Improvements Reading dictionary-encoded columns allocates the dictionary up front again, fixing an allocation regression introduced in v18.8.0 #1473 Bloom filter hashing pools its scratch buffers: hashing 64 values drops from 5.52us to 311ns, and dictionary Bloom filter writes take up to 32% less time (Intel Core Ultra 7 155H) #1378 Level and boolean decoding: nullable boolean bitmap expansion takes 85% less time (geometric mean), 1-bit packed levels decode in 55-60% less time, all-true and all-false RLE booleans decode 66-98% faster (amd64), and narrow RLE literal levels encode 2.25-2.61x faster #1405, #1404, #1417, #1419 Dictionary encoding: indices are encoded in RLE batches (65,536 constant indices drop from 187.7us to 26.0us), and FLBA and Int96 dictionary insertion is 29-33% faster with up to 50% fewer allocations #1328, #1312 DELTA encodings: constant DELTA_BINARY_PACKED blocks decode 2.1x faster, DELTA_LENGTH_BYTE_ARRAY decodes 1.62-2.00x and discards 2.43-2.74x faster, DELTA_BYTE_ARRAY prefix matching is up to 4.7x faster (AMD EPYC), and its discard path drops from 65,535 allocations to none #1396, #1415, #1388, #1321 Reads: optional columns use dense decoding for batches without nulls (up to 45% faster per row on AMD EPYC), and serial reads build struct child readers without goroutines, 37-43% faster (Apple M4 Max) #1447, #1434 Changelog Selected Changes A selection of the user-facing changes in this release. See the full changelog link below for every change in the release, including automated dependency updates. Arrow feat(arrow/ipc): add Reader.ContinueFrom for streams split across blobs by @zeroshade in #1346 feat(arrow/ipc): add WithZstdLevel option for IPC writer by @Asthenia0412 in #1376 feat(arrow/flight/flightsql/driver): accept grpc+tcp, grpc+tls and grpc+unix schemes by @nirajan1111 in #1420 fix: expose internal offsets in json/csv readers by @Willem-J-an in #1342 feat(arrow/scalar): add StringView and BinaryView scalars by @hcrosse in #1490 feat(arrow/scalar): add Decimal32 and Decimal64 scalars by @hcrosse in #1501 fix(arrow): support zero-length fixed-size lists by @jakthom in #1308 fix(array/record): bldr stop ignore unexpected EOF by @Willem-J-an in #1341 fix(arrow/ipc): open mapped files with dictionary fields by @hpaliwal8 in #1365 fix(arrow): handle dense union slices and empty validation by @skyon0522-ai in #1385 fix(arrow/array): validate bufferless run-end encoded arrays by @zeroshade in #1421 fix(arrow/flight/flightsql): send schema for empty ingest by @arnoldwakim in #1433 fix(arrow): compare LargeListType like ListType in TypeEqual by @paveon in #1440 fix(arrow/bitutil): preserve overlapping bitmap transfers by @fallintoplace in #1389 fix(arrow/decimal): widen Decimal64 rescale to 64 bits by @winklemad in #1307 fix(arrow/decimal): handle native word widths in big integer conversions by @zeroshade in #1409 fix(arrow/decimal): correct rescaling results and report cast errors by @hcrosse in #1513 fix(arrow/decimal): reject infinite decimal strings and bound extreme exponents by @hcrosse in #1514 fix(arrow/decimal): fail clearly for Decimal128 and Decimal256 scales past MaxScale by @hcrosse in #1515 fix(arrow/cdata): export extension arrays as their storage by @hcrosse in #1526 fix(arrow/cdata): export an empty view sizes buffer as the zero region by @hcrosse in #1544 fix(arrow/array): validate extension arrays by their storage type by @hcrosse in #1487 fix(arrow/array): keep the storage dictionary in NewExtensionArrayWithStorage by @hcrosse in #1540 fix(arrow/csv): preserve braces in list values by @peterolkhov in #1499 fix(arrow/csv): preserve lists holding a single empty string by @zeroshade in #1505 fix(arrow/csv): keep list items equal to the null value distinct from nulls by @zeroshade in #1508 Compute feat(arrow/compute): scalar aggregate framework with the count and sum kernels by @singhpratech in #1336 feat(arrow/compute): add IEEE 754 total-order float sorting option by @zeroshade in #1430 feat(arrow/compute): add starts_with kernel by @hcrosse in #1448 feat(arrow/compute): support string_view and binary_view in starts_with by @hcrosse in #1467 feat(arrow/compute): support IgnoreCase in starts_with by @hcrosse in #1468 fix(compute): detect temporal rounding overflow by @fallintoplace in #1127 fix(compute/exec): do not carry a cached null count across ArraySpan.SetSlice by @singhpratech in #1306 fix(arrow/compute): allow casting nullable struct fields to non-nullable when no nulls by @sankalpsthakur in #1363 fix(arrow/compute): fix NewDatumWithoutOwning ownership semantics by @Asthenia0412 in #1377 fix(arrow/compute): respect GOMAXPROCS for default NumParallel by @samueleresca in #1384 fix(arrow/compute): do not return a result from a cancelled call by @zeroshade in #1410 fix(arrow/compute): correct amd64 kernel assembly ABI by @zeroshade in #1432 fix(arrow/compute): honor the input offset in is_not_null by @hcrosse in #1456 fix(arrow/compute): register is_null, is_not_null and is_nan as unary functions by @hcrosse in #1457 fix(arrow/compute): honor the input offset in not by @hcrosse in #1458 fix(arrow/compute): treat null dictionary values as null in is_null and is_not_null by @hcrosse in #1461 fix(arrow/compute): handle null, union and run-end encoded arrays in is_null and is_not_null by @hcrosse in #1465 fix(arrow/compute): honor NullOptions.NanIsNull in is_null by @hcrosse in #1488 fix(arrow/compute/exprs): accept string and binary dictionary columns in field references by @hcrosse in #1449 fix(arrow/compute/exprs): fix offsets, nulls and ownership of nested field references by @zeroshade in #1474 fix(arrow/compute): use int16 kernels for casts to int16 by @hcrosse in #1503 Parquet feat(parquet): add ALP encoding for FLOAT and DOUBLE columns by @prtkgaur in #1289 feat(parquet/encryption): KMS support for reading PKMT1 key material by @EdwardShen125 in #1478 feat(parquet/file): Write geospatial parquet row group statistics for GEOMETRY column chunks by @C-Loftus in #1504 fix(parquet/file): write null_count statistics for GEOMETRY and GEOGRAPHY column chunks by @C-Loftus in #1533 fix(parquet/utils): make unpack32 NEON profiler-safe by @zeroshade in #1300 fix(parquet/metadata): stop bloom filter asm clobbering the frame pointer by @zeroshade in #1379 fix(parquet/variant): prevent metadata offset cursor overflow by @matanper in #1290 fix(parquet/variant): return &quot;&quot; for an empty long-form string by @CaptainAni187 in #1353 fix(parquet/metadata): count non-null values from the validity bitmap in UpdateSpaced by @CaptainAni187 in #1367 fix(parquet/pqarrow): honor offset and null slots when writing fixed-size lists by @zeroshade in #1382 fix(parquet/file): read encrypted page headers written by parquet-mr by @sankalpsthakur in #1390 fix(parquet/internal/encoding): do not leak delta bit width buffers from decoders by @zeroshade in #1394 fix(parquet/internal/encoding): reset delta bit pack encoder state after flush by @zeroshade in #1400 fix(parquet): return an error instead of panicking on a hidden encrypted column by @ump45nose in #1406 fix(parquet): preserve CTR algorithm in plaintext footers by @autoCAT6 in #1412 fix(parquet/pqarrow): release partial readers when building a field reader fails by @elsteveogrande in #1435 fix(parquet): release writer buffers when a column chunk flush fails by @twuebi in #1506 fix(parquet/metadata): count empty strings in byte array statistics from Arrow by @hcrosse in #1527 fix(parquet/compress): read to the end of a Brotli stream that fills dst by @hcrosse in #1530 fix(parquet/internal/encoding): clear buffered RLE booleans after each flush by @hcrosse in #1539 fix(parquet/file): count V2 page num_nulls from the definition levels by @hcrosse in #1543 fix(parquet/internal/encoding): clear buffered RLE booleans on Reset by @hcrosse in #1545 fix(parquet/internal/encoding): clear encoder state on Reset by @hcrosse in #1546 fix(parquet/file): return an error for dictionary encoded boolean columns by @hcrosse in #1547 Performance perf(arrow/array): compare optional validity bitmaps in bulk by @fallintoplace in #1327 perf(arrow/array): compare approximate list values by valid runs by @fallintoplace in #1330 perf(arrow/array): compare fixed-size binary values in bulk by @fallintoplace in #1347 perf(arrow/array): copy view buffers directly by @fallintoplace in #1324 perf(arrow/array): concatenate list offsets directly by @fallintoplace in #1323 perf(arrow/array): write bulk binary offsets and payloads directly by @fallintoplace in #1413 perf(arrow/array): reuse unchanged dictionaries across builder arrays by @hcrosse in #1444 perf(arrow/array): reuse dictionary mappings and handle empty chunks in Concatenate by @fallintoplace in #1403 perf(compute): vectorize mixed 64-bit filters by @fallintoplace in #1311 perf(compute): vectorize mixed 16-bit filters by @fallintoplace in #1349 perf(compute): vectorize mixed 8-bit filters by @fallintoplace in #1350 perf(compute): add AVX2 take-index compaction by @fallintoplace in #1348 perf(compute): reuse amd64 SIMD comparisons for numeric-to-bool casts by @fallintoplace in #1351 perf(arrow/compute): read flat take values directly by @fallintoplace in #1360 perf(arrow/compute): avoid getter wrappers in chunked binary take by @fallintoplace in #1359 perf(compute): radix-sort null-free integer indices by @fallintoplace in #1391 perf(arrow/compute): unify changing dictionaries incrementally by @fallintoplace in #1395 perf(compute): use typed memo insertion for binary hashes by @fallintoplace in #1315 perf(bitutil): pack boolean bitmap writes by byte by @fallintoplace in #1402 perf(bitutil): accelerate misaligned bitmap equality by @fallintoplace in #1414 perf(bitutil): bulk-transfer unaligned bitmap ranges by @fallintoplace in #1416 perf(bitutil): process unaligned bitmap operations by word by @fallintoplace in #1437 perf(arrow/csv): stream full-file CSV reads into builders by @fallintoplace in #1393 perf: zstd implement Pooling + DecodeAll with no new reader. by @samueleresca in #1331 perf(arrow/ipc): reuse zstd encoders across writers by @vdpoora in #1429 perf(parquet/metadata): pool bloom filter hash scratch buffers by @zeroshade in #1378 perf(parquet): use typed memo insertion for fixed-width dictionaries by @fallintoplace in #1312 perf(parquet): encode dictionary indices in RLE batches by @fallintoplace in #1328 perf(parquet): reuse DELTA_BYTE_ARRAY discard storage by @fallintoplace in #1321 perf(parquet): reuse DELTA decoder scratch by @fallintoplace in #1322 perf(parquet): speed up delta byte array prefix matching by @fallintoplace in #1388 perf(parquet): fast-path constant delta bitpacked blocks by @fallintoplace in #1396 perf(parquet): batch DELTA_LENGTH byte-array decoding and discard by @fallintoplace in #1415 perf(parquet): decode narrow packed levels directly by @fallintoplace in #1404 perf(parquet): expand nullable boolean bitmaps by byte by @fallintoplace in #1405 perf(parquet): decode RLE booleans directly into bool slices by @fallintoplace in #1417 perf(parquet): pack narrow RLE literal levels eight at a time by @fallintoplace in #1419 perf(parquet/file): use dense decoding for batches without nulls by @hcrosse in #1447 perf(parquet/pqarrow): build struct child readers serially when Parallel is false by @elsteveogrande in #1434 perf(parquet/internal/encoding): preallocate dictionaries bounded by page data by @zeroshade in #1473 New Contributors @matanper made their first contribution in #1290 @singhpratech made their first contribution in #1303 @jakthom made their first contribution in #1308 @CaptainAni187 made their first contribution in #1353 @samueleresca made their first contribution in #1331 @hpaliwal8 made their first contribution in #1365 @sankalpsthakur made their first contribution in #1363 @Asthenia0412 made their first contribution in #1376 @skyon0522-ai made their first contribution in #1385 @nirajan1111 made their first contribution in #1420 @autoCAT6 made their first contribution in #1412 @vdpoora made their first contribution in #1429 @elsteveogrande made their first contribution in #1435 @EdwardShen125 made their first contribution in #1439 @AynSiddiqui made their first contribution in #1477 @peterolkhov made their first contribution in #1499 @prtkgaur made their first contribution in #1289 Full Changelog: https://github.com/apache/arrow-go/compare/v18.8.0...v18.9.0]]></summary><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://arrow.apache.org/img/arrow-logo_horizontal_black-txt_white-bg.png" /><media:content medium="image" url="https://arrow.apache.org/img/arrow-logo_horizontal_black-txt_white-bg.png" xmlns:media="http://search.yahoo.com/mrss/" /></entry><entry><title type="html">Apache Arrow 26.0.1 Release</title><link href="https://arrow.apache.org/blog/2026/11/03/26.0.1-release/" rel="alternate" type="text/html" title="Apache Arrow 26.0.1 Release" /><published>2026-11-03T00:00:00-05:00</published><updated>2026-11-03T00:00:00-05:00</updated><id>https://arrow.apache.org/blog/2026/11/03/26.0.1-release</id><content type="html" xml:base="https://arrow.apache.org/blog/2026/11/03/26.0.1-release/"><![CDATA[<!--

-->
<p>The Apache Arrow team is pleased to announce the 26.0.1 release.
The 26.0.1 release is mostly a security bugfix release that includes <a href="https://github.com/apache/arrow/milestone/78?closed=1"><strong>20 resolved issues</strong></a>
on <a href="/release/26.0.1.html#contributors"><strong>24 distinct commits</strong></a> from <a href="/release/26.0.1.html#contributors"><strong>14 distinct contributors</strong></a>.</p>
<p>See the <a href="https://arrow.apache.org/install/">Install Page</a> to
learn how to get the libraries for your platform.</p>
<p>The release notes below are not exhaustive and only expose selected highlights
of the release. Many other bugfixes and improvements have been made: we refer
you to the <a href="/release/26.0.1.html#changelog">complete changelog</a>.</p>
<p>This release includes <strong>a security fix</strong> where we have bumped the bundled version for
Brotli which was affected by <a href="https://www.cve.org/CVERecord?id=CVE-2026-111497">CVE-2026-111497</a>.
The Brotli CVE affected PyArrow wheels and R binaries.</p>
<h2>C++ notes</h2>
<ul>
<li>Fix parquet treating unknown compression codec as uncompressed
(<a href="https://github.com/apache/arrow/issues/53518">GH-53518</a>).</li>
<li>Fix possibly reading out of bounds a MapArray on invalid IPC stream/file
(<a href="https://github.com/apache/arrow/issues/54936">GH-54936</a>).</li>
<li>Fix out of bounds read in <code>round(int, precision)</code> for the most negative precision on Gandiva
(<a href="https://github.com/apache/arrow/issues/50544">GH-50544</a>).</li>
<li>Fix Content Defined Chunking resetting its state on every row group
(<a href="https://github.com/apache/arrow/issues/51684">GH-51684</a>).</li>
</ul>
<h2>Python notes</h2>
<ul>
<li>Bump Brotli version on PyArrow (<a href="https://github.com/apache/arrow/issues/55165">GH-55165</a>)
to avoid bundling version with <a href="https://www.cve.org/CVERecord?id=CVE-2026-111497">CVE-2026-111497</a>.</li>
<li>Fix <code>pyarrow.compute.indices_nonzero</code> segfaulting on a ChunkedArray with zero chunks
(<a href="https://github.com/apache/arrow/issues/51018">GH-51018</a>).</li>
</ul>
<h2>R notes</h2>
<ul>
<li>Fix CRAN build for macOS where <a href="https://github.com/simdjson/simdjson">simdjson</a> was failing
to build with the macOS 11 SDK (<a href="https://github.com/apache/arrow/issues/55267">GH-55267</a>).</li>
<li>Bump Brotli version on R builds (<a href="https://github.com/apache/arrow/issues/55165">GH-55165</a>)
to avoid bundling version with <a href="https://www.cve.org/CVERecord?id=CVE-2026-111497">CVE-2026-111497</a>.</li>
</ul>
<h2>Linux packaging notes</h2>
<ul>
<li>Add missing opentelemetry dependency to libarrow2600 on Debian packages (<a href="https://github.com/apache/arrow/issues/55112">GH-55112</a>).</li>
</ul>
<h2>Other modules and languages</h2>
<p>No general changes were made to the other libraries or languages.</p>]]></content><author><name>pmc</name></author><category term="release" /><category term="security" /><summary type="html"><![CDATA[The Apache Arrow team is pleased to announce the 26.0.1 release. The 26.0.1 release is mostly a security bugfix release that includes 20 resolved issues on 24 distinct commits from 14 distinct contributors. See the Install Page to learn how to get the libraries for your platform. The release notes below are not exhaustive and only expose selected highlights of the release. Many other bugfixes and improvements have been made: we refer you to the complete changelog. This release includes a security fix where we have bumped the bundled version for Brotli which was affected by CVE-2026-111497. The Brotli CVE affected PyArrow wheels and R binaries. C++ notes Fix parquet treating unknown compression codec as uncompressed (GH-53518). Fix possibly reading out of bounds a MapArray on invalid IPC stream/file (GH-54936). Fix out of bounds read in round(int, precision) for the most negative precision on Gandiva (GH-50544). Fix Content Defined Chunking resetting its state on every row group (GH-51684). Python notes Bump Brotli version on PyArrow (GH-55165) to avoid bundling version with CVE-2026-111497. Fix pyarrow.compute.indices_nonzero segfaulting on a ChunkedArray with zero chunks (GH-51018). R notes Fix CRAN build for macOS where simdjson was failing to build with the macOS 11 SDK (GH-55267). Bump Brotli version on R builds (GH-55165) to avoid bundling version with CVE-2026-111497. Linux packaging notes Add missing opentelemetry dependency to libarrow2600 on Debian packages (GH-55112). Other modules and languages No general changes were made to the other libraries or languages.]]></summary><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://arrow.apache.org/img/arrow-logo_horizontal_black-txt_white-bg.png" /><media:content medium="image" url="https://arrow.apache.org/img/arrow-logo_horizontal_black-txt_white-bg.png" xmlns:media="http://search.yahoo.com/mrss/" /></entry><entry><title type="html">Apache Arrow ADBC 25 (Libraries) Release</title><link href="https://arrow.apache.org/blog/2026/10/26/adbc-25-release/" rel="alternate" type="text/html" title="Apache Arrow ADBC 25 (Libraries) Release" /><published>2026-10-26T00:00:00-04:00</published><updated>2026-10-26T00:00:00-04:00</updated><id>https://arrow.apache.org/blog/2026/10/26/adbc-25-release</id><content type="html" xml:base="https://arrow.apache.org/blog/2026/10/26/adbc-25-release/"><![CDATA[<!--

-->
<p>The Apache Arrow team is pleased to announce the version 25 release of
the Apache Arrow ADBC libraries. This release includes <a href="https://github.com/apache/arrow-adbc/issues?q=is%3Aissue%20state%3Aclosed%20milestone%3A%22ADBC%20Libraries%2025%22"><strong>56
resolved issues</strong></a> and <a href="https://github.com/apache/arrow-adbc/pulls?q=is%3Apr%20state%3Aclosed%20milestone%3A%22ADBC%20Libraries%2025%22%20-author%3Aapp%2Fdependabot"><strong>142 merged pull requests</strong></a> from
<a href="#contributors"><strong>25 distinct contributors</strong></a>.</p>
<p>This is a release of the <strong>libraries</strong>, which are at version 25.  The
<a href="https://arrow.apache.org/adbc/25/format/specification.html"><strong>API specification</strong></a> is versioned separately and is at
version 1.1.0.</p>
<p>The subcomponents are versioned independently:</p>
<ul>
<li>C/C++/GLib/Go/Python/Ruby: 1.13.0</li>
<li>C#: 1.0.0 (core), 0.25.0 (drivers)</li>
<li>Java: 0.25.0</li>
<li>JavaScript: 0.25.0</li>
<li>R: 0.25.0</li>
<li>Rust: 0.25.0</li>
</ul>
<p>The release notes below are not exhaustive and only expose selected
highlights of the release. Many other bugfixes and improvements have
been made: we refer you to the <a href="https://github.com/apache/arrow-adbc/blob/apache-arrow-adbc-25/CHANGELOG.md">complete changelog</a>.</p>
<h2>Release Highlights</h2>
<p>We now have a <a href="https://arrow.apache.org/adbc/current/visual_identity.html">logo</a> for ADBC (<a href="https://github.com/apache/arrow-adbc/pull/4870">#4870</a>).</p>
<p>We now provide some support for Windows-on-ARM, including Node.js packages, Python wheels, and Java JNI JARs.</p>
<p>The C++ libraries now require C++20 (<a href="https://github.com/apache/arrow-adbc/pull/4610">#4610</a>).</p>
<p>The C# core libraries are now version 1.0 and support ADBC 1.1 APIs (<a href="https://github.com/apache/arrow-adbc/pull/4923">#4923</a> and many more). The ADO.NET client now supports canonical extension types like UUID and Variant (<a href="https://github.com/apache/arrow-adbc/pull/4919">#4919</a>).</p>
<p>The Go libraries have been split: the core libraries and the drivers are now separate Go modules. This was done to stop driver-specific dependencies (and their associated CVEs) from leaking into <em>all</em> Go projects using ADBC, but is a breaking change for those who were directly depending on Go drivers from a Go project (<a href="https://github.com/apache/arrow-adbc/pull/4754">#4754</a>).</p>
<p>We now distribute Python wheels for Python 3.15 (<a href="https://github.com/apache/arrow-adbc/pull/4879">#4879</a>). Python 3.10 went EOL before release, so it was dropped (<a href="https://github.com/apache/arrow-adbc/pull/4859">#4859</a>).</p>
<p>The Rust libraries made several changes focused on reducing unsafe/undefined behavior, but note that certain low-level (but public) APIs had breaking changes as a result, and that exported drivers now require <code>Send</code> (<a href="https://github.com/apache/arrow-adbc/pull/4876">#4876</a>, <a href="https://github.com/apache/arrow-adbc/pull/4875">#4875</a>, <a href="https://github.com/apache/arrow-adbc/pull/4874">#4874</a>, <a href="https://github.com/apache/arrow-adbc/pull/4873">#4873</a>, <a href="https://github.com/apache/arrow-adbc/pull/4872">#4872</a>, <a href="https://github.com/apache/arrow-adbc/pull/4871">#4871</a>). Also, the existing <code>cancel</code> API has been deprecated as it was never usable in a safe manner; instead a new approach based on a cancellation handle has been added (<a href="https://github.com/apache/arrow-adbc/pull/3905">#3905</a>).</p>
<p>The Flight SQL driver now supports the new <code>is_update</code> field if reported by the server (<a href="https://github.com/apache/arrow-adbc/pull/4161">#4161</a>).</p>
<p>Several fixes were made to the PostgreSQL driver; temporary tables are now shown in GetObjects, and ingesting TIME types is now supported (<a href="https://github.com/apache/arrow-adbc/pull/4850">#4850</a>, <a href="https://github.com/apache/arrow-adbc/pull/4848">#4848</a>, <a href="https://github.com/apache/arrow-adbc/pull/4824">#4824</a>, <a href="https://github.com/apache/arrow-adbc/pull/4819">#4819</a>, <a href="https://github.com/apache/arrow-adbc/pull/4798">#4798</a>, <a href="https://github.com/apache/arrow-adbc/pull/4710">#4710</a>). <code>POSTGRESQL:type</code> field metadata is now included in result set schema fields to help disambiguate the exact data type for consumers (<a href="https://github.com/apache/arrow-adbc/pull/4687">#4687</a>). The UUID canonical extension type is now supported for queries and bulk ingest (<a href="https://github.com/apache/arrow-adbc/pull/4612">#4612</a>). Additionally performance of decimal ingestion was significantly improved on Windows (<a href="https://github.com/apache/arrow-adbc/pull/4847">#4847</a>). (This was previously optimized for other platforms, but Windows did not benefit as the platform compiler/toolchain did not support int128.) We now test the driver against various PostgreSQL-compatible vendors: AlloyDB Omni, CedarDB, Citus, CockroachDB, CrateDB, Materialize, ParadeDB, TimescaleDB, and YugabyteDB (<a href="https://github.com/apache/arrow-adbc/pull/4696">#4696</a>). We now support using a fixed lookup table for mapping PostgreSQL OIDs to their appropriate parsers; what this means is compatibility with certain vendors that don't implement certain system tables (some of the aforementioned vendors require this new option to be enabled), and improved startup performance for databases that have a large number of tables defined, but at the cost of not being able to read/write user-defined types. We plan to continue improving this feature to eliminate the downsides (<a href="https://github.com/apache/arrow-adbc/pull/4775">#4775</a>).</p>
<p>Several fixes were made to the SQLite driver to bring it more on par with other drivers (<a href="https://github.com/apache/arrow-adbc/pull/4845">#4845</a>, <a href="https://github.com/apache/arrow-adbc/pull/4843">#4843</a>, <a href="https://github.com/apache/arrow-adbc/pull/4842">#4842</a>, <a href="https://github.com/apache/arrow-adbc/pull/4841">#4841</a>, <a href="https://github.com/apache/arrow-adbc/pull/4826">#4826</a>).</p>
<p><a id="contributors"></a></p>
<h2>Contributors</h2>
<pre><code>$ git shortlog --perl-regexp --author='^((?!dependabot\[bot\]).*)$' -sn apache-arrow-adbc-24..apache-arrow-adbc-25
    76	David Li
    16	Curt Hagenlocher
     8	Bruce Irschick
     7	Bryce Mecum
     7	Ian Cook
     6	Fredrik Fornwall
     3	Efe
     2	Dewey Dunnington
     2	Jeremy Tan
     2	eitsupi
     1	Artur Rakhmatulin
     1	Hyukjin Kwon
     1	Jeroen Ooms
     1	Josh Ferguson
     1	Kevin Ushey
     1	Mandukhai Alimaa
     1	Pedro Matias
     1	Philip Moore
     1	Rusty Conover
     1	Sérgio Silveira
     1	Unik Dahal
     1	Yifan Chen
     1	grantframework0
     1	singhpratech
     1	İrem Çağın Yurttürk
</code></pre>
<h2>Roadmap</h2>
<p>We are working on the next revision of the API standard, focusing on missing
features (primarily metadata/catalog data). We welcome anyone interested in
contributing. Current progress can be found in the <a href="https://github.com/apache/arrow-adbc/milestone/9">1.2.0 specification
milestone</a>.</p>
<h2>Getting Involved</h2>
<p>We welcome questions and contributions from all interested.  Issues
can be filed on <a href="https://github.com/apache/arrow-adbc/issues">GitHub</a>, and questions can be directed to GitHub
or the <a href="/community/">Arrow mailing lists</a>.</p>]]></content><author><name>pmc</name></author><category term="release" /><summary type="html"><![CDATA[The Apache Arrow team is pleased to announce the version 25 release of the Apache Arrow ADBC libraries. This release includes 56 resolved issues and 142 merged pull requests from 25 distinct contributors. This is a release of the libraries, which are at version 25. The API specification is versioned separately and is at version 1.1.0. The subcomponents are versioned independently: C/C++/GLib/Go/Python/Ruby: 1.13.0 C#: 1.0.0 (core), 0.25.0 (drivers) Java: 0.25.0 JavaScript: 0.25.0 R: 0.25.0 Rust: 0.25.0 The release notes below are not exhaustive and only expose selected highlights of the release. Many other bugfixes and improvements have been made: we refer you to the complete changelog. Release Highlights We now have a logo for ADBC (#4870). We now provide some support for Windows-on-ARM, including Node.js packages, Python wheels, and Java JNI JARs. The C++ libraries now require C++20 (#4610). The C# core libraries are now version 1.0 and support ADBC 1.1 APIs (#4923 and many more). The ADO.NET client now supports canonical extension types like UUID and Variant (#4919). The Go libraries have been split: the core libraries and the drivers are now separate Go modules. This was done to stop driver-specific dependencies (and their associated CVEs) from leaking into all Go projects using ADBC, but is a breaking change for those who were directly depending on Go drivers from a Go project (#4754). We now distribute Python wheels for Python 3.15 (#4879). Python 3.10 went EOL before release, so it was dropped (#4859). The Rust libraries made several changes focused on reducing unsafe/undefined behavior, but note that certain low-level (but public) APIs had breaking changes as a result, and that exported drivers now require Send (#4876, #4875, #4874, #4873, #4872, #4871). Also, the existing cancel API has been deprecated as it was never usable in a safe manner; instead a new approach based on a cancellation handle has been added (#3905). The Flight SQL driver now supports the new is_update field if reported by the server (#4161). Several fixes were made to the PostgreSQL driver; temporary tables are now shown in GetObjects, and ingesting TIME types is now supported (#4850, #4848, #4824, #4819, #4798, #4710). POSTGRESQL:type field metadata is now included in result set schema fields to help disambiguate the exact data type for consumers (#4687). The UUID canonical extension type is now supported for queries and bulk ingest (#4612). Additionally performance of decimal ingestion was significantly improved on Windows (#4847). (This was previously optimized for other platforms, but Windows did not benefit as the platform compiler/toolchain did not support int128.) We now test the driver against various PostgreSQL-compatible vendors: AlloyDB Omni, CedarDB, Citus, CockroachDB, CrateDB, Materialize, ParadeDB, TimescaleDB, and YugabyteDB (#4696). We now support using a fixed lookup table for mapping PostgreSQL OIDs to their appropriate parsers; what this means is compatibility with certain vendors that don't implement certain system tables (some of the aforementioned vendors require this new option to be enabled), and improved startup performance for databases that have a large number of tables defined, but at the cost of not being able to read/write user-defined types. We plan to continue improving this feature to eliminate the downsides (#4775). Several fixes were made to the SQLite driver to bring it more on par with other drivers (#4845, #4843, #4842, #4841, #4826). Contributors $ git shortlog --perl-regexp --author='^((?!dependabot\[bot\]).*)$' -sn apache-arrow-adbc-24..apache-arrow-adbc-25 76 David Li 16 Curt Hagenlocher 8 Bruce Irschick 7 Bryce Mecum 7 Ian Cook 6 Fredrik Fornwall 3 Efe 2 Dewey Dunnington 2 Jeremy Tan 2 eitsupi 1 Artur Rakhmatulin 1 Hyukjin Kwon 1 Jeroen Ooms 1 Josh Ferguson 1 Kevin Ushey 1 Mandukhai Alimaa 1 Pedro Matias 1 Philip Moore 1 Rusty Conover 1 Sérgio Silveira 1 Unik Dahal 1 Yifan Chen 1 grantframework0 1 singhpratech 1 İrem Çağın Yurttürk Roadmap We are working on the next revision of the API standard, focusing on missing features (primarily metadata/catalog data). We welcome anyone interested in contributing. Current progress can be found in the 1.2.0 specification milestone. Getting Involved We welcome questions and contributions from all interested. Issues can be filed on GitHub, and questions can be directed to GitHub or the Arrow mailing lists.]]></summary><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://arrow.apache.org/img/arrow-logo_horizontal_black-txt_white-bg.png" /><media:content medium="image" url="https://arrow.apache.org/img/arrow-logo_horizontal_black-txt_white-bg.png" xmlns:media="http://search.yahoo.com/mrss/" /></entry><entry><title type="html">Apache Arrow 26.0.0 Release</title><link href="https://arrow.apache.org/blog/2026/10/08/26.0.0-release/" rel="alternate" type="text/html" title="Apache Arrow 26.0.0 Release" /><published>2026-10-08T00:00:00-04:00</published><updated>2026-10-08T00:00:00-04:00</updated><id>https://arrow.apache.org/blog/2026/10/08/26.0.0-release</id><content type="html" xml:base="https://arrow.apache.org/blog/2026/10/08/26.0.0-release/"><![CDATA[<!--

-->
<p>The Apache Arrow team is pleased to announce the 26.0.0 release. This release
covers over 3 months of development work and includes <a href="https://github.com/apache/arrow/milestone/75?closed=1"><strong>337 resolved
issues</strong></a> on <a href="/release/26.0.0.html#contributors"><strong>403 distinct commits</strong></a> from <a href="/release/26.0.0.html#contributors"><strong>108 distinct
contributors</strong></a>. See the <a href="https://arrow.apache.org/install/">Install Page</a> to
learn how to get the libraries for your platform.</p>
<p>The release notes below are not exhaustive and only expose selected highlights
of the release. Many other bugfixes and improvements have been made: we refer
you to the <a href="/release/26.0.0.html#changelog">complete changelog</a>.</p>
<h2>Community</h2>
<p>Since the 25.0.0 release, Kosta Tarasov, Zehua Zou and Tadeja Kadunc have
been invited to become committers. Jeffrey Vo has been invited to join
the Project Management Committee (PMC).</p>
<p>Thanks for your contributions and participation in the project!</p>
<h2>Format Notes</h2>
<p>The Tensor and SparseTensor IPC messages have been deprecated in the
specification, and the corresponding C++ APIs for reading and writing them have
been deprecated as well (<a href="https://github.com/apache/arrow/issues/51100">GH-51100</a>).</p>
<h2>Arrow Flight RPC Notes</h2>
<p>Work on the Flight SQL ODBC driver continued. RPM packages for the driver are now
built (<a href="https://github.com/apache/arrow/issues/47877">GH-47877</a>).
On Windows, the driver is now statically linked (<a href="https://github.com/apache/arrow/issues/49538">GH-49538</a>).</p>
<h2>C++ Notes</h2>
<p>Most of the C++ JSON handling (integration tests, extension type metadata,
Parquet, Flight SQL ODBC...) has been migrated from RapidJSON to
<a href="https://github.com/simdjson/simdjson">simdjson</a> for parsing and to a new
<code>JsonWriter</code> for serialization (<a href="https://github.com/apache/arrow/issues/50566">GH-50566</a>, <a href="https://github.com/apache/arrow/issues/50567">GH-50567</a> and
follow-ups).</p>
<p>The Feather reader and writer V1 APIs have been deprecated in favour of the Arrow
IPC APIs (<a href="https://github.com/apache/arrow/issues/49231">GH-49231</a>), in line with the corresponding Python and R deprecations.</p>
<h3>CSV</h3>
<p>Two new parse options:
<code>pad_short_rows</code> fills in nulls for rows with missing trailing fields
(<a href="https://github.com/apache/arrow/issues/50925">GH-50925</a>), and <code>ignore_extra_columns</code> drops extra fields from rows that have
more columns than expected (<a href="https://github.com/apache/arrow/issues/50967">GH-50967</a>).</p>
<h3>Compute</h3>
<p>New <code>remainder</code> and <code>modulo</code> compute functions have been added (<a href="https://github.com/apache/arrow/issues/46901">GH-46901</a>).</p>
<p>The temporal rounding functions (<code>floor_temporal</code>, <code>ceil_temporal</code> and
<code>round_temporal</code>) now accept duration inputs (<a href="https://github.com/apache/arrow/issues/50395">GH-50395</a>).</p>
<p>Hash aggregations now support <code>string_view</code> and <code>binary_view</code> grouping keys
(<a href="https://github.com/apache/arrow/issues/50223">GH-50223</a>), and the <code>unique</code>, <code>value_counts</code> and <code>dictionary_encode</code> functions
now support <code>float16</code> inputs (<a href="https://github.com/apache/arrow/issues/50512">GH-50512</a>).</p>
<h3>File Systems</h3>
<p><strong>Breaking change:</strong> the S3 filesystem and its AWS SDK dependency have been moved
out of <code>libarrow</code> into a separate <code>libarrow_s3</code> library (<a href="https://github.com/apache/arrow/issues/50194">GH-50194</a>). This
reduces the size of <code>libarrow</code> for users who do not need S3. C++ users of the S3
filesystem now need to either link against <code>libarrow_s3</code> or load it at runtime
using <code>LoadFileSystemFactories</code>.</p>
<h3>Parquet</h3>
<h4>CVE-2026-92287: Parquet file-controlled KMS URL is forwarded to pluggable KmsClient that skips host validation</h4>
<p>A security issue was fixed where KMS URLs embedded in Parquet files were forwarded to a user-implemented KMS client. This is now disabled by default but can be optionally enabled, in which case users should ensure they validate the KMS URLs received (<a href="https://github.com/apache/arrow/issues/51349">GH-51349</a>).</p>
<p>See our <a href="https://lists.apache.org/thread/0t6vry0msbk9dpsd0xv395ov5b84hj57">separate announcement</a> for further detail.</p>
<p>The schema nesting depth is now limited when parsing the Thrift metadata of a
Parquet file, turning a stack overflow on pathologically nested schemas into a
regular error (<a href="https://github.com/apache/arrow/issues/51238">GH-51238</a>). The default limit (100) can be raised if needed.</p>
<p>Reading Parquet files with unrecognized combinations of logical and physical
types is now tolerated instead of failing (<a href="https://github.com/apache/arrow/issues/50910">GH-50910</a>).</p>
<p>A new low-level dense decoding API allows <code>FIXED_LEN_BYTE_ARRAY</code> values to be
decoded directly into caller-provided buffers
(<a href="https://github.com/apache/arrow/issues/50333">GH-50333</a>).</p>
<p>Reading only the plaintext columns of a partially encrypted Parquet file with a
plaintext footer no longer fails with a decryption error (<a href="https://github.com/apache/arrow/issues/51361">GH-51361</a>).
<a href="https://github.com/apache/arrow/pull/51350">GH-51349</a> disables reading KMS URLs from file key material by default (<code>read_kms_url=false</code>). Applications relying on this behavior should
explicitly configure the KMS URL, or opt in with <code>read_kms_url=true</code> only if their KMS implementation validates the supplied URL.
Level histograms written to the Parquet size statistics could be incorrect for
inputs spanning multiple chunks; this has been fixed (<a href="https://github.com/apache/arrow/issues/50778">GH-50778</a>).</p>
<p>APIs deprecated in version 23.0.0 and earlier have been removed (<a href="https://github.com/apache/arrow/issues/50750">GH-50750</a>).</p>
<h3>Miscellaneous C++ changes</h3>
<p><code>ArrayBuilder</code> subclasses for binary data now raise a <code>CapacityError</code> instead of
silently truncating values of 2 GiB or more (<a href="https://github.com/apache/arrow/issues/51156">GH-51156</a>).</p>
<p>Bitmap copy and inversion operations have been sped up, by up to 2x in some
cases (<a href="https://github.com/apache/arrow/issues/50784">GH-50784</a>).</p>
<p>The <code>Datum</code> class has a new method <code>ComputeLogicalNullCount</code> (<a href="https://github.com/apache/arrow/issues/50338">GH-50338</a>),
complementing the method added to <code>ChunkedArray</code> in the previous release.</p>
<h2>Python Notes</h2>
<h3>Compatibility notes</h3>
<ul>
<li>Python 3.15 is supported and wheels created <a href="https://github.com/apache/arrow/issues/48172">GH-48172</a>.</li>
<li>Python 3.10 support dropped <a href="https://github.com/apache/arrow/issues/50410">GH-50410</a>.</li>
<li>Support for pandas &lt; 2.2.2 dropped <a href="https://github.com/apache/arrow/issues/48808">GH-48808</a> and <a href="https://github.com/apache/arrow/issues/48473">GH-48473</a>.</li>
<li>We now require numpy 2.x <a href="https://github.com/apache/arrow/issues/48473">GH-48473</a>.</li>
</ul>
<h3>New features</h3>
<ul>
<li>Conversion from numpy <code> numpy.dtypes.StringDType</code> to Arrow <code>string</code> (also <code>large_string</code> or <code>string_view</code>) is added <a href="https://github.com/apache/arrow/issues/51652">GH-51652</a>.</li>
<li>DLPack implementation now provides export capability for Arrow Arrays, Tensors <a href="https://github.com/apache/arrow/issues/39295">GH-39295</a> and <code>FixedShapeTensorArray</code>/<code>FixedShapeTensorScalar</code> <a href="https://github.com/apache/arrow/issues/38868">GH-38868</a> for consumers implementing DLPack protocol. Also the protocol version has been updated to use version 1.3 <a href="https://github.com/apache/arrow/issues/41670">GH-41670</a>.</li>
<li><code>record_batch_reader_source</code> Acero node (<code>RecordBatchReaderSourceNodeOptions</code>) is exposed <a href="https://github.com/apache/arrow/issues/50801">GH-50801</a>.</li>
<li><code>search_sorted</code> kernel for all primitive types and run-end encoded arrays is available <a href="https://github.com/apache/arrow/issues/49677">GH-49677</a>.</li>
<li><code>RecordBatchFileReader.count_rows()</code> binding is added <a href="https://github.com/apache/arrow/issues/49305">GH-49305</a>.</li>
<li><code>List</code>, <code>ListView</code>, <code>LargeList</code>, <code>LargeListView</code>, <code>Map</code>, <code>FixedSizeList</code> and <code>Struct</code> values are supported in the <code>run_end_encode</code> and the <code>run_end_decode</code> paths <a href="https://github.com/apache/arrow/issues/44183">GH-44183</a>.</li>
<li><code>eol</code> and <code>null_string</code> csv <code>WriteOptions</code> have been exposed <a href="https://github.com/apache/arrow/issues/34577">GH-34577</a>.</li>
</ul>
<h3>Other improvements</h3>
<ul>
<li><code>.slice</code> methods accept Arrow integer scalars <a href="https://github.com/apache/arrow/issues/51145">GH-51145</a>.</li>
<li><code>arrow_to_pandas</code> <code>DateOffset</code> handling for nanoseconds/microseconds is simplified <a href="https://github.com/apache/arrow/issues/50464">GH-50464</a>.</li>
<li>First step towards abi3 wheels support for pyarrow has been merged <a href="https://github.com/apache/arrow/issues/50431">GH-50431</a>.</li>
<li><code>is_nested</code> now uses C++ type traits <a href="https://github.com/apache/arrow/issues/40282">GH-40282</a>.</li>
</ul>
<h3>Relevant bug fixes</h3>
<ul>
<li>Converting <code>arrow.uuid arrays</code> and <code>table</code> columns to pandas now returns <code>uuid.UUID</code> objects instead of <code>bytes</code> <a href="https://github.com/apache/arrow/issues/50312">GH-50312</a>.</li>
<li>Scalar arithmetic dunders now return <code>NotImplemented</code> instead of raising <code>TypeError</code> when the other operand is not a recognized Arrow/NumPy type <a href="https://github.com/apache/arrow/issues/49826">GH-49826</a>.</li>
<li>Raise instead of crashing on unopened <code>MemoryMappedFile().resize()</code> <a href="https://github.com/apache/arrow/issues/51229">GH-51229</a>.</li>
<li>Writer options are synced between <code>write_table</code> and <code>write_to_dataset</code>/<code>ParquetFileFormat.make_write_options(...)</code> <a href="https://github.com/apache/arrow/issues/51220">GH-51220</a>.</li>
<li><code>BufferReader.readinto()</code> segfaults on a read-only destination are fixed <a href="https://github.com/apache/arrow/issues/51044">GH-51044</a>.</li>
<li><code>FunctionOptions.deserialize()</code> segfaults on non-buffer arguments are fixed <a href="https://github.com/apache/arrow/issues/51041">GH-51041</a>.</li>
<li>Correct type of <code>ParquetLogicalType</code> is now returned for Geometry/Geography <a href="https://github.com/apache/arrow/issues/50849">GH-50849</a>.</li>
<li><code>pa.types.is_nested()</code> now returns <code>True</code> for run-end encoded types <a href="https://github.com/apache/arrow/issues/50847">GH-50847</a>.</li>
<li>The reference cycle between the C++ FlightServerBase and the Python object now breaks to avoid leaking the server <a href="https://github.com/apache/arrow/issues/50684">GH-50684</a>.</li>
<li>Reference leak in <code>ConvertToSequenceAndInferSize</code> when an iterator raises on array conversion is fixed <a href="https://github.com/apache/arrow/issues/50591">GH-50591</a>.</li>
<li><code>pa.Table.from_struct_array</code> failure on empty array is fixed <a href="https://github.com/apache/arrow/issues/48344">GH-48344</a>.</li>
<li>Invalid operator error message in <code>pyarrow.parquet</code> is fixed <a href="https://github.com/apache/arrow/issues/47369">GH-47369</a>.</li>
<li>Preserving leading zeros in <code>read_csv</code> is now better documented <a href="https://github.com/apache/arrow/issues/46853">GH-46853</a>.</li>
<li><code>write_dataset</code> with csv file format ignoring delimiter is fixed <a href="https://github.com/apache/arrow/issues/39961">GH-39961</a>.</li>
<li><code>pyarrow.array</code> silently rewriting dictionary types to use signed integers for indexes is fixed <a href="https://github.com/apache/arrow/issues/37476">GH-37476</a>.</li>
<li>Extension types with list-based storage cast segfault is fixed <a href="https://github.com/apache/arrow/issues/37004">GH-37004</a>.</li>
</ul>
<h2>R Notes</h2>
<ul>
<li>register_scalar_function() now errors if named fields in in_type do not match the corresponding
argument names of fun (after the initial context argument), instead of silently ignoring those names (<a href="https://github.com/apache/arrow/pull/51324">GH-37761</a>).</li>
</ul>
<h3>Breaking changes</h3>
<ul>
<li><code>read_feather()</code> and <code>write_feather()</code> are deprecated in favour of <code>read_ipc_file()</code> and <code>write_ipc_file()</code>, and <code>format = &quot;feather&quot;</code> in <code>open_dataset()</code> and <code>write_dataset()</code> is deprecated in favour of <code>format = &quot;ipc&quot;</code> (<a href="https://github.com/apache/arrow/issues/49237">GH-49237</a>).</li>
<li>The long-deprecated <code>.data.frame</code> argument to <code>map_batches()</code> has been removed, and the planned change to <code>pull()</code> returning a <code>ChunkedArray</code> by default has been dropped, so <code>pull()</code> no longer warns (<a href="https://github.com/apache/arrow/issues/51655">GH-51655</a>).</li>
</ul>
<h3>New features</h3>
<ul>
<li>Azure Blob Storage is now supported via the new <code>AzureFileSystem</code> class and <code>az_container()</code> helper, analogous to <code>S3FileSystem</code> and <code>s3_bucket()</code> (@marberts, <a href="https://github.com/apache/arrow/issues/32123">GH-32123</a>).</li>
</ul>
<h3>Minor improvements and fixes</h3>
<ul>
<li>A number of dplyr fixes: variables sharing a name with a function such as <code>date</code> now work in verbs (<a href="https://github.com/apache/arrow/issues/39688">GH-39688</a>), <code>if_else()</code> works with a bare <code>NA</code> branch alongside dates (<a href="https://github.com/apache/arrow/issues/38358">GH-38358</a>), and <code>summarise()</code> after <code>arrange()</code> no longer errors (<a href="https://github.com/apache/arrow/issues/45373">GH-45373</a>).</li>
<li>Factors and integer types inside list columns are now unified across the whole column when reading in multiple batches (<a href="https://github.com/apache/arrow/issues/50514">GH-50514</a>), and Parquet <code>float16</code> columns now read correctly (<a href="https://github.com/apache/arrow/issues/50378">GH-50378</a>).</li>
</ul>
<h2>Ruby and C GLib Notes</h2>
<ul>
<li>Flight RPC: Fixed GC related problems: <a href="https://github.com/apache/arrow/pull/50401">GH-50401</a></li>
<li>Apache Parquet: Added support for buffered reader properties: <a href="https://github.com/apache/arrow/pull/51277">GH-51277</a></li>
</ul>
<h3>Ruby</h3>
<ul>
<li>red-arrow-format: Added support for building Apache Arrow data from Ruby objects.</li>
<li>red-arrow-format: Added support for comparing arrays.</li>
<li>red-arrow-format: Added support for comparing types.</li>
<li>red-arrow-format: Added support for getting Ruby objects from Apache Arrow data.</li>
</ul>
<h3>C GLib</h3>
<p>No C GLib only notes.</p>
<h2>Java, JavaScript, Go, .NET, Swift and Rust Notes</h2>
<p>The Java, JavaScript, Go, .NET, Swift and Rust projects have moved to separate
repositories outside the main Arrow <a href="https://github.com/apache/arrow">monorepo</a>.</p>
<ul>
<li>For notes on the latest release of the <a href="https://github.com/apache/arrow-java">Java
implementation</a>, see the latest <a href="https://github.com/apache/arrow-java/releases">Arrow
Java changelog</a>.</li>
<li>For notes on the latest release of the <a href="https://github.com/apache/arrow-js">JavaScript
implementation</a>, see the latest <a href="https://github.com/apache/arrow-js/releases">Arrow
JavaScript changelog</a>.</li>
<li>For notes on the latest release of the <a href="https://github.com/apache/arrow-rs">Rust
implementation</a> see the latest <a href="https://github.com/apache/arrow-rs/blob/main/CHANGELOG.md">Arrow Rust
changelog</a>.</li>
<li>For notes on the latest release of the <a href="https://github.com/apache/arrow-go">Go
implementation</a>, see the latest <a href="https://github.com/apache/arrow-go/releases">Arrow Go
changelog</a>.</li>
<li>For notes on the latest release of the <a href="https://github.com/apache/arrow-dotnet">.NET
implementation</a>, see the latest <a href="https://github.com/apache/arrow-dotnet/releases">Arrow  .NET changelog</a>.</li>
<li>For notes on the latest release of the <a href="https://github.com/apache/arrow-swift">Swift implementation</a>, see the latest <a href="https://github.com/apache/arrow-swift/releases">Arrow Swift changelog</a>.</li>
</ul>]]></content><author><name>pmc</name></author><category term="release" /><summary type="html"><![CDATA[The Apache Arrow team is pleased to announce the 26.0.0 release. This release covers over 3 months of development work and includes 337 resolved issues on 403 distinct commits from 108 distinct contributors. See the Install Page to learn how to get the libraries for your platform. The release notes below are not exhaustive and only expose selected highlights of the release. Many other bugfixes and improvements have been made: we refer you to the complete changelog. Community Since the 25.0.0 release, Kosta Tarasov, Zehua Zou and Tadeja Kadunc have been invited to become committers. Jeffrey Vo has been invited to join the Project Management Committee (PMC). Thanks for your contributions and participation in the project! Format Notes The Tensor and SparseTensor IPC messages have been deprecated in the specification, and the corresponding C++ APIs for reading and writing them have been deprecated as well (GH-51100). Arrow Flight RPC Notes Work on the Flight SQL ODBC driver continued. RPM packages for the driver are now built (GH-47877). On Windows, the driver is now statically linked (GH-49538). C++ Notes Most of the C++ JSON handling (integration tests, extension type metadata, Parquet, Flight SQL ODBC...) has been migrated from RapidJSON to simdjson for parsing and to a new JsonWriter for serialization (GH-50566, GH-50567 and follow-ups). The Feather reader and writer V1 APIs have been deprecated in favour of the Arrow IPC APIs (GH-49231), in line with the corresponding Python and R deprecations. CSV Two new parse options: pad_short_rows fills in nulls for rows with missing trailing fields (GH-50925), and ignore_extra_columns drops extra fields from rows that have more columns than expected (GH-50967). Compute New remainder and modulo compute functions have been added (GH-46901). The temporal rounding functions (floor_temporal, ceil_temporal and round_temporal) now accept duration inputs (GH-50395). Hash aggregations now support string_view and binary_view grouping keys (GH-50223), and the unique, value_counts and dictionary_encode functions now support float16 inputs (GH-50512). File Systems Breaking change: the S3 filesystem and its AWS SDK dependency have been moved out of libarrow into a separate libarrow_s3 library (GH-50194). This reduces the size of libarrow for users who do not need S3. C++ users of the S3 filesystem now need to either link against libarrow_s3 or load it at runtime using LoadFileSystemFactories. Parquet CVE-2026-92287: Parquet file-controlled KMS URL is forwarded to pluggable KmsClient that skips host validation A security issue was fixed where KMS URLs embedded in Parquet files were forwarded to a user-implemented KMS client. This is now disabled by default but can be optionally enabled, in which case users should ensure they validate the KMS URLs received (GH-51349). See our separate announcement for further detail. The schema nesting depth is now limited when parsing the Thrift metadata of a Parquet file, turning a stack overflow on pathologically nested schemas into a regular error (GH-51238). The default limit (100) can be raised if needed. Reading Parquet files with unrecognized combinations of logical and physical types is now tolerated instead of failing (GH-50910). A new low-level dense decoding API allows FIXED_LEN_BYTE_ARRAY values to be decoded directly into caller-provided buffers (GH-50333). Reading only the plaintext columns of a partially encrypted Parquet file with a plaintext footer no longer fails with a decryption error (GH-51361). GH-51349 disables reading KMS URLs from file key material by default (read_kms_url=false). Applications relying on this behavior should explicitly configure the KMS URL, or opt in with read_kms_url=true only if their KMS implementation validates the supplied URL. Level histograms written to the Parquet size statistics could be incorrect for inputs spanning multiple chunks; this has been fixed (GH-50778). APIs deprecated in version 23.0.0 and earlier have been removed (GH-50750). Miscellaneous C++ changes ArrayBuilder subclasses for binary data now raise a CapacityError instead of silently truncating values of 2 GiB or more (GH-51156). Bitmap copy and inversion operations have been sped up, by up to 2x in some cases (GH-50784). The Datum class has a new method ComputeLogicalNullCount (GH-50338), complementing the method added to ChunkedArray in the previous release. Python Notes Compatibility notes Python 3.15 is supported and wheels created GH-48172. Python 3.10 support dropped GH-50410. Support for pandas &lt; 2.2.2 dropped GH-48808 and GH-48473. We now require numpy 2.x GH-48473. New features Conversion from numpy numpy.dtypes.StringDType to Arrow string (also large_string or string_view) is added GH-51652. DLPack implementation now provides export capability for Arrow Arrays, Tensors GH-39295 and FixedShapeTensorArray/FixedShapeTensorScalar GH-38868 for consumers implementing DLPack protocol. Also the protocol version has been updated to use version 1.3 GH-41670. record_batch_reader_source Acero node (RecordBatchReaderSourceNodeOptions) is exposed GH-50801. search_sorted kernel for all primitive types and run-end encoded arrays is available GH-49677. RecordBatchFileReader.count_rows() binding is added GH-49305. List, ListView, LargeList, LargeListView, Map, FixedSizeList and Struct values are supported in the run_end_encode and the run_end_decode paths GH-44183. eol and null_string csv WriteOptions have been exposed GH-34577. Other improvements .slice methods accept Arrow integer scalars GH-51145. arrow_to_pandas DateOffset handling for nanoseconds/microseconds is simplified GH-50464. First step towards abi3 wheels support for pyarrow has been merged GH-50431. is_nested now uses C++ type traits GH-40282. Relevant bug fixes Converting arrow.uuid arrays and table columns to pandas now returns uuid.UUID objects instead of bytes GH-50312. Scalar arithmetic dunders now return NotImplemented instead of raising TypeError when the other operand is not a recognized Arrow/NumPy type GH-49826. Raise instead of crashing on unopened MemoryMappedFile().resize() GH-51229. Writer options are synced between write_table and write_to_dataset/ParquetFileFormat.make_write_options(...) GH-51220. BufferReader.readinto() segfaults on a read-only destination are fixed GH-51044. FunctionOptions.deserialize() segfaults on non-buffer arguments are fixed GH-51041. Correct type of ParquetLogicalType is now returned for Geometry/Geography GH-50849. pa.types.is_nested() now returns True for run-end encoded types GH-50847. The reference cycle between the C++ FlightServerBase and the Python object now breaks to avoid leaking the server GH-50684. Reference leak in ConvertToSequenceAndInferSize when an iterator raises on array conversion is fixed GH-50591. pa.Table.from_struct_array failure on empty array is fixed GH-48344. Invalid operator error message in pyarrow.parquet is fixed GH-47369. Preserving leading zeros in read_csv is now better documented GH-46853. write_dataset with csv file format ignoring delimiter is fixed GH-39961. pyarrow.array silently rewriting dictionary types to use signed integers for indexes is fixed GH-37476. Extension types with list-based storage cast segfault is fixed GH-37004. R Notes register_scalar_function() now errors if named fields in in_type do not match the corresponding argument names of fun (after the initial context argument), instead of silently ignoring those names (GH-37761). Breaking changes read_feather() and write_feather() are deprecated in favour of read_ipc_file() and write_ipc_file(), and format = &quot;feather&quot; in open_dataset() and write_dataset() is deprecated in favour of format = &quot;ipc&quot; (GH-49237). The long-deprecated .data.frame argument to map_batches() has been removed, and the planned change to pull() returning a ChunkedArray by default has been dropped, so pull() no longer warns (GH-51655). New features Azure Blob Storage is now supported via the new AzureFileSystem class and az_container() helper, analogous to S3FileSystem and s3_bucket() (@marberts, GH-32123). Minor improvements and fixes A number of dplyr fixes: variables sharing a name with a function such as date now work in verbs (GH-39688), if_else() works with a bare NA branch alongside dates (GH-38358), and summarise() after arrange() no longer errors (GH-45373). Factors and integer types inside list columns are now unified across the whole column when reading in multiple batches (GH-50514), and Parquet float16 columns now read correctly (GH-50378). Ruby and C GLib Notes Flight RPC: Fixed GC related problems: GH-50401 Apache Parquet: Added support for buffered reader properties: GH-51277 Ruby red-arrow-format: Added support for building Apache Arrow data from Ruby objects. red-arrow-format: Added support for comparing arrays. red-arrow-format: Added support for comparing types. red-arrow-format: Added support for getting Ruby objects from Apache Arrow data. C GLib No C GLib only notes. Java, JavaScript, Go, .NET, Swift and Rust Notes The Java, JavaScript, Go, .NET, Swift and Rust projects have moved to separate repositories outside the main Arrow monorepo. For notes on the latest release of the Java implementation, see the latest Arrow Java changelog. For notes on the latest release of the JavaScript implementation, see the latest Arrow JavaScript changelog. For notes on the latest release of the Rust implementation see the latest Arrow Rust changelog. For notes on the latest release of the Go implementation, see the latest Arrow Go changelog. For notes on the latest release of the .NET implementation, see the latest Arrow .NET changelog. For notes on the latest release of the Swift implementation, see the latest Arrow Swift changelog.]]></summary><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://arrow.apache.org/img/arrow-logo_horizontal_black-txt_white-bg.png" /><media:content medium="image" url="https://arrow.apache.org/img/arrow-logo_horizontal_black-txt_white-bg.png" xmlns:media="http://search.yahoo.com/mrss/" /></entry><entry><title type="html">Apache Arrow Rust 60.0.0 Release</title><link href="https://arrow.apache.org/blog/2026/09/29/arrow-rs-60.0.0/" rel="alternate" type="text/html" title="Apache Arrow Rust 60.0.0 Release" /><published>2026-09-29T00:00:00-04:00</published><updated>2026-09-29T00:00:00-04:00</updated><id>https://arrow.apache.org/blog/2026/09/29/arrow-rs-60.0.0</id><content type="html" xml:base="https://arrow.apache.org/blog/2026/09/29/arrow-rs-60.0.0/"><![CDATA[<!--

-->
<p>The Apache Arrow team is pleased to announce that the v60.0.0 release of Apache Arrow
Rust is now available on crates.io (<a href="https://crates.io/crates/arrow">arrow</a> and <a href="https://crates.io/crates/parquet">parquet</a>) and as <a href="https://dist.apache.org/repos/dist/release/arrow/arrow-rs-60.0.0">source download</a>.</p>
<p>See the <a href="https://github.com/apache/arrow-rs/blob/60.0.0/CHANGELOG.md">60.0.0 changelog</a> for a full list of changes.</p>
<h2>Highlights</h2>
<p>This release includes new Parquet encoding support, a sustained effort to remove
panics from the codebase, a large number of kernel performance improvements, and
many bug fixes across decimals, take/filter, and FFI.</p>
<p>MSRV is bumped to 1.88,
<code>rand</code> upgraded to 0.10, <code>object_store</code> to 0.14.1, and several long-deprecated
items have been removed. See the <a href="https://github.com/apache/arrow-rs/blob/60.0.0/CHANGELOG.md">60.0.0 changelog</a> for the full breaking
changes list.</p>
<h2>New Features</h2>
<h3>Parquet ALP Encoder/Decoder Support</h3>
<p><a href="https://github.com/sdf-jkl">@sdf-jkl</a> added support for the <a href="https://parquet.apache.org/blog/2026/09/22/alp-adaptive-lossless-floating-point-encoding-in-apache-parquet/">Adaptive Lossless Floating-Point (ALP)</a>
encoding in Parquet (<a href="https://github.com/apache/arrow-rs/pull/9372">#9372</a>). ALP is a high-performance encoding for
floating-point columns that enables compact storage, fast decoding and single value random-access.
This makes arrow-rs the first Parquet implementations to support ALP,
keeping the <a href="https://crates.io/crates/parquet">parquet</a> crate aligned with the latest Parquet specification.</p>
<h3>Parquet Page Index Improvements</h3>
<p><a href="https://github.com/etseidl">@etseidl</a> made several improvements to Parquet page index handling: a new
<a href="https://docs.rs/parquet/60.0.0/parquet/file/metadata/page_index/struct.PageIndex.html"><code>PageIndex</code></a> struct encapsulating column and offset indexes (<a href="https://github.com/apache/arrow-rs/pull/10719">#10719</a>), new
<a href="https://docs.rs/parquet/60.0.0/parquet/file/metadata/page_index/struct.PageIndexBuilder.html"><code>PageIndexBuilder</code></a> and <a href="https://docs.rs/parquet/60.0.0/parquet/file/metadata/page_index/trait.PageIndexProvider.html"><code>PageIndexProvider</code></a> types (<a href="https://github.com/apache/arrow-rs/pull/10842">#10842</a>), and page index
decoders are now public (<a href="https://github.com/apache/arrow-rs/pull/10899">#10899</a>). One major use case enabled by these
changes is supplying external or cached <code>PageIndex</code>es directly to the reader
(for example, indexes previously fetched and stored outside the file),
avoiding redundant I/O on repeated scans. This also opens the door for
selective population of the page indexes which will help reduce the latency
of filtered scans against very wide tables.</p>
<h3>Parquet NDV Statistics, FILE Logical Type, and IEEE 754 Total Order</h3>
<p><a href="https://github.com/Rich-T-kid">@Rich-T-kid</a> implemented writing num distinct values (NDV) statistics in the
Parquet writer (<a href="https://github.com/apache/arrow-rs/pull/10654">#10654</a>) and added row-group-level distinct counts to
<code>StatisticsConverter</code> (<a href="https://github.com/apache/arrow-rs/pull/10652">#10652</a>). Query engines use these to make better
planning decisions such as join strategy selection and filter selectivity
estimates.</p>
<p><a href="https://github.com/brkyvz">@brkyvz</a> added support for the <code>FILE</code> logical type (<a href="https://github.com/apache/arrow-rs/pull/10109">#10109</a>). <a href="https://github.com/etseidl">@etseidl</a>
implemented <a href="https://issues.apache.org/jira/browse/PARQUET-2249">PARQUET-2249</a> IEEE 754 total order for floating-point comparisons
(<a href="https://github.com/apache/arrow-rs/pull/9619">#9619</a>) and the corresponding <code>INT96</code> timestamp <code>ColumnOrder</code> (<a href="https://github.com/apache/arrow-rs/pull/10106">#10106</a>).
<a href="https://github.com/etseidl">@etseidl</a> also removed the previous limit of 32,768 row groups per file
(<a href="https://github.com/apache/arrow-rs/pull/10149">#10149</a>).</p>
<h3>New <code>arrow-cmp</code> Crate</h3>
<p><a href="https://github.com/Rich-T-kid">@Rich-T-kid</a> introduced the <a href="https://crates.io/crates/arrow-cmp"><code>arrow-cmp</code></a> crate (<a href="https://github.com/apache/arrow-rs/pull/10325">#10325</a>), splitting
comparison kernel logic out of <a href="https://crates.io/crates/arrow-ord"><code>arrow-ord</code></a> into its own focused crate. This
gives users a more targeted dependency for Arrow comparison operations without
pulling in the broader <code>arrow-ord</code> crate.</p>
<h2>Panic Elimination</h2>
<p>A lot of work went into converting panics into proper <code>Result</code> returns this
release. <a href="https://github.com/emilk">@emilk</a> fixed <code>try_</code> functions still panicking in edge cases (<a href="https://github.com/apache/arrow-rs/pull/10730">#10730</a>,
<a href="https://github.com/apache/arrow-rs/pull/10755">#10755</a>) and removed panics from <code>GenericByteArray::from_iter_values</code> (<a href="https://github.com/apache/arrow-rs/pull/10729">#10729</a>).
<a href="https://github.com/bit2swaz">@bit2swaz</a> replaced panics with errors in IPC schema and footer parsing
(<a href="https://github.com/apache/arrow-rs/pull/10647">#10647</a>, <a href="https://github.com/apache/arrow-rs/pull/10744">#10744</a>) and closed an FFI <code>Drop</code> soundness hole (<a href="https://github.com/apache/arrow-rs/pull/10431">#10431</a>).
<a href="https://github.com/Rich-T-kid">@Rich-T-kid</a> fixed a potential use-after-free in <code>Buffer::shrink_to_fit</code> (<a href="https://github.com/apache/arrow-rs/pull/10932">#10932</a>)
and converted <code>MutableBuffer</code> panicking callsites to fallible methods (<a href="https://github.com/apache/arrow-rs/pull/10641">#10641</a>,
<a href="https://github.com/apache/arrow-rs/pull/10317">#10317</a>). <a href="https://github.com/okhsunrog">@okhsunrog</a> made <code>interleave</code>/<code>concat</code> return errors on dictionary key
overflow (<a href="https://github.com/apache/arrow-rs/pull/10675">#10675</a>). <a href="https://github.com/ranflarion">@ranflarion</a> converted a Parquet buffer push panic into an
error (<a href="https://github.com/apache/arrow-rs/pull/10564">#10564</a>) and <a href="https://github.com/dhruvxvaishnav">@dhruvxvaishnav</a> did the same for invalid dictionary index
bit widths (<a href="https://github.com/apache/arrow-rs/pull/10725">#10725</a>).</p>
<h2>Performance Improvements</h2>
<p>This release has a large number of performance improvements across kernels,
casting, and Parquet I/O.</p>
<h3>Take/Filter Kernel Improvements</h3>
<p><a href="https://github.com/Rich-T-kid">@Rich-T-kid</a> landed a series of take/filter kernel speedups this release:
<code>take</code> on <code>FixedSizeList</code> is up to <strong>3.3x faster</strong> (<a href="https://github.com/apache/arrow-rs/pull/10441">#10441</a>), <code>take</code> on
<code>List&lt;T&gt;</code> up to <strong>1.8x faster</strong> (<a href="https://github.com/apache/arrow-rs/pull/10812">#10812</a>), <code>take</code> on boolean and null buffers
up to <strong>1.7x faster</strong> (<a href="https://github.com/apache/arrow-rs/pull/10813">#10813</a>), and <code>filter</code> on <code>FixedSizeBinary</code> up to
<strong>1.4x faster</strong> (<a href="https://github.com/apache/arrow-rs/pull/10993">#10993</a>). A <code>NullBuffer::expand</code> rewrite for aligned counts
delivers up to <strong>143x faster</strong> buffer expansion (<a href="https://github.com/apache/arrow-rs/pull/10976">#10976</a>, <a href="https://github.com/apache/arrow-rs/pull/10980">#10980</a>).</p>
<p><a href="https://github.com/YUZHEthefool">@YUZHEthefool</a> improved <code>rank</code> on <code>StringViewArray</code> by up to <strong>3.2x faster</strong>
via prefix key caching (<a href="https://github.com/apache/arrow-rs/pull/10605">#10605</a>). <a href="https://github.com/giladkl">@giladkl</a> made equality/inequality
comparisons of byte-view arrays against short scalars up to <strong>2.5x faster</strong>
(<a href="https://github.com/apache/arrow-rs/pull/10689">#10689</a>). <a href="https://github.com/cakeni">@cakeni</a> replaced <code>BufferBuilder</code> with <code>Vec</code> across
<code>concat_elements</code>, <code>take_run</code>, <code>substring</code>, and other kernels
(<a href="https://github.com/apache/arrow-rs/pull/10629">#10629</a>-<a href="https://github.com/apache/arrow-rs/pull/10634">#10634</a>), and <a href="https://github.com/kowanietz">@kowanietz</a> did the same for primitive unary
operations (<a href="https://github.com/apache/arrow-rs/pull/10783">#10783</a>) and fixed-size binary take (<a href="https://github.com/apache/arrow-rs/pull/10773">#10773</a>), cutting
unnecessary allocations across the board. <a href="https://github.com/haohuaijin">@haohuaijin</a> fixed in-place
bitwise ops corrupting bits outside the requested range (<a href="https://github.com/apache/arrow-rs/pull/10444">#10444</a>).</p>
<h3>Decimal Casting</h3>
<p><a href="https://github.com/neilconway">@neilconway</a> made decimal parsing dramatically faster across the board. A
foundational rewrite of the decimal-from-string path (<a href="https://github.com/apache/arrow-rs/pull/10668">#10668</a>) alone delivers
<strong>up to ~8x faster</strong> casting from strings to <code>Decimal128</code> and up to <strong>92% lower parsing time</strong>
on individual parse microbenchmarks. A follow-up pass for separate
integer/fractional digit scanning (<a href="https://github.com/apache/arrow-rs/pull/10974">#10974</a>) added another <strong>5–15%</strong> on top,
and skipping redundant precision checks (<a href="https://github.com/apache/arrow-rs/pull/10998">#10998</a>) added up to <strong>~10%</strong> more.
Formatting <code>i256</code> without <code>num-bigint</code> (<a href="https://github.com/apache/arrow-rs/pull/11000">#11000</a>) rounds things out with
<strong>~1.8x faster</strong> <code>Decimal256</code> formatting for typical 38-digit values.</p>
<h3>Parquet Reading Improvements</h3>
<p>Plain Parquet string column reads into dictionary arrays are <strong>~1.5x faster</strong>
(<a href="https://github.com/apache/arrow-rs/pull/10614">#10614</a>). <a href="https://github.com/adriangb">@adriangb</a> sped up <code>DELTA_BYTE_ARRAY</code> shared-prefix decoding by
scanning a block at a time (<a href="https://github.com/apache/arrow-rs/pull/10549">#10549</a>), reaching <strong>up to 19x faster</strong> on the
prefix-scan microbenchmark and <strong>4.2x faster</strong> end-to-end for small
shared-prefix strings. <a href="https://github.com/haohuaijin">@haohuaijin</a> removed redundant copies in Parquet
mask-backed intersection/union (<a href="https://github.com/apache/arrow-rs/pull/10446">#10446</a>), delivering <strong>2–6x faster</strong> mask
operations depending on alignment. <a href="https://github.com/hhhizzz">@hhhizzz</a> fixed cached Mask reads crossing
unloaded sparse pages (<a href="https://github.com/apache/arrow-rs/pull/10735">#10735</a>).</p>
<h3>Binary Size Reduction</h3>
<p><a href="https://github.com/alamb">@alamb</a> reduced binary size by about <strong>2%</strong> by monomorphizing <code>PrimitiveArray</code>
to <code>ArrayData</code> conversion and <code>Debug</code> formatting helpers (<a href="https://github.com/apache/arrow-rs/pull/10893">#10893</a>, <a href="https://github.com/apache/arrow-rs/pull/10890">#10890</a>) —
the Debug change alone cut <strong>~12% of LLVM IR</strong> from the <code>cast_kernels</code> binary.</p>
<h2>Bug Fixes</h2>
<p><a href="https://github.com/neilconway">@neilconway</a> fixed a cluster of decimal bugs including incorrect <code>Decimal256</code>
casting to signed integers (<a href="https://github.com/apache/arrow-rs/pull/10857">#10857</a>), wrong min/max statistics for <code>BYTE_ARRAY</code>
decimals (<a href="https://github.com/apache/arrow-rs/pull/10861">#10861</a>), mismatched scales in <code>make_comparator</code> (<a href="https://github.com/apache/arrow-rs/pull/10864">#10864</a>),
formatting bugs (<a href="https://github.com/apache/arrow-rs/pull/10869">#10869</a>), and a spurious assert when <code>scale == precision</code>
(<a href="https://github.com/apache/arrow-rs/pull/10875">#10875</a>).</p>
<p><a href="https://github.com/yongster">@yongster</a> fixed <code>take</code> for null indices in dense union and run-end encoded
arrays (<a href="https://github.com/apache/arrow-rs/pull/10909">#10909</a>).</p>
<p><a href="https://github.com/1fanwang">@1fanwang</a> and <a href="https://github.com/tpoterba">@tpoterba</a> fixed <code>take</code> and <code>interleave</code> on zero-width
<code>FixedSizeListArray</code> (<a href="https://github.com/apache/arrow-rs/pull/10915">#10915</a>, <a href="https://github.com/apache/arrow-rs/pull/11026">#11026</a>).</p>
<p><a href="https://github.com/jaideeppyne">@jaideeppyne</a> fixed struct <code>ArrayData</code> slicing double-counting the offset
(<a href="https://github.com/apache/arrow-rs/pull/10835">#10835</a>, <a href="https://github.com/apache/arrow-rs/pull/10934">#10934</a>). <a href="https://github.com/linhongyu510">@linhongyu510</a> fixed struct null validation not
accounting for parent offsets (<a href="https://github.com/apache/arrow-rs/pull/10970">#10970</a>).</p>
<p><a href="https://github.com/PlenoraETL">@PlenoraETL</a> made the IPC reader return an error for a <code>DictionaryBatch</code>
missing its data instead of silently producing wrong results (<a href="https://github.com/apache/arrow-rs/pull/11020">#11020</a>).
<a href="https://github.com/bit2swaz">@bit2swaz</a> made <code>FFI_ArrowSchema::with_metadata</code> <code>unsafe</code> to close a
soundness gap (<a href="https://github.com/apache/arrow-rs/pull/10764">#10764</a>).</p>
<p><a href="https://github.com/adriangb">@adriangb</a> fixed <code>DELTA_BYTE_ARRAY</code> deduplication for values larger than
the page size limit (<a href="https://github.com/apache/arrow-rs/pull/10505">#10505</a>).</p>
<h2>Other Notable Changes</h2>
<ul>
<li><a href="https://github.com/adamreeve">@adamreeve</a> added round-trip support for <code>Dictionary(_, Utf8View/BinaryView)</code>
columns in Parquet (<a href="https://github.com/apache/arrow-rs/pull/10831">#10831</a>)</li>
<li><a href="https://github.com/ranflarion">@ranflarion</a> improved Parquet bloom filters: populate from the dictionary
while a column is dictionary encoded (<a href="https://github.com/apache/arrow-rs/pull/10966">#10966</a>) and added a writer option to
skip bloom filters for all-dictionary column chunks (<a href="https://github.com/apache/arrow-rs/pull/10963">#10963</a>)</li>
<li><a href="https://github.com/haohuaijin">@haohuaijin</a> added row-group-local <code>RowSelection</code> support to the push
decoder (<a href="https://github.com/apache/arrow-rs/pull/10702">#10702</a>)</li>
<li><a href="https://github.com/1fanwang">@1fanwang</a> added <code>Utf8View</code> and <code>BinaryView</code> support to <code>substring</code> (<a href="https://github.com/apache/arrow-rs/pull/10672">#10672</a>)</li>
<li><a href="https://github.com/etseidl">@etseidl</a> removed previously deprecated Parquet functions (<a href="https://github.com/apache/arrow-rs/pull/10565">#10565</a>) and
deprecated <code>ColumnOrder::sort_order_for_type</code> (<a href="https://github.com/apache/arrow-rs/pull/10104">#10104</a>) as a follow-up to
the INT96/IEEE 754 correctness work — both are breaking changes users will
notice at compile time.</li>
</ul>
<h2>Thanks to Our Contributors</h2>
<pre><code class="language-console">$ git shortlog -sn 59.2.0..60.0.0
    38  Richard Baah
    22  Jeffrey Vo
    19  Andrew Lamb
    17  Emil Ernerfeldt
    16  Neil Conway
    12  Adrian Garcia Badaracco
    11  WaterWhisperer
    11  cakeni
    10  Ed Seidl
     8  Kosta Tarasov
     6  Huaijin
     6  ranflarion
     5  Aditya Mishra
     5  yongster
     4  Stefan Wang
     3  Abhishek
     3  Dhruv Vaishnav
     3  Thefool
     3  WissssleyL
     2  Andrea Bozzo
     2  Bharadwaj Pendyala
     2  Emily Matheys
     2  Ethan Tang
     2  Giladi
     2  Jaideep Pyne
     2  Oleks V
     2  Peter L
     2  Peter Lee
     2  Xinyao Zhang
     2  Yiming Qiao
     2  hylin
     2  pawan
     1  AarryaSaraf
     1  Adam Reeve
     1  Adam Reichold
     1  Alexander Rafferty
     1  Ali Asghar
     1  Ben Kowanietz
     1  Burak Yavuz
     1  Chao Sun
     1  Congxian Qiu
     1  Connor Tsui
     1  Danila Gornushko
     1  Fredrik Fornwall
     1  Han You
     1  Haresh Khanna
     1  Harshal Joshi
     1  Huang Qiwei
     1  Johan Vaz
     1  Jonas Dedden
     1  Jordan Epstein
     1  Langning Zhang
     1  Liam Bao
     1  Marcelo Tesla
     1  Marco Bonamente
     1  Michal Piatkowski
     1  MsfPablo
     1  Narendran K T
     1  Nuno Faria
     1  Phoenix
     1  Qi Zhu
     1  Raghvendra Singh
     1  Raz Luvaton
     1  Seowoo Jang
     1  Thor
     1  Tim Poterba
     1  Yin Li
     1  anchor
     1  kowanietz
     1  linfeng
     1  subotac
     1  wterrr
     1  yanglongwei
     1  张跃哲
</code></pre>
<p>Thanks to everyone who contributed bug reports, reviews, and pull requests for
this release!</p>]]></content><author><name>Richard Baah &amp; pmc</name></author><category term="release" /><summary type="html"><![CDATA[The Apache Arrow team is pleased to announce that the v60.0.0 release of Apache Arrow Rust is now available on crates.io (arrow and parquet) and as source download. See the 60.0.0 changelog for a full list of changes. Highlights This release includes new Parquet encoding support, a sustained effort to remove panics from the codebase, a large number of kernel performance improvements, and many bug fixes across decimals, take/filter, and FFI. MSRV is bumped to 1.88, rand upgraded to 0.10, object_store to 0.14.1, and several long-deprecated items have been removed. See the 60.0.0 changelog for the full breaking changes list. New Features Parquet ALP Encoder/Decoder Support @sdf-jkl added support for the Adaptive Lossless Floating-Point (ALP) encoding in Parquet (#9372). ALP is a high-performance encoding for floating-point columns that enables compact storage, fast decoding and single value random-access. This makes arrow-rs the first Parquet implementations to support ALP, keeping the parquet crate aligned with the latest Parquet specification. Parquet Page Index Improvements @etseidl made several improvements to Parquet page index handling: a new PageIndex struct encapsulating column and offset indexes (#10719), new PageIndexBuilder and PageIndexProvider types (#10842), and page index decoders are now public (#10899). One major use case enabled by these changes is supplying external or cached PageIndexes directly to the reader (for example, indexes previously fetched and stored outside the file), avoiding redundant I/O on repeated scans. This also opens the door for selective population of the page indexes which will help reduce the latency of filtered scans against very wide tables. Parquet NDV Statistics, FILE Logical Type, and IEEE 754 Total Order @Rich-T-kid implemented writing num distinct values (NDV) statistics in the Parquet writer (#10654) and added row-group-level distinct counts to StatisticsConverter (#10652). Query engines use these to make better planning decisions such as join strategy selection and filter selectivity estimates. @brkyvz added support for the FILE logical type (#10109). @etseidl implemented PARQUET-2249 IEEE 754 total order for floating-point comparisons (#9619) and the corresponding INT96 timestamp ColumnOrder (#10106). @etseidl also removed the previous limit of 32,768 row groups per file (#10149). New arrow-cmp Crate @Rich-T-kid introduced the arrow-cmp crate (#10325), splitting comparison kernel logic out of arrow-ord into its own focused crate. This gives users a more targeted dependency for Arrow comparison operations without pulling in the broader arrow-ord crate. Panic Elimination A lot of work went into converting panics into proper Result returns this release. @emilk fixed try_ functions still panicking in edge cases (#10730, #10755) and removed panics from GenericByteArray::from_iter_values (#10729). @bit2swaz replaced panics with errors in IPC schema and footer parsing (#10647, #10744) and closed an FFI Drop soundness hole (#10431). @Rich-T-kid fixed a potential use-after-free in Buffer::shrink_to_fit (#10932) and converted MutableBuffer panicking callsites to fallible methods (#10641, #10317). @okhsunrog made interleave/concat return errors on dictionary key overflow (#10675). @ranflarion converted a Parquet buffer push panic into an error (#10564) and @dhruvxvaishnav did the same for invalid dictionary index bit widths (#10725). Performance Improvements This release has a large number of performance improvements across kernels, casting, and Parquet I/O. Take/Filter Kernel Improvements @Rich-T-kid landed a series of take/filter kernel speedups this release: take on FixedSizeList is up to 3.3x faster (#10441), take on List&lt;T&gt; up to 1.8x faster (#10812), take on boolean and null buffers up to 1.7x faster (#10813), and filter on FixedSizeBinary up to 1.4x faster (#10993). A NullBuffer::expand rewrite for aligned counts delivers up to 143x faster buffer expansion (#10976, #10980). @YUZHEthefool improved rank on StringViewArray by up to 3.2x faster via prefix key caching (#10605). @giladkl made equality/inequality comparisons of byte-view arrays against short scalars up to 2.5x faster (#10689). @cakeni replaced BufferBuilder with Vec across concat_elements, take_run, substring, and other kernels (#10629-#10634), and @kowanietz did the same for primitive unary operations (#10783) and fixed-size binary take (#10773), cutting unnecessary allocations across the board. @haohuaijin fixed in-place bitwise ops corrupting bits outside the requested range (#10444). Decimal Casting @neilconway made decimal parsing dramatically faster across the board. A foundational rewrite of the decimal-from-string path (#10668) alone delivers up to ~8x faster casting from strings to Decimal128 and up to 92% lower parsing time on individual parse microbenchmarks. A follow-up pass for separate integer/fractional digit scanning (#10974) added another 5–15% on top, and skipping redundant precision checks (#10998) added up to ~10% more. Formatting i256 without num-bigint (#11000) rounds things out with ~1.8x faster Decimal256 formatting for typical 38-digit values. Parquet Reading Improvements Plain Parquet string column reads into dictionary arrays are ~1.5x faster (#10614). @adriangb sped up DELTA_BYTE_ARRAY shared-prefix decoding by scanning a block at a time (#10549), reaching up to 19x faster on the prefix-scan microbenchmark and 4.2x faster end-to-end for small shared-prefix strings. @haohuaijin removed redundant copies in Parquet mask-backed intersection/union (#10446), delivering 2–6x faster mask operations depending on alignment. @hhhizzz fixed cached Mask reads crossing unloaded sparse pages (#10735). Binary Size Reduction @alamb reduced binary size by about 2% by monomorphizing PrimitiveArray to ArrayData conversion and Debug formatting helpers (#10893, #10890) — the Debug change alone cut ~12% of LLVM IR from the cast_kernels binary. Bug Fixes @neilconway fixed a cluster of decimal bugs including incorrect Decimal256 casting to signed integers (#10857), wrong min/max statistics for BYTE_ARRAY decimals (#10861), mismatched scales in make_comparator (#10864), formatting bugs (#10869), and a spurious assert when scale == precision (#10875). @yongster fixed take for null indices in dense union and run-end encoded arrays (#10909). @1fanwang and @tpoterba fixed take and interleave on zero-width FixedSizeListArray (#10915, #11026). @jaideeppyne fixed struct ArrayData slicing double-counting the offset (#10835, #10934). @linhongyu510 fixed struct null validation not accounting for parent offsets (#10970). @PlenoraETL made the IPC reader return an error for a DictionaryBatch missing its data instead of silently producing wrong results (#11020). @bit2swaz made FFI_ArrowSchema::with_metadata unsafe to close a soundness gap (#10764). @adriangb fixed DELTA_BYTE_ARRAY deduplication for values larger than the page size limit (#10505). Other Notable Changes @adamreeve added round-trip support for Dictionary(_, Utf8View/BinaryView) columns in Parquet (#10831) @ranflarion improved Parquet bloom filters: populate from the dictionary while a column is dictionary encoded (#10966) and added a writer option to skip bloom filters for all-dictionary column chunks (#10963) @haohuaijin added row-group-local RowSelection support to the push decoder (#10702) @1fanwang added Utf8View and BinaryView support to substring (#10672) @etseidl removed previously deprecated Parquet functions (#10565) and deprecated ColumnOrder::sort_order_for_type (#10104) as a follow-up to the INT96/IEEE 754 correctness work — both are breaking changes users will notice at compile time. Thanks to Our Contributors $ git shortlog -sn 59.2.0..60.0.0 38 Richard Baah 22 Jeffrey Vo 19 Andrew Lamb 17 Emil Ernerfeldt 16 Neil Conway 12 Adrian Garcia Badaracco 11 WaterWhisperer 11 cakeni 10 Ed Seidl 8 Kosta Tarasov 6 Huaijin 6 ranflarion 5 Aditya Mishra 5 yongster 4 Stefan Wang 3 Abhishek 3 Dhruv Vaishnav 3 Thefool 3 WissssleyL 2 Andrea Bozzo 2 Bharadwaj Pendyala 2 Emily Matheys 2 Ethan Tang 2 Giladi 2 Jaideep Pyne 2 Oleks V 2 Peter L 2 Peter Lee 2 Xinyao Zhang 2 Yiming Qiao 2 hylin 2 pawan 1 AarryaSaraf 1 Adam Reeve 1 Adam Reichold 1 Alexander Rafferty 1 Ali Asghar 1 Ben Kowanietz 1 Burak Yavuz 1 Chao Sun 1 Congxian Qiu 1 Connor Tsui 1 Danila Gornushko 1 Fredrik Fornwall 1 Han You 1 Haresh Khanna 1 Harshal Joshi 1 Huang Qiwei 1 Johan Vaz 1 Jonas Dedden 1 Jordan Epstein 1 Langning Zhang 1 Liam Bao 1 Marcelo Tesla 1 Marco Bonamente 1 Michal Piatkowski 1 MsfPablo 1 Narendran K T 1 Nuno Faria 1 Phoenix 1 Qi Zhu 1 Raghvendra Singh 1 Raz Luvaton 1 Seowoo Jang 1 Thor 1 Tim Poterba 1 Yin Li 1 anchor 1 kowanietz 1 linfeng 1 subotac 1 wterrr 1 yanglongwei 1 张跃哲 Thanks to everyone who contributed bug reports, reviews, and pull requests for this release!]]></summary><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://arrow.apache.org/img/arrow-logo_horizontal_black-txt_white-bg.png" /><media:content medium="image" url="https://arrow.apache.org/img/arrow-logo_horizontal_black-txt_white-bg.png" xmlns:media="http://search.yahoo.com/mrss/" /></entry><entry><title type="html">Apache Arrow Go 18.8.0 Release</title><link href="https://arrow.apache.org/blog/2026/09/08/arrow-go-18.8.0/" rel="alternate" type="text/html" title="Apache Arrow Go 18.8.0 Release" /><published>2026-09-08T00:00:00-04:00</published><updated>2026-09-08T00:00:00-04:00</updated><id>https://arrow.apache.org/blog/2026/09/08/arrow-go-18.8.0</id><content type="html" xml:base="https://arrow.apache.org/blog/2026/09/08/arrow-go-18.8.0/"><![CDATA[<!--

-->
<p>The Apache Arrow team is pleased to announce the v18.8.0 release of Apache Arrow Go.
This minor release covers 283 commits from 16 distinct contributors.</p>
<h2>Contributors</h2>
<pre><code class="language-console">$ git shortlog -sn v18.7.0..v18.8.0
   251	Minh Vu
    10	Matt Topol
     4	Colton Loftus
     4	Digvijay
     4	Tom Frank
     1	Derek Perkins
     1	Lucas Valente
     1	Madan kumar
     1	Neelesh Salian
     1	Nick Ripley
     1	Pedro Matias
     1	Raúl Cumplido
     1	Ruihao Chen
     1	Tobias Pütz
     1	Tomáš Karela Procházka
     1	Willem Jan
</code></pre>
<h2>Highlights</h2>
<h3>Arrow</h3>
<h4>Features</h4>
<ul>
<li>Added <code>VariantGet</code> for efficient path extraction from variant arrays <a href="https://github.com/apache/arrow-go/pull/1206">#1206</a></li>
<li>Added <code>list_element</code>, <code>cumulative_sum</code>, <code>cumulative_sum_checked</code>, and <code>dictionary_encode</code> compute functions <a href="https://github.com/apache/arrow-go/pull/1138">#1138</a>,
<a href="https://github.com/apache/arrow-go/pull/1139">#1139</a>, <a href="https://github.com/apache/arrow-go/pull/1140">#1140</a></li>
<li>Added <code>ValueAsAny</code> for accessing Arrow array values as native Go values <a href="https://github.com/apache/arrow-go/pull/1170">#1170</a></li>
<li>Strengthened IPC file validation for malformed block framing and lengths <a href="https://github.com/apache/arrow-go/pull/1097">#1097</a></li>
</ul>
<h4>Bug Fixes</h4>
<ul>
<li>Fixed a memory leak when calling <code>Resize(0)</code> on non-empty builders <a href="https://github.com/apache/arrow-go/pull/995">#995</a></li>
<li>Preserved exact precision when casting strings to Decimal128 and Decimal256 scalars <a href="https://github.com/apache/arrow-go/pull/1107">#1107</a></li>
</ul>
<h4>Performance Improvements</h4>
<p>This is a performance-heavy release: 32 of the merged pull requests are dedicated
optimizations. Unless noted otherwise, the figures below are the PR-reported
benchmarks on an Apple M1 Pro, comparing the new path against the scalar
baseline it replaces.</p>
<ul>
<li>Expanded SIMD acceleration for compute arithmetic, comparisons, and filtering on ARM64 and AMD64. Arithmetic kernels run 4.1-4.4x faster on int64 and float64
adds, int64 array/array comparisons drop from 313.6us to 29.3us per op (10.7x), narrow 8- and 16-bit integer comparisons reach 12.8-15.1x, and null-free
32-bit fixed-width filters compress up to 11.8x faster on 1M-element inputs <a href="https://github.com/apache/arrow-go/pull/1265">#1265</a>, <a href="https://github.com/apache/arrow-go/pull/1266">#1266</a>,
<a href="https://github.com/apache/arrow-go/pull/1284">#1284</a>, <a href="https://github.com/apache/arrow-go/pull/1286">#1286</a>, <a href="https://github.com/apache/arrow-go/pull/1287">#1287</a>, <a href="https://github.com/apache/arrow-go/pull/1288">#1288</a></li>
<li>Building take indices from fragmented selection vectors is no longer quadratic. For a 1M-element alternating filter it drops from 4.31s to 515us, and a
serial <code>FilterRecordBatch</code> over the same filter falls from 2.06s to 831us <a href="https://github.com/apache/arrow-go/pull/1287">#1287</a></li>
<li>Added SIMD-accelerated packing of boolean slices and validity data into Arrow bitmaps: <code>BooleanBuilder</code> packing of 65,536 values improves roughly 4-5x
(23.4us to 3.9us), batched validity packing is up to 4.4x faster, and ARM64 NEON bitmap operations gain 2.2-2.9x <a href="https://github.com/apache/arrow-go/pull/1283">#1283</a>,
<a href="https://github.com/apache/arrow-go/pull/1185">#1185</a>, <a href="https://github.com/apache/arrow-go/pull/1264">#1264</a>, <a href="https://github.com/apache/arrow-go/pull/1090">#1090</a></li>
<li>Batched run-end encoded appends turn a millisecond-scale operation into a microsecond-scale one for null and empty-value runs, cutting 1.16MB across 55
allocations per op down to 1.7KB across 19 <a href="https://github.com/apache/arrow-go/pull/1214">#1214</a></li>
<li>Concatenation avoids temporary buffers and copies bitmaps directly: 65,536 int64 values spread over 8,192 chunks go from 571.7us to 214.5us (-62%) with
allocations falling from 8,199 to 6 per op <a href="https://github.com/apache/arrow-go/pull/1194">#1194</a>, <a href="https://github.com/apache/arrow-go/pull/1224">#1224</a></li>
<li>Serial and small-batch execution skips goroutine and span overhead: single-span kernel dispatch nearly halves latency at length 1 (342.0ns to 189.0ns),
serial record filtering over 128 columns goes from 709.8us to 420.9us, and serial <code>Take</code> plus IPC encoder scratch reuse cut per-op allocations across the
board <a href="https://github.com/apache/arrow-go/pull/1233">#1233</a>, <a href="https://github.com/apache/arrow-go/pull/1245">#1245</a>, <a href="https://github.com/apache/arrow-go/pull/1226">#1226</a>, <a href="https://github.com/apache/arrow-go/pull/1221">#1221</a></li>
<li>CSV writing no longer materializes rows first, cutting a 1,000-row by 16-column write from 4.78ms to 0.84ms (-82%) and allocated bytes by 81% <a href="https://github.com/apache/arrow-go/pull/1190">#1190</a></li>
</ul>
<h3>Parquet</h3>
<h4>Features</h4>
<ul>
<li>Added read and write support for GeoArrow extension types using Parquet geometry and geography logical types <a href="https://github.com/apache/arrow-go/pull/960">#960</a>,
<a href="https://github.com/apache/arrow-go/pull/969">#969</a></li>
<li>Added codec-aware dictionary cost fallback and per-column writer controls <a href="https://github.com/apache/arrow-go/pull/1036">#1036</a></li>
</ul>
<h4>Bug Fixes</h4>
<ul>
<li>Correctly populated Bloom filters from dictionary-encoded values <a href="https://github.com/apache/arrow-go/pull/1164">#1164</a></li>
<li>Rejected decimal values that overflow Parquet INT32 or INT64 columns instead of silently narrowing them <a href="https://github.com/apache/arrow-go/pull/1122">#1122</a>,
<a href="https://github.com/apache/arrow-go/pull/1161">#1161</a></li>
<li>Fixed reading fixed-size lists whose parent values are null <a href="https://github.com/apache/arrow-go/pull/1099">#1099</a></li>
</ul>
<h4>Performance Improvements</h4>
<ul>
<li>Added SIMD <code>BYTE_STREAM_SPLIT</code> encoding on ARM64 and AMD64: 16.9x faster for 4-byte values (113,877ns to 6,745ns per op) and 7.6x for 8-byte values <a href="https://github.com/apache/arrow-go/pull/1260">#1260</a></li>
<li>Boolean decoding now targets Arrow bitmaps directly. All-true RLE boolean pages of 1M values decode in 2.45us instead of 2.65ms, and dense boolean column
reads through pqarrow drop 90% in time and 65% in heap use <a href="https://github.com/apache/arrow-go/pull/1246">#1246</a>, <a href="https://github.com/apache/arrow-go/pull/1182">#1182</a></li>
<li>Batched encoding and decoding across the core codecs: plain <code>BYTE_ARRAY</code> encoding is 45-62% faster, RLE level encoding is 79% faster for all-defined
levels, and <code>DELTA_BINARY_PACKED</code> decoding is 34-66% faster <a href="https://github.com/apache/arrow-go/pull/1193">#1193</a>, <a href="https://github.com/apache/arrow-go/pull/1177">#1177</a>, <a href="https://github.com/apache/arrow-go/pull/1169">#1169</a></li>
<li>Dictionary encoding allocates far less: numeric insertion of 65,535 float32 values goes from 64,905 allocations per op to 13, and typed byte-array memo
insertion cuts time 24% and allocations 50% (Apple M3 Max). Fixed-width dictionary materialization is also vectorized on AVX2 systems <a href="https://github.com/apache/arrow-go/pull/1178">#1178</a>,
<a href="https://github.com/apache/arrow-go/pull/1272">#1272</a>, <a href="https://github.com/apache/arrow-go/pull/1269">#1269</a></li>
<li>Reduced copying when writing binary and fixed-size binary Arrow values to Parquet: 16-byte <code>FixedSizeBinary</code> writes are 72% faster for required columns
with statistics disabled, and 64K String writes go from 1.798ms to 1.260ms <a href="https://github.com/apache/arrow-go/pull/1262">#1262</a>, <a href="https://github.com/apache/arrow-go/pull/1263">#1263</a></li>
<li>Compressors are reused across pages: random 64KiB gzip page encoding is 6.1x faster (94.7us to 15.6us) with allocations down from 15 to 1 per op, and
Brotli encoding drops from about 22 allocations per op to 1 <a href="https://github.com/apache/arrow-go/pull/1238">#1238</a>, <a href="https://github.com/apache/arrow-go/pull/1254">#1254</a></li>
<li>Serial reads skip goroutine orchestration, trimming 18% from single-column row-group reads <a href="https://github.com/apache/arrow-go/pull/1235">#1235</a></li>
</ul>
<h2>Changelog</h2>
<h3>Selected Changes</h3>
<p>A selection of the user-facing changes in this release. See the full changelog link
below for all 285 merged pull requests.</p>
<h4>Arrow</h4>
<ul>
<li>fix: remove superfluous struct field index compare by @Willem-J-an in <a href="https://github.com/apache/arrow-go/pull/970">#970</a></li>
<li>fix(ipc): reject truncated message frames by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/991">#991</a></li>
<li>fix: Fix arm64 assembly stack frame manipulation by @nsrip-dd in <a href="https://github.com/apache/arrow-go/pull/984">#984</a></li>
<li>fix(arrow/array): <code>Resize(0)</code> on non-empty builder causes memory leak by @serramatutu in <a href="https://github.com/apache/arrow-go/pull/995">#995</a></li>
<li>fix(arrow/csv): preserve decimal precision by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1022">#1022</a></li>
<li>refactor(arrow/avro): migrate from hamba/avro to twmb/avro by @prochac in <a href="https://github.com/apache/arrow-go/pull/830">#830</a></li>
<li>feat(flightsql): allow configuring the driver memory allocator by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1076">#1076</a></li>
<li>feat(arrow/flight/sql): Add is_update field to ActionCreatePreparedStatementResult by @ennuite in <a href="https://github.com/apache/arrow-go/pull/732">#732</a></li>
<li>feat(arrow/array): roll back JSON builder state after failed rows by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1113">#1113</a></li>
<li>feat(arrow/array): add ValueAsAny for native Go values by @dgvj-work in <a href="https://github.com/apache/arrow-go/pull/1170">#1170</a></li>
<li>feat(extensions): add VariantGet for path extraction from variant arrays by @nssalian in <a href="https://github.com/apache/arrow-go/pull/1206">#1206</a></li>
<li>fix(arrow/extensions): use canonical Variant extension name by @dgvj-work in <a href="https://github.com/apache/arrow-go/pull/1241">#1241</a></li>
<li>fix(arrow/extensions): reject Null Variant typed_value by @dgvj-work in <a href="https://github.com/apache/arrow-go/pull/1243">#1243</a></li>
<li>fix(arrow/ipc): validate file block framing and lengths by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1097">#1097</a></li>
<li>fix(arrow/ipc): avoid deadlock on compression errors by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1142">#1142</a></li>
<li>fix(arrow/scalar): preserve exact decimal string casts by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1107">#1107</a></li>
<li>fix(arrow/scalar): align timestamp timezone parsing by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1109">#1109</a></li>
<li>fix(arrow): validate precision in NewDecimalType by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1162">#1162</a></li>
<li>fix(arrow): reject duplicate union type codes by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1131">#1131</a></li>
<li>fix(arrow/csv): preserve first row during headerless inference by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1145">#1145</a></li>
<li>fix(arrow/csv): append null for later parse failures by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1149">#1149</a></li>
<li>fix(arrow/array): clamp run ends when concatenating a sliced RunEndEncoded array by @winklemad in <a href="https://github.com/apache/arrow-go/pull/1219">#1219</a></li>
<li>fix(arrow/cdata): validate imported schema topology by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1046">#1046</a></li>
</ul>
<h4>Compute</h4>
<ul>
<li>feat(compute): add list_element by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1138">#1138</a></li>
<li>feat(compute): add cumulative_sum and cumulative_sum_checked by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1139">#1139</a></li>
<li>feat(compute): add dictionary_encode by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1140">#1140</a></li>
<li>fix(compute): round negative HalfToOdd ties correctly by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1094">#1094</a></li>
<li>fix(compute): handle odd temporal multiples by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1119">#1119</a></li>
<li>fix(compute): preserve caller context during execution by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1154">#1154</a></li>
<li>fix(compute): discard results after cancellation by @zeroshade in <a href="https://github.com/apache/arrow-go/pull/1278">#1278</a></li>
<li>fix(compute/exprs): avoid empty field reference panic by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1130">#1130</a></li>
</ul>
<h4>Parquet</h4>
<ul>
<li>feat(parquet/schema): Wire up logical type functionality for writing geo types by @C-Loftus in <a href="https://github.com/apache/arrow-go/pull/960">#960</a></li>
<li>feat(parquet/pqarrow): Read support for extension types / geo by @C-Loftus in <a href="https://github.com/apache/arrow-go/pull/969">#969</a></li>
<li>feat(parquet): add WithDictionaryCostFallback writer property by @tom-eon in <a href="https://github.com/apache/arrow-go/pull/1036">#1036</a></li>
<li>fix(parquet): wrap DELTA_BINARY_PACKED deltas at the physical type width by @tom-eon in <a href="https://github.com/apache/arrow-go/pull/1027">#1027</a></li>
<li>fix(parquet/encoding): read ahead in the streaming value buffer by @joechenrh in <a href="https://github.com/apache/arrow-go/pull/937">#937</a></li>
<li>fix(parquet/pqarrow): prefer stored schema extension type on read by @twuebi in <a href="https://github.com/apache/arrow-go/pull/1051">#1051</a></li>
<li>fix(parquet/pqarrow): require Variant value field when writing by @dgvj-work in <a href="https://github.com/apache/arrow-go/pull/1242">#1242</a></li>
<li>fix(parquet): guard byte-stream-split encoder release against never-flushed buffer by @tom-eon in <a href="https://github.com/apache/arrow-go/pull/1273">#1273</a></li>
<li>fix(parquet): stop BYTE_STREAM_SPLIT FLBA spaced decode aliasing output by @zeroshade in <a href="https://github.com/apache/arrow-go/pull/1256">#1256</a></li>
<li>fix(parquet): read null fixed-size-list parents by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1099">#1099</a></li>
<li>fix(parquet/pqarrow): reject decimal overflow in integer columns by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1122">#1122</a></li>
<li>fix(parquet/pqarrow): reject decimal overflow when writing integers by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1161">#1161</a></li>
<li>fix(parquet): build bloom filters from dictionary entries by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1164">#1164</a></li>
<li>fix(parquet): preserve dictionary index types by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1098">#1098</a></li>
<li>fix(parquet): restore Arrow duration units from schema metadata by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1095">#1095</a></li>
<li>fix(parquet/compress): honor LZ4 raw destination contract by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1147">#1147</a></li>
<li>fix(parquet): row-align V1 pages with offset indexes by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1075">#1075</a></li>
<li>fix(parquet/variant): validate metadata offset tables by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1063">#1063</a></li>
</ul>
<h4>Performance</h4>
<ul>
<li>perf(arrow/bitutil): add SetBitSwap/ClearBitSwap, use in builder.SetNull by @tom-eon in <a href="https://github.com/apache/arrow-go/pull/1090">#1090</a></li>
<li>perf(bitutil): add ARM64 NEON bitmap operations by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1264">#1264</a></li>
<li>perf(arrow/array): pack validity values in batches by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1185">#1185</a></li>
<li>perf(arrow/array): SIMD-pack boolean slices into bitmaps by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1283">#1283</a></li>
<li>perf(arrow/array): batch run-end encoded appends by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1214">#1214</a></li>
<li>perf(arrow/array): avoid temporary buffers when concatenating by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1194">#1194</a></li>
<li>perf(arrow/array): concatenate bitmaps directly by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1224">#1224</a></li>
<li>perf(compute): add ARM64 NEON arithmetic kernels by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1265">#1265</a></li>
<li>perf(compute): add ARM64 NEON comparison kernels by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1266">#1266</a></li>
<li>perf(compute): add ARM64 NEON comparisons for 8- and 16-bit integers by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1288">#1288</a></li>
<li>perf(compute): add ARM64 NEON compression for null-free 32-bit fixed-width filters by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1286">#1286</a></li>
<li>perf(compute): vectorize mixed int32 filters with AVX2 by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1284">#1284</a></li>
<li>perf(compute): SIMD-compact fragmented filters into take indices by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1287">#1287</a></li>
<li>perf(compute): fast-path single-span execution by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1233">#1233</a></li>
<li>perf(arrow/compute): avoid goroutines for serial record filtering by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1245">#1245</a></li>
<li>perf(arrow/compute): avoid goroutines for serial take by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1226">#1226</a></li>
<li>perf(parquet): SIMD BYTE_STREAM_SPLIT encoding by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1260">#1260</a></li>
<li>perf(parquet): vectorize fixed-width dictionary materialization by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1269">#1269</a></li>
<li>perf(parquet): optimize numeric dictionary insertion by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1178">#1178</a></li>
<li>perf(parquet): use typed memo insertion for byte-array dictionaries by @derekperkins in <a href="https://github.com/apache/arrow-go/pull/1272">#1272</a></li>
<li>perf(parquet): batch plain BYTE_ARRAY encoding by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1193">#1193</a></li>
<li>perf(parquet): encode RLE levels in batches by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1177">#1177</a></li>
<li>perf(parquet): batch delta binary packed decoding by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1169">#1169</a></li>
<li>perf(parquet): decode RLE booleans into bitmaps by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1246">#1246</a></li>
<li>perf(parquet/pqarrow): decode booleans directly to bitmaps by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1182">#1182</a></li>
<li>perf(parquet/pqarrow): write binary values from Arrow offsets by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1262">#1262</a></li>
<li>perf(parquet/pqarrow): write FixedSizeBinary values directly by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1263">#1263</a></li>
<li>perf(parquet/pqarrow): use synchronous path for serial reads by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1235">#1235</a></li>
<li>perf(parquet/compress): reuse gzip writers by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1238">#1238</a></li>
<li>perf(parquet/compress): reuse Brotli writers across encodes by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1254">#1254</a></li>
<li>perf(arrow/csv): avoid materializing rows before writing by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1190">#1190</a></li>
<li>perf(arrow/ipc): reuse record encoder scratch by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/1221">#1221</a></li>
</ul>
<h3>New Contributors</h3>
<ul>
<li>@C-Loftus made their first contribution in <a href="https://github.com/apache/arrow-go/pull/960">#960</a></li>
<li>@tom-eon made their first contribution in <a href="https://github.com/apache/arrow-go/pull/1027">#1027</a></li>
<li>@prochac made their first contribution in <a href="https://github.com/apache/arrow-go/pull/830">#830</a></li>
<li>@dgvj-work made their first contribution in <a href="https://github.com/apache/arrow-go/pull/1170">#1170</a></li>
<li>@ennuite made their first contribution in <a href="https://github.com/apache/arrow-go/pull/732">#732</a></li>
<li>@derekperkins made their first contribution in <a href="https://github.com/apache/arrow-go/pull/1272">#1272</a></li>
<li>@winklemad made their first contribution in <a href="https://github.com/apache/arrow-go/pull/1219">#1219</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a href="https://github.com/apache/arrow-go/compare/v18.7.0...v18.8.0">https://github.com/apache/arrow-go/compare/v18.7.0...v18.8.0</a></p>]]></content><author><name>pmc</name></author><category term="release" /><summary type="html"><![CDATA[The Apache Arrow team is pleased to announce the v18.8.0 release of Apache Arrow Go. This minor release covers 283 commits from 16 distinct contributors. Contributors $ git shortlog -sn v18.7.0..v18.8.0 251 Minh Vu 10 Matt Topol 4 Colton Loftus 4 Digvijay 4 Tom Frank 1 Derek Perkins 1 Lucas Valente 1 Madan kumar 1 Neelesh Salian 1 Nick Ripley 1 Pedro Matias 1 Raúl Cumplido 1 Ruihao Chen 1 Tobias Pütz 1 Tomáš Karela Procházka 1 Willem Jan Highlights Arrow Features Added VariantGet for efficient path extraction from variant arrays #1206 Added list_element, cumulative_sum, cumulative_sum_checked, and dictionary_encode compute functions #1138, #1139, #1140 Added ValueAsAny for accessing Arrow array values as native Go values #1170 Strengthened IPC file validation for malformed block framing and lengths #1097 Bug Fixes Fixed a memory leak when calling Resize(0) on non-empty builders #995 Preserved exact precision when casting strings to Decimal128 and Decimal256 scalars #1107 Performance Improvements This is a performance-heavy release: 32 of the merged pull requests are dedicated optimizations. Unless noted otherwise, the figures below are the PR-reported benchmarks on an Apple M1 Pro, comparing the new path against the scalar baseline it replaces. Expanded SIMD acceleration for compute arithmetic, comparisons, and filtering on ARM64 and AMD64. Arithmetic kernels run 4.1-4.4x faster on int64 and float64 adds, int64 array/array comparisons drop from 313.6us to 29.3us per op (10.7x), narrow 8- and 16-bit integer comparisons reach 12.8-15.1x, and null-free 32-bit fixed-width filters compress up to 11.8x faster on 1M-element inputs #1265, #1266, #1284, #1286, #1287, #1288 Building take indices from fragmented selection vectors is no longer quadratic. For a 1M-element alternating filter it drops from 4.31s to 515us, and a serial FilterRecordBatch over the same filter falls from 2.06s to 831us #1287 Added SIMD-accelerated packing of boolean slices and validity data into Arrow bitmaps: BooleanBuilder packing of 65,536 values improves roughly 4-5x (23.4us to 3.9us), batched validity packing is up to 4.4x faster, and ARM64 NEON bitmap operations gain 2.2-2.9x #1283, #1185, #1264, #1090 Batched run-end encoded appends turn a millisecond-scale operation into a microsecond-scale one for null and empty-value runs, cutting 1.16MB across 55 allocations per op down to 1.7KB across 19 #1214 Concatenation avoids temporary buffers and copies bitmaps directly: 65,536 int64 values spread over 8,192 chunks go from 571.7us to 214.5us (-62%) with allocations falling from 8,199 to 6 per op #1194, #1224 Serial and small-batch execution skips goroutine and span overhead: single-span kernel dispatch nearly halves latency at length 1 (342.0ns to 189.0ns), serial record filtering over 128 columns goes from 709.8us to 420.9us, and serial Take plus IPC encoder scratch reuse cut per-op allocations across the board #1233, #1245, #1226, #1221 CSV writing no longer materializes rows first, cutting a 1,000-row by 16-column write from 4.78ms to 0.84ms (-82%) and allocated bytes by 81% #1190 Parquet Features Added read and write support for GeoArrow extension types using Parquet geometry and geography logical types #960, #969 Added codec-aware dictionary cost fallback and per-column writer controls #1036 Bug Fixes Correctly populated Bloom filters from dictionary-encoded values #1164 Rejected decimal values that overflow Parquet INT32 or INT64 columns instead of silently narrowing them #1122, #1161 Fixed reading fixed-size lists whose parent values are null #1099 Performance Improvements Added SIMD BYTE_STREAM_SPLIT encoding on ARM64 and AMD64: 16.9x faster for 4-byte values (113,877ns to 6,745ns per op) and 7.6x for 8-byte values #1260 Boolean decoding now targets Arrow bitmaps directly. All-true RLE boolean pages of 1M values decode in 2.45us instead of 2.65ms, and dense boolean column reads through pqarrow drop 90% in time and 65% in heap use #1246, #1182 Batched encoding and decoding across the core codecs: plain BYTE_ARRAY encoding is 45-62% faster, RLE level encoding is 79% faster for all-defined levels, and DELTA_BINARY_PACKED decoding is 34-66% faster #1193, #1177, #1169 Dictionary encoding allocates far less: numeric insertion of 65,535 float32 values goes from 64,905 allocations per op to 13, and typed byte-array memo insertion cuts time 24% and allocations 50% (Apple M3 Max). Fixed-width dictionary materialization is also vectorized on AVX2 systems #1178, #1272, #1269 Reduced copying when writing binary and fixed-size binary Arrow values to Parquet: 16-byte FixedSizeBinary writes are 72% faster for required columns with statistics disabled, and 64K String writes go from 1.798ms to 1.260ms #1262, #1263 Compressors are reused across pages: random 64KiB gzip page encoding is 6.1x faster (94.7us to 15.6us) with allocations down from 15 to 1 per op, and Brotli encoding drops from about 22 allocations per op to 1 #1238, #1254 Serial reads skip goroutine orchestration, trimming 18% from single-column row-group reads #1235 Changelog Selected Changes A selection of the user-facing changes in this release. See the full changelog link below for all 285 merged pull requests. Arrow fix: remove superfluous struct field index compare by @Willem-J-an in #970 fix(ipc): reject truncated message frames by @fallintoplace in #991 fix: Fix arm64 assembly stack frame manipulation by @nsrip-dd in #984 fix(arrow/array): Resize(0) on non-empty builder causes memory leak by @serramatutu in #995 fix(arrow/csv): preserve decimal precision by @fallintoplace in #1022 refactor(arrow/avro): migrate from hamba/avro to twmb/avro by @prochac in #830 feat(flightsql): allow configuring the driver memory allocator by @fallintoplace in #1076 feat(arrow/flight/sql): Add is_update field to ActionCreatePreparedStatementResult by @ennuite in #732 feat(arrow/array): roll back JSON builder state after failed rows by @fallintoplace in #1113 feat(arrow/array): add ValueAsAny for native Go values by @dgvj-work in #1170 feat(extensions): add VariantGet for path extraction from variant arrays by @nssalian in #1206 fix(arrow/extensions): use canonical Variant extension name by @dgvj-work in #1241 fix(arrow/extensions): reject Null Variant typed_value by @dgvj-work in #1243 fix(arrow/ipc): validate file block framing and lengths by @fallintoplace in #1097 fix(arrow/ipc): avoid deadlock on compression errors by @fallintoplace in #1142 fix(arrow/scalar): preserve exact decimal string casts by @fallintoplace in #1107 fix(arrow/scalar): align timestamp timezone parsing by @fallintoplace in #1109 fix(arrow): validate precision in NewDecimalType by @fallintoplace in #1162 fix(arrow): reject duplicate union type codes by @fallintoplace in #1131 fix(arrow/csv): preserve first row during headerless inference by @fallintoplace in #1145 fix(arrow/csv): append null for later parse failures by @fallintoplace in #1149 fix(arrow/array): clamp run ends when concatenating a sliced RunEndEncoded array by @winklemad in #1219 fix(arrow/cdata): validate imported schema topology by @fallintoplace in #1046 Compute feat(compute): add list_element by @fallintoplace in #1138 feat(compute): add cumulative_sum and cumulative_sum_checked by @fallintoplace in #1139 feat(compute): add dictionary_encode by @fallintoplace in #1140 fix(compute): round negative HalfToOdd ties correctly by @fallintoplace in #1094 fix(compute): handle odd temporal multiples by @fallintoplace in #1119 fix(compute): preserve caller context during execution by @fallintoplace in #1154 fix(compute): discard results after cancellation by @zeroshade in #1278 fix(compute/exprs): avoid empty field reference panic by @fallintoplace in #1130 Parquet feat(parquet/schema): Wire up logical type functionality for writing geo types by @C-Loftus in #960 feat(parquet/pqarrow): Read support for extension types / geo by @C-Loftus in #969 feat(parquet): add WithDictionaryCostFallback writer property by @tom-eon in #1036 fix(parquet): wrap DELTA_BINARY_PACKED deltas at the physical type width by @tom-eon in #1027 fix(parquet/encoding): read ahead in the streaming value buffer by @joechenrh in #937 fix(parquet/pqarrow): prefer stored schema extension type on read by @twuebi in #1051 fix(parquet/pqarrow): require Variant value field when writing by @dgvj-work in #1242 fix(parquet): guard byte-stream-split encoder release against never-flushed buffer by @tom-eon in #1273 fix(parquet): stop BYTE_STREAM_SPLIT FLBA spaced decode aliasing output by @zeroshade in #1256 fix(parquet): read null fixed-size-list parents by @fallintoplace in #1099 fix(parquet/pqarrow): reject decimal overflow in integer columns by @fallintoplace in #1122 fix(parquet/pqarrow): reject decimal overflow when writing integers by @fallintoplace in #1161 fix(parquet): build bloom filters from dictionary entries by @fallintoplace in #1164 fix(parquet): preserve dictionary index types by @fallintoplace in #1098 fix(parquet): restore Arrow duration units from schema metadata by @fallintoplace in #1095 fix(parquet/compress): honor LZ4 raw destination contract by @fallintoplace in #1147 fix(parquet): row-align V1 pages with offset indexes by @fallintoplace in #1075 fix(parquet/variant): validate metadata offset tables by @fallintoplace in #1063 Performance perf(arrow/bitutil): add SetBitSwap/ClearBitSwap, use in builder.SetNull by @tom-eon in #1090 perf(bitutil): add ARM64 NEON bitmap operations by @fallintoplace in #1264 perf(arrow/array): pack validity values in batches by @fallintoplace in #1185 perf(arrow/array): SIMD-pack boolean slices into bitmaps by @fallintoplace in #1283 perf(arrow/array): batch run-end encoded appends by @fallintoplace in #1214 perf(arrow/array): avoid temporary buffers when concatenating by @fallintoplace in #1194 perf(arrow/array): concatenate bitmaps directly by @fallintoplace in #1224 perf(compute): add ARM64 NEON arithmetic kernels by @fallintoplace in #1265 perf(compute): add ARM64 NEON comparison kernels by @fallintoplace in #1266 perf(compute): add ARM64 NEON comparisons for 8- and 16-bit integers by @fallintoplace in #1288 perf(compute): add ARM64 NEON compression for null-free 32-bit fixed-width filters by @fallintoplace in #1286 perf(compute): vectorize mixed int32 filters with AVX2 by @fallintoplace in #1284 perf(compute): SIMD-compact fragmented filters into take indices by @fallintoplace in #1287 perf(compute): fast-path single-span execution by @fallintoplace in #1233 perf(arrow/compute): avoid goroutines for serial record filtering by @fallintoplace in #1245 perf(arrow/compute): avoid goroutines for serial take by @fallintoplace in #1226 perf(parquet): SIMD BYTE_STREAM_SPLIT encoding by @fallintoplace in #1260 perf(parquet): vectorize fixed-width dictionary materialization by @fallintoplace in #1269 perf(parquet): optimize numeric dictionary insertion by @fallintoplace in #1178 perf(parquet): use typed memo insertion for byte-array dictionaries by @derekperkins in #1272 perf(parquet): batch plain BYTE_ARRAY encoding by @fallintoplace in #1193 perf(parquet): encode RLE levels in batches by @fallintoplace in #1177 perf(parquet): batch delta binary packed decoding by @fallintoplace in #1169 perf(parquet): decode RLE booleans into bitmaps by @fallintoplace in #1246 perf(parquet/pqarrow): decode booleans directly to bitmaps by @fallintoplace in #1182 perf(parquet/pqarrow): write binary values from Arrow offsets by @fallintoplace in #1262 perf(parquet/pqarrow): write FixedSizeBinary values directly by @fallintoplace in #1263 perf(parquet/pqarrow): use synchronous path for serial reads by @fallintoplace in #1235 perf(parquet/compress): reuse gzip writers by @fallintoplace in #1238 perf(parquet/compress): reuse Brotli writers across encodes by @fallintoplace in #1254 perf(arrow/csv): avoid materializing rows before writing by @fallintoplace in #1190 perf(arrow/ipc): reuse record encoder scratch by @fallintoplace in #1221 New Contributors @C-Loftus made their first contribution in #960 @tom-eon made their first contribution in #1027 @prochac made their first contribution in #830 @dgvj-work made their first contribution in #1170 @ennuite made their first contribution in #732 @derekperkins made their first contribution in #1272 @winklemad made their first contribution in #1219 Full Changelog: https://github.com/apache/arrow-go/compare/v18.7.0...v18.8.0]]></summary><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://arrow.apache.org/img/arrow-logo_horizontal_black-txt_white-bg.png" /><media:content medium="image" url="https://arrow.apache.org/img/arrow-logo_horizontal_black-txt_white-bg.png" xmlns:media="http://search.yahoo.com/mrss/" /></entry><entry><title type="html">Apache Arrow nanoarrow 0.9.0 Release</title><link href="https://arrow.apache.org/blog/2026/08/14/nanoarrow-0.9.0-release/" rel="alternate" type="text/html" title="Apache Arrow nanoarrow 0.9.0 Release" /><published>2026-08-14T00:00:00-04:00</published><updated>2026-08-14T00:00:00-04:00</updated><id>https://arrow.apache.org/blog/2026/08/14/nanoarrow-0.9.0-release</id><content type="html" xml:base="https://arrow.apache.org/blog/2026/08/14/nanoarrow-0.9.0-release/"><![CDATA[<!--

-->
<p>The Apache Arrow team is pleased to announce the 0.9.0 release of
Apache Arrow nanoarrow. This release consists of 38 resolved GitHub issues from
5 contributors.</p>
<h2>Release Highlights</h2>
<p>In addition to a number of bugfixes and minor build system improvements, we
added several new features in nanoarrow 0.9.0.</p>
<ul>
<li>Dictionary decoding support in IPC reader</li>
<li>Reference-counted array/buffer support</li>
<li>LZ4 decompression support in R and Python bindings</li>
</ul>
<p>See the
<a href="https://github.com/apache/arrow-nanoarrow/blob/apache-arrow-nanoarrow-0.9.0/CHANGELOG.md">Changelog</a>
for a detailed list of contributions to this release.</p>
<h2>Features</h2>
<h3>Dictionary decode support</h3>
<p>Whereas the nanoarrow IPC reader suppports most Arrow IPC features, dictionary support
was a long requested gap in the reader functionality (mostly requested by users of
the <a href="https://github.com/paleolimbot/duckdb-nanoarrow">DuckDB nanoarrow extension</a>, which uses nanoarrow's reader). Dictionary encoding is used to reduce the size of frequently
repeated values and is the serialized equivalent of the &quot;dictionary&quot; data type that
is exposed in most Arrow implementations.</p>
<p>In nanoarrow 0.9.0 built with the IPC feature enabled, streams that include the most
common forms of dictionary encoding (i.e., dictionary replacement) should now work
out of the box. This includes nested/complex dictionary types and dictionary
replacement but does not include &quot;delta&quot; dictionaries (i.e., dictionaires that
grow larger as more values are encountered in the encoded values).</p>
<p>In R this is accessible via <code>read_nanoarrow()</code>; in Python this is accessible via
<code>nanoarrow.ArrayStream.from_readable()</code>; in C this is available via the
higher level <code>ArrowIpcArrayStreamReader</code> API. Lower level users of the
<code>ArrowIpcDecoder</code> will have to update existing usage to use the
<code>ArrowIpcDecoder...WithDictionaries()</code> variants of some functions
to support input with dictionary schemas or batches.</p>
<h3>Reference-counted array/buffer support</h3>
<p>In previous versions (since the introduction of the IPC reader), the
<code>ArrowIpcSharedBuffer</code> has supported reading IPC streams and sharing an
underlying set of data buffers for a group of arrays; however, this wasn't
quite sufficient for the more complex case of decoding a dictionary and
attaching cheaply-cloned shared &quot;values&quot; arrays for potentially many batches.
Version 0.9.0 moves this functionality to the <code>ArrowSharedBuffer</code> and expands it
to support moving all of an array's buffers into a shared state that can be
more cheaply cloned.</p>
<h3>LZ4 decompression support in R and Python</h3>
<p>While LZ4 decompression support was has long been available via the pluggable decoder
framework (and available since 0.8.0 as a built-in compile time option), reading
IPC streams with LZ4 buffer compression was not possible in the R or Python bindings.
In 0.9.0, the requisite configuration options were added such that the packages are
built with LZ4 when it is available on the system.</p>
<pre><code class="language-python">import io
import nanoarrow as na
import pyarrow as pa

buf = io.BytesIO()
batch = pa.record_batch({&quot;x&quot;: range(1000)})
with pa.ipc.new_stream(buf, batch.schema, options=pa.ipc.IpcWriteOptions(compression=&quot;lz4&quot;)) as w:
    w.write_batch(batch)

buf.seek(0)
na.ArrayStream.from_readable(buf).read_all()
# nanoarrow.Array&lt;non-nullable struct&lt;x: int64&gt;&gt;[1000]
# {'x': 0}
# {'x': 1}
# {'x': 2}
# {'x': 3}
# {'x': 4}
# {'x': 5}
# {'x': 6}
# {'x': 7}
# {'x': 8}
# {'x': 9}
# ...and 990 more items
</code></pre>
<pre><code class="language-r">library(nanoarrow)
library(reticulate)

# IPC Write with compression not available in arrow/R
pa &lt;- reticulate::import(&quot;pyarrow&quot;)
io &lt;- reticulate::import(&quot;io&quot;)

buf &lt;- io$BytesIO()
batch &lt;- arrow::record_batch(x = 1:1000)
writer &lt;- pa$ipc$new_stream(buf, batch$schema, options = pa$ipc$IpcWriteOptions(compression = &quot;lz4&quot;))
writer$write_batch(batch)
writer$close()

nanoarrow::read_nanoarrow(as.raw(buf$getvalue())) |&gt;
  tibble::as_tibble()
#&gt; # A tibble: 1,000 × 1
#&gt;        x
#&gt;    &lt;int&gt;
#&gt;  1     1
#&gt;  2     2
#&gt;  3     3
#&gt;  4     4
#&gt;  5     5
#&gt;  6     6
#&gt;  7     7
#&gt;  8     8
#&gt;  9     9
#&gt; 10    10
#&gt; # ℹ 990 more rows
</code></pre>
<h2>Contributors</h2>
<p>This release consists of contributions from 5 contributors in addition
to the invaluable advice and support of the Apache Arrow community.</p>
<pre><code class="language-console">$ git shortlog -sn apache-arrow-nanoarrow-0.9.0.dev..apache-arrow-nanoarrow-0.9.0
    36  Dewey Dunnington
     2  Andrew Kane
     2  Bryce Mecum
     1  Michael Osipov
     1  Oliver Borchert
</code></pre>]]></content><author><name>pmc</name></author><category term="release" /><summary type="html"><![CDATA[The Apache Arrow team is pleased to announce the 0.9.0 release of Apache Arrow nanoarrow. This release consists of 38 resolved GitHub issues from 5 contributors. Release Highlights In addition to a number of bugfixes and minor build system improvements, we added several new features in nanoarrow 0.9.0. Dictionary decoding support in IPC reader Reference-counted array/buffer support LZ4 decompression support in R and Python bindings See the Changelog for a detailed list of contributions to this release. Features Dictionary decode support Whereas the nanoarrow IPC reader suppports most Arrow IPC features, dictionary support was a long requested gap in the reader functionality (mostly requested by users of the DuckDB nanoarrow extension, which uses nanoarrow's reader). Dictionary encoding is used to reduce the size of frequently repeated values and is the serialized equivalent of the &quot;dictionary&quot; data type that is exposed in most Arrow implementations. In nanoarrow 0.9.0 built with the IPC feature enabled, streams that include the most common forms of dictionary encoding (i.e., dictionary replacement) should now work out of the box. This includes nested/complex dictionary types and dictionary replacement but does not include &quot;delta&quot; dictionaries (i.e., dictionaires that grow larger as more values are encountered in the encoded values). In R this is accessible via read_nanoarrow(); in Python this is accessible via nanoarrow.ArrayStream.from_readable(); in C this is available via the higher level ArrowIpcArrayStreamReader API. Lower level users of the ArrowIpcDecoder will have to update existing usage to use the ArrowIpcDecoder...WithDictionaries() variants of some functions to support input with dictionary schemas or batches. Reference-counted array/buffer support In previous versions (since the introduction of the IPC reader), the ArrowIpcSharedBuffer has supported reading IPC streams and sharing an underlying set of data buffers for a group of arrays; however, this wasn't quite sufficient for the more complex case of decoding a dictionary and attaching cheaply-cloned shared &quot;values&quot; arrays for potentially many batches. Version 0.9.0 moves this functionality to the ArrowSharedBuffer and expands it to support moving all of an array's buffers into a shared state that can be more cheaply cloned. LZ4 decompression support in R and Python While LZ4 decompression support was has long been available via the pluggable decoder framework (and available since 0.8.0 as a built-in compile time option), reading IPC streams with LZ4 buffer compression was not possible in the R or Python bindings. In 0.9.0, the requisite configuration options were added such that the packages are built with LZ4 when it is available on the system. import io import nanoarrow as na import pyarrow as pa buf = io.BytesIO() batch = pa.record_batch({&quot;x&quot;: range(1000)}) with pa.ipc.new_stream(buf, batch.schema, options=pa.ipc.IpcWriteOptions(compression=&quot;lz4&quot;)) as w: w.write_batch(batch) buf.seek(0) na.ArrayStream.from_readable(buf).read_all() # nanoarrow.Array&lt;non-nullable struct&lt;x: int64&gt;&gt;[1000] # {'x': 0} # {'x': 1} # {'x': 2} # {'x': 3} # {'x': 4} # {'x': 5} # {'x': 6} # {'x': 7} # {'x': 8} # {'x': 9} # ...and 990 more items library(nanoarrow) library(reticulate) # IPC Write with compression not available in arrow/R pa &lt;- reticulate::import(&quot;pyarrow&quot;) io &lt;- reticulate::import(&quot;io&quot;) buf &lt;- io$BytesIO() batch &lt;- arrow::record_batch(x = 1:1000) writer &lt;- pa$ipc$new_stream(buf, batch$schema, options = pa$ipc$IpcWriteOptions(compression = &quot;lz4&quot;)) writer$write_batch(batch) writer$close() nanoarrow::read_nanoarrow(as.raw(buf$getvalue())) |&gt; tibble::as_tibble() #&gt; # A tibble: 1,000 × 1 #&gt; x #&gt; &lt;int&gt; #&gt; 1 1 #&gt; 2 2 #&gt; 3 3 #&gt; 4 4 #&gt; 5 5 #&gt; 6 6 #&gt; 7 7 #&gt; 8 8 #&gt; 9 9 #&gt; 10 10 #&gt; # ℹ 990 more rows Contributors This release consists of contributions from 5 contributors in addition to the invaluable advice and support of the Apache Arrow community. $ git shortlog -sn apache-arrow-nanoarrow-0.9.0.dev..apache-arrow-nanoarrow-0.9.0 36 Dewey Dunnington 2 Andrew Kane 2 Bryce Mecum 1 Michael Osipov 1 Oliver Borchert]]></summary><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://arrow.apache.org/img/arrow-logo_horizontal_black-txt_white-bg.png" /><media:content medium="image" url="https://arrow.apache.org/img/arrow-logo_horizontal_black-txt_white-bg.png" xmlns:media="http://search.yahoo.com/mrss/" /></entry><entry><title type="html">Apache Arrow 25.0.1 Release</title><link href="https://arrow.apache.org/blog/2026/08/10/25.0.1-release/" rel="alternate" type="text/html" title="Apache Arrow 25.0.1 Release" /><published>2026-08-10T00:00:00-04:00</published><updated>2026-08-10T00:00:00-04:00</updated><id>https://arrow.apache.org/blog/2026/08/10/25.0.1-release</id><content type="html" xml:base="https://arrow.apache.org/blog/2026/08/10/25.0.1-release/"><![CDATA[<!--

-->
<p>The Apache Arrow team is pleased to announce the 25.0.1 release.
The 25.0.1 release is mostly a bugfix release that includes <a href="https://github.com/apache/arrow/milestone/76?closed=1"><strong>9 resolved issues</strong></a>
on <a href="/release/25.0.1.html#contributors"><strong>10 distinct commits</strong></a> from <a href="/release/25.0.1.html#contributors"><strong>6 distinct contributors</strong></a>.</p>
<p>See the <a href="https://arrow.apache.org/install/">Install Page</a> to
learn how to get the libraries for your platform.</p>
<p>The release notes below are not exhaustive and only expose selected highlights
of the release. Many other bugfixes and improvements have been made: we refer
you to the <a href="/release/25.0.1.html#changelog">complete changelog</a>.</p>
<h2>C++ notes</h2>
<ul>
<li>Fix possible Segfault on SVE128 unpack (<a href="https://github.com/apache/arrow/issues/50503">GH-50503</a>),
(<a href="https://github.com/apache/arrow/issues/50605">GH-50605</a>).</li>
<li>Bump bundled mimalloc to fix possible Segfault with bundled mimalloc (<a href="https://github.com/apache/arrow/issues/50471">GH-50471</a>),
(<a href="https://github.com/apache/arrow/issues/50428">GH-50428</a>).</li>
</ul>
<h2>Python notes</h2>
<ul>
<li>Add performance improvement in order to solve Apache Spark's
performance regression on their switch from regular Python UDFs to
Arrow serialization by default (<a href="https://github.com/apache/arrow/issues/50326">GH-50326</a>).</li>
<li>Narrow the Feather deprecation added in 25.0.0 to legacy V1 only.
<code>pyarrow.feather</code> V2 reads and writes no longer warn, while V1 reads and
writes emit <code>DeprecationWarning</code> instead of <code>FutureWarning</code> (<a href="https://github.com/apache/arrow/issues/50808">GH-50808</a>).</li>
</ul>
<h2>Other modules and languages</h2>
<p>No general changes were made to the other libraries or languages.</p>]]></content><author><name>pmc</name></author><category term="release" /><summary type="html"><![CDATA[The Apache Arrow team is pleased to announce the 25.0.1 release. The 25.0.1 release is mostly a bugfix release that includes 9 resolved issues on 10 distinct commits from 6 distinct contributors. See the Install Page to learn how to get the libraries for your platform. The release notes below are not exhaustive and only expose selected highlights of the release. Many other bugfixes and improvements have been made: we refer you to the complete changelog. C++ notes Fix possible Segfault on SVE128 unpack (GH-50503), (GH-50605). Bump bundled mimalloc to fix possible Segfault with bundled mimalloc (GH-50471), (GH-50428). Python notes Add performance improvement in order to solve Apache Spark's performance regression on their switch from regular Python UDFs to Arrow serialization by default (GH-50326). Narrow the Feather deprecation added in 25.0.0 to legacy V1 only. pyarrow.feather V2 reads and writes no longer warn, while V1 reads and writes emit DeprecationWarning instead of FutureWarning (GH-50808). Other modules and languages No general changes were made to the other libraries or languages.]]></summary><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://arrow.apache.org/img/arrow-logo_horizontal_black-txt_white-bg.png" /><media:content medium="image" url="https://arrow.apache.org/img/arrow-logo_horizontal_black-txt_white-bg.png" xmlns:media="http://search.yahoo.com/mrss/" /></entry><entry><title type="html">Apache Arrow ADBC 24 (Libraries) Release</title><link href="https://arrow.apache.org/blog/2026/07/28/adbc-24-release/" rel="alternate" type="text/html" title="Apache Arrow ADBC 24 (Libraries) Release" /><published>2026-07-28T00:00:00-04:00</published><updated>2026-07-28T00:00:00-04:00</updated><id>https://arrow.apache.org/blog/2026/07/28/adbc-24-release</id><content type="html" xml:base="https://arrow.apache.org/blog/2026/07/28/adbc-24-release/"><![CDATA[<!--

-->
<p>The Apache Arrow team is pleased to announce the version 24 release of
the Apache Arrow ADBC libraries. This release includes <a href="https://github.com/apache/arrow-adbc/issues?q=is%3Aissue%20state%3Aclosed%20milestone%3A%22ADBC%20Libraries%2024%22"><strong>57
resolved issues</strong></a> and <a href="https://github.com/apache/arrow-adbc/pulls?q=is%3Apr%20state%3Aclosed%20milestone%3A%22ADBC%20Libraries%2024%22%20-author%3Aapp%2Fdependabot"><strong>142 merged pull requests</strong></a> from
<a href="#contributors"><strong>28 distinct contributors</strong></a>.</p>
<p>This is a release of the <strong>libraries</strong>, which are at version 24.  The
<a href="https://arrow.apache.org/adbc/24/format/specification.html"><strong>API specification</strong></a> is versioned separately and is at
version 1.1.0.</p>
<p>The subcomponents are versioned independently:</p>
<ul>
<li>C/C++/GLib/Go/Python/Ruby: 1.12.0</li>
<li>C#: 0.24.0</li>
<li>Java: 0.24.0</li>
<li>JavaScript: 0.24.0</li>
<li>R: 0.24.0</li>
<li>Rust: 0.24.0</li>
</ul>
<p>The release notes below are not exhaustive and only expose selected
highlights of the release. Many other bugfixes and improvements have
been made: we refer you to the <a href="https://github.com/apache/arrow-adbc/blob/apache-arrow-adbc-24/CHANGELOG.md">complete changelog</a>.</p>
<h2>Release Highlights</h2>
<p>Note: we are planning to require C++20 starting from the next release. Also,
we will drop support for Python 3.10 no earlier than the release after next
i.e. no earlier than release 26. Release 26 is expected in about 3-4 months
(October~November 2026), and the EOL for Python 3.10 is October 2026.</p>
<h3>Breaking Changes and Deprecations</h3>
<p>Development of the ADBC drivers for Apache DataFusion, BigQuery, Databricks,
and Snowflake has moved to the <a href="https://adbc-drivers.org/">ADBC Driver
Foundry</a>, an independent, community-maintained
project separate from Apache Arrow. The drivers remain available and continue
to be developed there. No further releases of those drivers will come from
this project, but existing packages will remain available, possibly as
archived listings. For background, see <a href="https://arrow.apache.org/adbc/24/driver/authoring.html#why-the-driver-foundry-not-this-repository">the ADBC documentation on driver
development</a>.
Current driver locations and maintainer information are listed under
<a href="https://arrow.apache.org/adbc/24/driver/index.html">Drivers</a>.</p>
<p>Similarly, the experimental, incomplete support for Amazon Redshift in the
PostgreSQL driver has been removed. A dedicated <a href="https://adbc-drivers.org/drivers/redshift/">ADBC driver for Amazon
Redshift</a> is available from the
ADBC Driver Foundry.</p>
<p>The ADBC drivers for Apache Arrow Flight SQL, PostgreSQL, and SQLite continue
to be maintained and released by this project.</p>
<p>Java API definitions were narrowed so that <code>close</code> is only declared to throw
<code>AdbcException</code> as a checked
exception. (<a href="https://github.com/apache/arrow-adbc/pull/4451">#4451</a>)</p>
<p>The PostgreSQL driver now lazily initializes transactions to make it work
better with connection
pools. (<a href="https://github.com/apache/arrow-adbc/pull/4424">#4424</a>)</p>
<h3>Documentation</h3>
<p>The documentation has been overhauled and is now organized around three common
tasks: <a href="https://arrow.apache.org/adbc/24/driver/index.html">finding and installing
drivers</a>, <a href="https://arrow.apache.org/adbc/24/client_libraries.html">connecting
through a client
library</a>, and
<a href="https://arrow.apache.org/adbc/24/driver/authoring.html">developing a
driver</a>.
(<a href="https://github.com/apache/arrow-adbc/pull/4525">#4525</a>)</p>
<p>The revised docs emphasize ADBC's cross-language driver model: drivers are
typically distributed as shared libraries that can be used from any supported
language or client.</p>
<p>A new <a href="https://arrow.apache.org/adbc/24/integrations.html">Tools &amp;
Integrations</a> page
highlights tools and frameworks that integrate with ADBC. We welcome
contributions to add more integrations.</p>
<p>A new <a href="https://arrow.apache.org/adbc/24/connection_profiles.html">Connection
Profiles</a> page shows
how to use reusable profiles to configure ADBC connections.</p>
<p>The docs also describe how most drivers now recognize URIs with schemes
matching their names, and how <a href="https://arrow.apache.org/adbc/24/format/driver_manifests.html#resolving-a-driver-from-a-connection-uri">driver managers can use the scheme to resolve
the
driver</a>
when it is not otherwise specified.</p>
<h3>Core APIs &amp; Client Libraries</h3>
<p>C# now has a native driver manager, allowing it to load drivers, manifests,
and profiles (<a href="https://github.com/apache/arrow-adbc/pull/4075">#4075</a>,
<a href="https://github.com/apache/arrow-adbc/pull/4340">#4340</a>,
<a href="https://github.com/apache/arrow-adbc/pull/4341">#4341</a>,
<a href="https://github.com/apache/arrow-adbc/pull/4330">#4330</a>). The core libraries
are now compatible with AOT compilation, enabling drivers to be built as
standalone shared libaries
(<a href="https://github.com/apache/arrow-adbc/pull/4243">#4243</a>,
<a href="https://github.com/apache/arrow-adbc/pull/4318">#4318</a>).</p>
<p>The Go <code>database/sql</code> adapter now supports converting more Arrow types to Go
types (<a href="https://github.com/apache/arrow-adbc/pull/4416">#4416</a>).</p>
<p>The Java core APIs now support a &quot;fluent&quot; style ingest API (#4466). Also,
support for dynamically loading drivers for use in Java has been greatly
expanded and should now support all of the ADBC APIs and features expected of
a client library (<a href="https://github.com/apache/arrow-adbc/pull/4452">#4452</a>,
<a href="https://github.com/apache/arrow-adbc/pull/4211">#4211</a>,
<a href="https://github.com/apache/arrow-adbc/pull/4411">#4411</a>,
<a href="https://github.com/apache/arrow-adbc/pull/4202">#4202</a>,
<a href="https://github.com/apache/arrow-adbc/pull/4359">#4359</a>,
<a href="https://github.com/apache/arrow-adbc/pull/4212">#4212</a>,
<a href="https://github.com/apache/arrow-adbc/pull/4263">#4263</a>,
<a href="https://github.com/apache/arrow-adbc/pull/4229">#4229</a>,
<a href="https://github.com/apache/arrow-adbc/pull/4203">#4203</a>,
<a href="https://github.com/apache/arrow-adbc/pull/4361">#4361</a>,
<a href="https://github.com/apache/arrow-adbc/pull/4362">#4362</a>,
<a href="https://github.com/apache/arrow-adbc/pull/4249">#4249</a>,
<a href="https://github.com/apache/arrow-adbc/pull/4250">#4250</a>,
<a href="https://github.com/apache/arrow-adbc/pull/4398">#4398</a>,
<a href="https://github.com/apache/arrow-adbc/pull/4397">#4397</a>,
<a href="https://github.com/apache/arrow-adbc/pull/4423">#4423</a>,
<a href="https://github.com/apache/arrow-adbc/pull/4395">#4395</a>,
<a href="https://github.com/apache/arrow-adbc/pull/4396">#4396</a>,
<a href="https://github.com/apache/arrow-adbc/pull/4391">#4391</a>).</p>
<p>The JavaScript client library no longer requires the <code>driver</code> parameter and
can infer the driver to load based on the URI, or can accept a profile
(<a href="https://github.com/apache/arrow-adbc/pull/4357">#4357</a>).</p>
<p>The R client library no longer requires the <code>driver</code> parameter and can infer
the driver to load based on the URI, or can accept a profile
(<a href="https://github.com/apache/arrow-adbc/pull/4535">#4535</a>).</p>
<p>Some tweaks have been made to the Rust core APIs to better support interop
with dynamically loaded drivers and make certain conventions clearer
(<a href="https://github.com/apache/arrow-adbc/pull/4427">#4427</a>,
<a href="https://github.com/apache/arrow-adbc/pull/4510">#4510</a>,
<a href="https://github.com/apache/arrow-adbc/pull/4350">#4350</a>,
<a href="https://github.com/apache/arrow-adbc/pull/4141">#4141</a>,
<a href="https://github.com/apache/arrow-adbc/pull/4181">#4181</a>,
<a href="https://github.com/apache/arrow-adbc/pull/4473">#4473</a>,
<a href="https://github.com/apache/arrow-adbc/pull/4469">#4469</a>).</p>
<h3>Drivers</h3>
<p>This project continues to maintain and release the ADBC drivers for Apache
Arrow Flight SQL, PostgreSQL, and SQLite. As mentioned above, the DataFusion,
BigQuery, Databricks, and Snowflake drivers are now maintained in the <a href="https://adbc-drivers.org/">ADBC
Driver Foundry</a>, alongside many others.</p>
<p>The Flight SQL driver now recognizes URIs with the <code>flightsql://</code> scheme
(<a href="https://github.com/apache/arrow-adbc/pull/4488">#4488</a>). It also has more
support for logging and OpenTelemetry tracing
(<a href="https://github.com/apache/arrow-adbc/pull/4322">#4322</a>,
<a href="https://github.com/apache/arrow-adbc/pull/4486">#4486</a>).</p>
<p>The PostgreSQL driver now uses libpq version 18.4 (up from 16.9)
(<a href="https://github.com/apache/arrow-adbc/pull/4566">#4566</a>). Several bugs have
been fixed around handling of the NUMERIC type, and it has been optimized on
platforms where int128 is available (generally, platforms other than Windows)
(<a href="https://github.com/apache/arrow-adbc/pull/4536">#4536</a>,
<a href="https://github.com/apache/arrow-adbc/pull/4499">#4499</a>,
<a href="https://github.com/apache/arrow-adbc/pull/4523">#4523</a>,
<a href="https://github.com/apache/arrow-adbc/pull/4498">#4498</a>). GetObjects now
populates the <code>xdbc_type_name</code> field
(<a href="https://github.com/apache/arrow-adbc/pull/4457">#4457</a>). JSON columns are
now returned with the <code>arrow.json</code> extension type
(<a href="https://github.com/apache/arrow-adbc/pull/4415">#4415</a>), and ingesting into
JSONB columns is now supported
(<a href="https://github.com/apache/arrow-adbc/pull/4505">#4505</a>).</p>
<p>The SQLite driver now uses SQLite version 3.53.1 (up from 3.51.2)
(<a href="https://github.com/apache/arrow-adbc/pull/4566">#4566</a>). It now recognizes
URIs with the <code>sqlite://</code> scheme
(<a href="https://github.com/apache/arrow-adbc/pull/4463">#4463</a>).</p>
<p><a id="contributors"></a></p>
<h2>Contributors</h2>
<pre><code>$ git shortlog --perl-regexp --author='^((?!dependabot\[bot\]).*)$' -sn apache-arrow-adbc-23..apache-arrow-adbc-24
    58	David Li
    16	Bryce Mecum
    11	Fredrik Fornwall
     7	Curt Hagenlocher
     7	eitsupi
     6	Mandukhai Alimaa
     5	davidhcoe
     4	Matt Topol
     4	takuya kodama
     3	Ian Cook
     3	복준수
     2	Artur Rakhmatulin
     2	Austin Bonander
     2	Bruce Irschick
     2	Daniel_McBride
     2	Emil Sadek
     1	Arnold Wakim
     1	Aurélien Pupier
     1	Dan Liu
     1	Felipe Oliveira Carvalho
     1	Kent Wu
     1	Neal Richardson
     1	Nir Portal
     1	Pavel Agafonov
     1	Rishav Rungta
     1	Shubham Pandey
     1	mete
     1	xinyu.lin
</code></pre>
<h2>Roadmap</h2>
<p>We are working on the next revision of the API standard, focusing on missing
features (primarily metadata/catalog data). We welcome anyone interested in
contributing. Current progress can be found in the <a href="https://github.com/apache/arrow-adbc/milestone/9">1.2.0 specification
milestone</a>.</p>
<h2>Getting Involved</h2>
<p>We welcome questions and contributions from all interested.  Issues
can be filed on <a href="https://github.com/apache/arrow-adbc/issues">GitHub</a>, and questions can be directed to GitHub
or the <a href="/community/">Arrow mailing lists</a>.</p>]]></content><author><name>pmc</name></author><category term="release" /><summary type="html"><![CDATA[The Apache Arrow team is pleased to announce the version 24 release of the Apache Arrow ADBC libraries. This release includes 57 resolved issues and 142 merged pull requests from 28 distinct contributors. This is a release of the libraries, which are at version 24. The API specification is versioned separately and is at version 1.1.0. The subcomponents are versioned independently: C/C++/GLib/Go/Python/Ruby: 1.12.0 C#: 0.24.0 Java: 0.24.0 JavaScript: 0.24.0 R: 0.24.0 Rust: 0.24.0 The release notes below are not exhaustive and only expose selected highlights of the release. Many other bugfixes and improvements have been made: we refer you to the complete changelog. Release Highlights Note: we are planning to require C++20 starting from the next release. Also, we will drop support for Python 3.10 no earlier than the release after next i.e. no earlier than release 26. Release 26 is expected in about 3-4 months (October~November 2026), and the EOL for Python 3.10 is October 2026. Breaking Changes and Deprecations Development of the ADBC drivers for Apache DataFusion, BigQuery, Databricks, and Snowflake has moved to the ADBC Driver Foundry, an independent, community-maintained project separate from Apache Arrow. The drivers remain available and continue to be developed there. No further releases of those drivers will come from this project, but existing packages will remain available, possibly as archived listings. For background, see the ADBC documentation on driver development. Current driver locations and maintainer information are listed under Drivers. Similarly, the experimental, incomplete support for Amazon Redshift in the PostgreSQL driver has been removed. A dedicated ADBC driver for Amazon Redshift is available from the ADBC Driver Foundry. The ADBC drivers for Apache Arrow Flight SQL, PostgreSQL, and SQLite continue to be maintained and released by this project. Java API definitions were narrowed so that close is only declared to throw AdbcException as a checked exception. (#4451) The PostgreSQL driver now lazily initializes transactions to make it work better with connection pools. (#4424) Documentation The documentation has been overhauled and is now organized around three common tasks: finding and installing drivers, connecting through a client library, and developing a driver. (#4525) The revised docs emphasize ADBC's cross-language driver model: drivers are typically distributed as shared libraries that can be used from any supported language or client. A new Tools &amp; Integrations page highlights tools and frameworks that integrate with ADBC. We welcome contributions to add more integrations. A new Connection Profiles page shows how to use reusable profiles to configure ADBC connections. The docs also describe how most drivers now recognize URIs with schemes matching their names, and how driver managers can use the scheme to resolve the driver when it is not otherwise specified. Core APIs &amp; Client Libraries C# now has a native driver manager, allowing it to load drivers, manifests, and profiles (#4075, #4340, #4341, #4330). The core libraries are now compatible with AOT compilation, enabling drivers to be built as standalone shared libaries (#4243, #4318). The Go database/sql adapter now supports converting more Arrow types to Go types (#4416). The Java core APIs now support a &quot;fluent&quot; style ingest API (#4466). Also, support for dynamically loading drivers for use in Java has been greatly expanded and should now support all of the ADBC APIs and features expected of a client library (#4452, #4211, #4411, #4202, #4359, #4212, #4263, #4229, #4203, #4361, #4362, #4249, #4250, #4398, #4397, #4423, #4395, #4396, #4391). The JavaScript client library no longer requires the driver parameter and can infer the driver to load based on the URI, or can accept a profile (#4357). The R client library no longer requires the driver parameter and can infer the driver to load based on the URI, or can accept a profile (#4535). Some tweaks have been made to the Rust core APIs to better support interop with dynamically loaded drivers and make certain conventions clearer (#4427, #4510, #4350, #4141, #4181, #4473, #4469). Drivers This project continues to maintain and release the ADBC drivers for Apache Arrow Flight SQL, PostgreSQL, and SQLite. As mentioned above, the DataFusion, BigQuery, Databricks, and Snowflake drivers are now maintained in the ADBC Driver Foundry, alongside many others. The Flight SQL driver now recognizes URIs with the flightsql:// scheme (#4488). It also has more support for logging and OpenTelemetry tracing (#4322, #4486). The PostgreSQL driver now uses libpq version 18.4 (up from 16.9) (#4566). Several bugs have been fixed around handling of the NUMERIC type, and it has been optimized on platforms where int128 is available (generally, platforms other than Windows) (#4536, #4499, #4523, #4498). GetObjects now populates the xdbc_type_name field (#4457). JSON columns are now returned with the arrow.json extension type (#4415), and ingesting into JSONB columns is now supported (#4505). The SQLite driver now uses SQLite version 3.53.1 (up from 3.51.2) (#4566). It now recognizes URIs with the sqlite:// scheme (#4463). Contributors $ git shortlog --perl-regexp --author='^((?!dependabot\[bot\]).*)$' -sn apache-arrow-adbc-23..apache-arrow-adbc-24 58 David Li 16 Bryce Mecum 11 Fredrik Fornwall 7 Curt Hagenlocher 7 eitsupi 6 Mandukhai Alimaa 5 davidhcoe 4 Matt Topol 4 takuya kodama 3 Ian Cook 3 복준수 2 Artur Rakhmatulin 2 Austin Bonander 2 Bruce Irschick 2 Daniel_McBride 2 Emil Sadek 1 Arnold Wakim 1 Aurélien Pupier 1 Dan Liu 1 Felipe Oliveira Carvalho 1 Kent Wu 1 Neal Richardson 1 Nir Portal 1 Pavel Agafonov 1 Rishav Rungta 1 Shubham Pandey 1 mete 1 xinyu.lin Roadmap We are working on the next revision of the API standard, focusing on missing features (primarily metadata/catalog data). We welcome anyone interested in contributing. Current progress can be found in the 1.2.0 specification milestone. Getting Involved We welcome questions and contributions from all interested. Issues can be filed on GitHub, and questions can be directed to GitHub or the Arrow mailing lists.]]></summary><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://arrow.apache.org/img/arrow-logo_horizontal_black-txt_white-bg.png" /><media:content medium="image" url="https://arrow.apache.org/img/arrow-logo_horizontal_black-txt_white-bg.png" xmlns:media="http://search.yahoo.com/mrss/" /></entry><entry><title type="html">Apache Arrow Go 18.7.0 Release</title><link href="https://arrow.apache.org/blog/2026/07/21/arrow-go-18.7.0/" rel="alternate" type="text/html" title="Apache Arrow Go 18.7.0 Release" /><published>2026-07-21T00:00:00-04:00</published><updated>2026-07-21T00:00:00-04:00</updated><id>https://arrow.apache.org/blog/2026/07/21/arrow-go-18.7.0</id><content type="html" xml:base="https://arrow.apache.org/blog/2026/07/21/arrow-go-18.7.0/"><![CDATA[<!--

-->
<p>The Apache Arrow team is pleased to announce the v18.7.0 release of Apache Arrow Go.
This minor release covers 101 commits from 23 distinct contributors.</p>
<h2>Contributors</h2>
<pre><code class="language-console">$ git shortlog -sn v18.6.0..v18.7.0
    52	Minh Vu
    21	Matt Topol
     3	Fredrik Fornwall
     3	Sai Asish Y
     2	Jared Yu (余启正)
     2	Neelesh Salian
     2	Ondřej Pavela
     2	abir
     1	Alex Dubov
     1	Bryce Mecum
     1	David Li
     1	David Zhao
     1	Dima Kuznetsov
     1	Lucas Valente
     1	Mahdi Dibaiee
     1	Matan Rosenberg
     1	Patzifist
     1	Ruihao Chen
     1	Samuel Arnold
     1	Tobias Pütz
     1	Truffle
     1	Varun Venkatesh
     1	daniel-adam-tfs
</code></pre>
<h2>Highlights</h2>
<h3>Arrow</h3>
<h4>Features</h4>
<ul>
<li>Support casts between binary/utf8 and binary_view/utf8_view <a href="https://github.com/apache/arrow-go/pull/802">#802</a></li>
<li>Added a Resize method to RecordBuilder <a href="https://github.com/apache/arrow-go/pull/805">#805</a></li>
<li>Support dictionary arrays for Filter and Take <a href="https://github.com/apache/arrow-go/pull/716">#716</a></li>
<li>Add an UnshredVariant method for collapsing a shredded variant array <a href="https://github.com/apache/arrow-go/pull/859">#859</a></li>
<li>Add implementation of <code>TimestampWithOffset</code> canonical extension type <a href="https://github.com/apache/arrow-go/pull/558">#558</a></li>
</ul>
<h4>Bug Fixes</h4>
<ul>
<li>Made C Data interface stream memory release deterministic <a href="https://github.com/apache/arrow-go/pull/793">#793</a></li>
<li>Preserve large integer precision in JSON decoding <a href="https://github.com/apache/arrow-go/pull/816">#816</a></li>
</ul>
<h3>Parquet</h3>
<h4>Features</h4>
<ul>
<li>Support writing LARGE_LIST arrays <a href="https://github.com/apache/arrow-go/pull/838">#838</a></li>
<li>Opt-in streaming reads for large data pages <a href="https://github.com/apache/arrow-go/pull/880">#880</a></li>
</ul>
<h4>Bug Fixes</h4>
<ul>
<li>Corrected a couple variant bugs <a href="https://github.com/apache/arrow-go/pull/841">#841</a>, <a href="https://github.com/apache/arrow-go/pull/840">#840</a>, <a href="https://github.com/apache/arrow-go/pull/939">#939</a></li>
<li>Properly align DataPageV2 pages to row boundaries in spaced writes <a href="https://github.com/apache/arrow-go/pull/883">#883</a></li>
</ul>
<h4>Performance Improvements</h4>
<ul>
<li>Added SIMD-accelerated byte-stream-split decoding <a href="https://github.com/apache/arrow-go/pull/654">#654</a></li>
<li>Avoid constructing ColunChunkMetaData in page index range determination <a href="https://github.com/apache/arrow-go/pull/831">#831</a></li>
<li>Improve BloomFilter memory recycling and pooling <a href="https://github.com/apache/arrow-go/pull/864">#864</a></li>
</ul>
<h2>Changelog</h2>
<h3>What's Changed</h3>
<ul>
<li>fix(release): keys download by @zeroshade in <a href="https://github.com/apache/arrow-go/pull/781">#781</a></li>
<li>parquet/compress: enable WithAllLitEntropyCompression(true) for zstd by @varun0630 in <a href="https://github.com/apache/arrow-go/pull/779">#779</a></li>
<li>release: fix syntax error in verify_rc.sh by @amoeba in <a href="https://github.com/apache/arrow-go/pull/782">#782</a></li>
<li>fix(parquet): align dictionary fallback with parquet-mr by @twuebi in <a href="https://github.com/apache/arrow-go/pull/786">#786</a></li>
<li>fix(arrow/cdata): make nativeCRecordBatchReader deterministic by @zeroshade in <a href="https://github.com/apache/arrow-go/pull/793">#793</a></li>
<li>fix(compute/kernels): map FSB byte-width to numeric memo type for is_in by @SAY-5 in <a href="https://github.com/apache/arrow-go/pull/797">#797</a></li>
<li>fix(arrow/flight): deliver response headers eagerly in streaming client middleware by @zeroshade in <a href="https://github.com/apache/arrow-go/pull/801">#801</a></li>
<li>fix(ipc): preserve map child field metadata by @dimakuz in <a href="https://github.com/apache/arrow-go/pull/809">#809</a></li>
<li>fix(avro): error on complex (non-nullable) unions instead of silently dropping (#777) by @SAY-5 in <a href="https://github.com/apache/arrow-go/pull/808">#808</a></li>
<li>fix(parquet/metadata): do not pre-set hasDistinctCount in stat factories by @SAY-5 in <a href="https://github.com/apache/arrow-go/pull/807">#807</a></li>
<li>feat(compute): support casts between binary/string and binary_view/string_view by @zeroshade in <a href="https://github.com/apache/arrow-go/pull/802">#802</a></li>
<li>fix(arrow/array): preserve large integer precision in JSON decoding by @zeroshade in <a href="https://github.com/apache/arrow-go/pull/816">#816</a></li>
<li>feat(arrow/array): add Resize method to RecordBuilder by @oakad in <a href="https://github.com/apache/arrow-go/pull/805">#805</a></li>
<li>perf(parquet/pqarrow): cap RecordReader batch size to actual row count by @paveon in <a href="https://github.com/apache/arrow-go/pull/817">#817</a></li>
<li>docs(parquet): correct DefaultDataPageSize comment from 1K to 1M by @zeroshade in <a href="https://github.com/apache/arrow-go/pull/822">#822</a></li>
<li>fix(arrow/array): silence copylocks warning in numeric_generic.go by @zeroshade in <a href="https://github.com/apache/arrow-go/pull/823">#823</a></li>
<li>feat(arrow/compute/exprs): support expr.IntervalYearToMonthLiteral in literalToDatum by @zeroshade in <a href="https://github.com/apache/arrow-go/pull/825">#825</a></li>
<li>fix(parquet): return error instead of panicking on first-write failure by @zeroshade in <a href="https://github.com/apache/arrow-go/pull/824">#824</a></li>
<li>[Parquet] Add SIMD-accelerated byte-stream-split decoding by @daniel-adam-tfs in <a href="https://github.com/apache/arrow-go/pull/654">#654</a></li>
<li>perf(parquet/metadata): avoid constructing ColumnChunkMetaData in page index range determination by @paveon in <a href="https://github.com/apache/arrow-go/pull/831">#831</a></li>
<li>feat: filter and take for dictionary arrays by @lesam in <a href="https://github.com/apache/arrow-go/pull/716">#716</a></li>
<li>fix(arrow/array): adding string boundary checking for (*String).Value by @happydave1 in <a href="https://github.com/apache/arrow-go/pull/844">#844</a></li>
<li>fix(parquet/variant): correct binary search bounds in ObjectValue.ValueByKey by @qzyu999 in <a href="https://github.com/apache/arrow-go/pull/841">#841</a></li>
<li>fix(parquet/variant): correct is_large bit position in valueSize for arrays by @qzyu999 in <a href="https://github.com/apache/arrow-go/pull/840">#840</a></li>
<li>feat(avro): support local-timestamp logical types by @abir-eon in <a href="https://github.com/apache/arrow-go/pull/832">#832</a></li>
<li>fix(avro): append raw bytes by @matan129 in <a href="https://github.com/apache/arrow-go/pull/850">#850</a></li>
<li>fix(arrow/array): use scale-aware ValueStr in decimal array String() by @zeroshade in <a href="https://github.com/apache/arrow-go/pull/849">#849</a></li>
<li>feat(parquet/pqarrow): support writing LARGE_LIST types by @lidavidm in <a href="https://github.com/apache/arrow-go/pull/838">#838</a></li>
<li>fix(arrow/scalar): implement Release()/Retain() on *scalar.Extension by @zeroshade in <a href="https://github.com/apache/arrow-go/pull/851">#851</a></li>
<li>docs(arrow/ipc): document that Writer and FileWriter are not concurrency safe by @zeroshade in <a href="https://github.com/apache/arrow-go/pull/856">#856</a></li>
<li>fix(parquet): add WriteBatchSpacedWithError to surface spaced-write failures by @zeroshade in <a href="https://github.com/apache/arrow-go/pull/852">#852</a></li>
<li>ci: pin Go to go.mod in RC verify so the matching toolchain is used by @zeroshade in <a href="https://github.com/apache/arrow-go/pull/857">#857</a></li>
<li>feat(arrow/extensions): add UnshredVariant to collapse a shredded variant array by @nssalian in <a href="https://github.com/apache/arrow-go/pull/859">#859</a></li>
<li>fix(arrow/compute): accept type max value in safe decimal-to-int cast by @zeroshade in <a href="https://github.com/apache/arrow-go/pull/862">#862</a></li>
<li>fix(array): validate negative bounds in NewSliceData by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/870">#870</a></li>
<li>fix(array): dictionary null index zeroing by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/871">#871</a></li>
<li>fix(array): struct stringer required child mask by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/873">#873</a></li>
<li>fix(array): empty chunked approximate equality by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/874">#874</a></li>
<li>fix(array): validate dense union offset lengths by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/875">#875</a></li>
<li>Use semantic type equality in AddColumn by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/878">#878</a></li>
<li>fix(array): clear dictionary data on reset by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/876">#876</a></li>
<li>Reject negative chunked slice bounds by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/879">#879</a></li>
<li>fix(array): array data hashing coverage by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/872">#872</a></li>
<li>Support dense union slice offsets by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/877">#877</a></li>
<li>Support sparse union sliced type ids by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/881">#881</a></li>
<li>fix(build): correct phony target declaration by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/890">#890</a></li>
<li>fix(parquet/compress): Gzip CompressBound by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/886">#886</a></li>
<li>fix(parquet): free excluded stats bytes in ApplyStatSizeLimits by @mdibaiee in <a href="https://github.com/apache/arrow-go/pull/885">#885</a></li>
<li>fix(tmpl): preserve escaped JSON string bytes by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/889">#889</a></li>
<li>fix(parquet/compress): Brotli CompressBound by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/884">#884</a></li>
<li>fix(parquet): align DataPageV2 pages to row boundaries in spaced writes by @truffle-dev in <a href="https://github.com/apache/arrow-go/pull/883">#883</a></li>
<li>fix(array): guard BinaryViewBuilder size checks by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/888">#888</a></li>
<li>fix: clarify StringViewBuilder UnmarshalJSON error message by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/894">#894</a></li>
<li>fix: clarify StringViewBuilder.UnmarshalOne expected type by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/897">#897</a></li>
<li>fix: guard multiBufferBuilder UnsafeAppend against short copy by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/896">#896</a></li>
<li>fix(array): make SetNull idempotent and track null count by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/905">#905</a></li>
<li>fix(ci): accept --json and pin gobenchdata by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/906">#906</a></li>
<li>build: verify Apache RAT jar checksum before execution by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/908">#908</a></li>
<li>fix(parquet): read full bloom filter buffers by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/892">#892</a></li>
<li>fix(docs): fix CPU markdown link in README by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/907">#907</a></li>
<li>fix(parquet): validate DataPageV2 decoded length by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/911">#911</a></li>
<li>fix(arreflect): validate primitive conversion assignment overflow checks by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/903">#903</a></li>
<li>fix(parquet): clamp repLevels in byte-array DataPageV2 row-boundary alignment by @zeroshade in <a href="https://github.com/apache/arrow-go/pull/919">#919</a></li>
<li>fix(parquet/metadata): fix BloomFilter memory recycling, doc contract, and benchmark by @Patzifist in <a href="https://github.com/apache/arrow-go/pull/864">#864</a></li>
<li>fix(flight/flightsql): recover real error when prepared DoPut stream closes early by @fornwall in <a href="https://github.com/apache/arrow-go/pull/920">#920</a></li>
<li>fix: guard protobuf oneof/union reflection on nil oneof values by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/910">#910</a></li>
<li>fix(parquet): copy ByteArray statistics min/max to prevent use-after-free by @zeroshade in <a href="https://github.com/apache/arrow-go/pull/917">#917</a></li>
<li>fix(array): bounds check FixedSizeBinary.Value by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/923">#923</a></li>
<li>fix(array): reject unsupported type IDs by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/925">#925</a></li>
<li>fix(array): validate JSON start offsets by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/929">#929</a></li>
<li>fix(scalar): validate dictionary indices before conversion by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/927">#927</a></li>
<li>fix(array): retain dictionary before replacement by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/926">#926</a></li>
<li>feat(parquet): opt-in streaming reads for large data pages by @joechenrh in <a href="https://github.com/apache/arrow-go/pull/880">#880</a></li>
<li>fix(parquet): propagate row-group and page-index write errors by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/921">#921</a></li>
<li>fix(array): honor struct offsets in constructors and validity masks by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/922">#922</a></li>
<li>fix(parquet): make column reader cleanup idempotent by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/924">#924</a></li>
<li>fix(scalar): hash list values logically by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/928">#928</a></li>
<li>fix(array): recursively validate nested arrays by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/930">#930</a></li>
<li>feat(parquet/variant): add Metadata.SizeBytes to split a concatenated variant by @nssalian in <a href="https://github.com/apache/arrow-go/pull/939">#939</a></li>
<li>fix(cdata): validate imported array sizes by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/931">#931</a></li>
<li>fix(avro): decode nullable logical timestamps by @abir-eon in <a href="https://github.com/apache/arrow-go/pull/935">#935</a></li>
<li>fix(flight/flightsql/example): support dictionary-encoded parameter columns by @fornwall in <a href="https://github.com/apache/arrow-go/pull/936">#936</a></li>
<li>fix(arrow/scalar): reject negative indices in GetScalar by @fornwall in <a href="https://github.com/apache/arrow-go/pull/940">#940</a></li>
<li>fix(flight): reject malformed unary bearer headers by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/941">#941</a></li>
<li>fix(flight): reject Basic credentials without a separator by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/942">#942</a></li>
<li>fix(memory): keep allocator ownership across Buffer.Reset by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/947">#947</a></li>
<li>fix(parquet): limit page allocations by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/953">#953</a></li>
<li>fix(parquet): return decompression errors by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/954">#954</a></li>
<li>fix(parquet): return errors for malformed encrypted payloads by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/945">#945</a></li>
<li>fix: repair failing benchmarks and stop bench.sh masking failures by @zeroshade in <a href="https://github.com/apache/arrow-go/pull/938">#938</a></li>
<li>fix(parquet): handle short input in GetBatchBools by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/946">#946</a></li>
<li>fix(arrow): validate timestamp timezones by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/952">#952</a></li>
<li>fix(ipc): validate message framing bounds by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/955">#955</a></li>
<li>fix(parquet): reject truncated bit-packed batches by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/943">#943</a></li>
<li>fix(compute): handle comparison output offsets by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/956">#956</a></li>
<li>fix(arrow/array): avoid allocating for ReserveData(0) by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/895">#895</a></li>
<li>fix(compute): release temporary arrays in binary view casts by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/898">#898</a></li>
<li>fix(arrow/array): validate map builder entry lengths by @fallintoplace in <a href="https://github.com/apache/arrow-go/pull/957">#957</a></li>
<li>Add <code>TimestampWithOffset</code> canonical extension type by @serramatutu in <a href="https://github.com/apache/arrow-go/pull/558">#558</a></li>
<li>chore: bump version number by @zeroshade in <a href="https://github.com/apache/arrow-go/pull/959">#959</a></li>
</ul>
<h3>New Contributors</h3>
<ul>
<li>@varun0630 made their first contribution in <a href="https://github.com/apache/arrow-go/pull/779">#779</a></li>
<li>@twuebi made their first contribution in <a href="https://github.com/apache/arrow-go/pull/786">#786</a></li>
<li>@SAY-5 made their first contribution in <a href="https://github.com/apache/arrow-go/pull/797">#797</a></li>
<li>@oakad made their first contribution in <a href="https://github.com/apache/arrow-go/pull/805">#805</a></li>
<li>@paveon made their first contribution in <a href="https://github.com/apache/arrow-go/pull/817">#817</a></li>
<li>@lesam made their first contribution in <a href="https://github.com/apache/arrow-go/pull/716">#716</a></li>
<li>@happydave1 made their first contribution in <a href="https://github.com/apache/arrow-go/pull/844">#844</a></li>
<li>@qzyu999 made their first contribution in <a href="https://github.com/apache/arrow-go/pull/841">#841</a></li>
<li>@abir-eon made their first contribution in <a href="https://github.com/apache/arrow-go/pull/832">#832</a></li>
<li>@matan129 made their first contribution in <a href="https://github.com/apache/arrow-go/pull/850">#850</a></li>
<li>@nssalian made their first contribution in <a href="https://github.com/apache/arrow-go/pull/859">#859</a></li>
<li>@fallintoplace made their first contribution in <a href="https://github.com/apache/arrow-go/pull/870">#870</a></li>
<li>@mdibaiee made their first contribution in <a href="https://github.com/apache/arrow-go/pull/885">#885</a></li>
<li>@truffle-dev made their first contribution in <a href="https://github.com/apache/arrow-go/pull/883">#883</a></li>
<li>@Patzifist made their first contribution in <a href="https://github.com/apache/arrow-go/pull/864">#864</a></li>
<li>@fornwall made their first contribution in <a href="https://github.com/apache/arrow-go/pull/920">#920</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a href="https://github.com/apache/arrow-go/compare/v18.6.0...v18.7.0">https://github.com/apache/arrow-go/compare/v18.6.0...v18.7.0</a></p>]]></content><author><name>pmc</name></author><category term="release" /><summary type="html"><![CDATA[The Apache Arrow team is pleased to announce the v18.7.0 release of Apache Arrow Go. This minor release covers 101 commits from 23 distinct contributors. Contributors $ git shortlog -sn v18.6.0..v18.7.0 52 Minh Vu 21 Matt Topol 3 Fredrik Fornwall 3 Sai Asish Y 2 Jared Yu (余启正) 2 Neelesh Salian 2 Ondřej Pavela 2 abir 1 Alex Dubov 1 Bryce Mecum 1 David Li 1 David Zhao 1 Dima Kuznetsov 1 Lucas Valente 1 Mahdi Dibaiee 1 Matan Rosenberg 1 Patzifist 1 Ruihao Chen 1 Samuel Arnold 1 Tobias Pütz 1 Truffle 1 Varun Venkatesh 1 daniel-adam-tfs Highlights Arrow Features Support casts between binary/utf8 and binary_view/utf8_view #802 Added a Resize method to RecordBuilder #805 Support dictionary arrays for Filter and Take #716 Add an UnshredVariant method for collapsing a shredded variant array #859 Add implementation of TimestampWithOffset canonical extension type #558 Bug Fixes Made C Data interface stream memory release deterministic #793 Preserve large integer precision in JSON decoding #816 Parquet Features Support writing LARGE_LIST arrays #838 Opt-in streaming reads for large data pages #880 Bug Fixes Corrected a couple variant bugs #841, #840, #939 Properly align DataPageV2 pages to row boundaries in spaced writes #883 Performance Improvements Added SIMD-accelerated byte-stream-split decoding #654 Avoid constructing ColunChunkMetaData in page index range determination #831 Improve BloomFilter memory recycling and pooling #864 Changelog What's Changed fix(release): keys download by @zeroshade in #781 parquet/compress: enable WithAllLitEntropyCompression(true) for zstd by @varun0630 in #779 release: fix syntax error in verify_rc.sh by @amoeba in #782 fix(parquet): align dictionary fallback with parquet-mr by @twuebi in #786 fix(arrow/cdata): make nativeCRecordBatchReader deterministic by @zeroshade in #793 fix(compute/kernels): map FSB byte-width to numeric memo type for is_in by @SAY-5 in #797 fix(arrow/flight): deliver response headers eagerly in streaming client middleware by @zeroshade in #801 fix(ipc): preserve map child field metadata by @dimakuz in #809 fix(avro): error on complex (non-nullable) unions instead of silently dropping (#777) by @SAY-5 in #808 fix(parquet/metadata): do not pre-set hasDistinctCount in stat factories by @SAY-5 in #807 feat(compute): support casts between binary/string and binary_view/string_view by @zeroshade in #802 fix(arrow/array): preserve large integer precision in JSON decoding by @zeroshade in #816 feat(arrow/array): add Resize method to RecordBuilder by @oakad in #805 perf(parquet/pqarrow): cap RecordReader batch size to actual row count by @paveon in #817 docs(parquet): correct DefaultDataPageSize comment from 1K to 1M by @zeroshade in #822 fix(arrow/array): silence copylocks warning in numeric_generic.go by @zeroshade in #823 feat(arrow/compute/exprs): support expr.IntervalYearToMonthLiteral in literalToDatum by @zeroshade in #825 fix(parquet): return error instead of panicking on first-write failure by @zeroshade in #824 [Parquet] Add SIMD-accelerated byte-stream-split decoding by @daniel-adam-tfs in #654 perf(parquet/metadata): avoid constructing ColumnChunkMetaData in page index range determination by @paveon in #831 feat: filter and take for dictionary arrays by @lesam in #716 fix(arrow/array): adding string boundary checking for (*String).Value by @happydave1 in #844 fix(parquet/variant): correct binary search bounds in ObjectValue.ValueByKey by @qzyu999 in #841 fix(parquet/variant): correct is_large bit position in valueSize for arrays by @qzyu999 in #840 feat(avro): support local-timestamp logical types by @abir-eon in #832 fix(avro): append raw bytes by @matan129 in #850 fix(arrow/array): use scale-aware ValueStr in decimal array String() by @zeroshade in #849 feat(parquet/pqarrow): support writing LARGE_LIST types by @lidavidm in #838 fix(arrow/scalar): implement Release()/Retain() on *scalar.Extension by @zeroshade in #851 docs(arrow/ipc): document that Writer and FileWriter are not concurrency safe by @zeroshade in #856 fix(parquet): add WriteBatchSpacedWithError to surface spaced-write failures by @zeroshade in #852 ci: pin Go to go.mod in RC verify so the matching toolchain is used by @zeroshade in #857 feat(arrow/extensions): add UnshredVariant to collapse a shredded variant array by @nssalian in #859 fix(arrow/compute): accept type max value in safe decimal-to-int cast by @zeroshade in #862 fix(array): validate negative bounds in NewSliceData by @fallintoplace in #870 fix(array): dictionary null index zeroing by @fallintoplace in #871 fix(array): struct stringer required child mask by @fallintoplace in #873 fix(array): empty chunked approximate equality by @fallintoplace in #874 fix(array): validate dense union offset lengths by @fallintoplace in #875 Use semantic type equality in AddColumn by @fallintoplace in #878 fix(array): clear dictionary data on reset by @fallintoplace in #876 Reject negative chunked slice bounds by @fallintoplace in #879 fix(array): array data hashing coverage by @fallintoplace in #872 Support dense union slice offsets by @fallintoplace in #877 Support sparse union sliced type ids by @fallintoplace in #881 fix(build): correct phony target declaration by @fallintoplace in #890 fix(parquet/compress): Gzip CompressBound by @fallintoplace in #886 fix(parquet): free excluded stats bytes in ApplyStatSizeLimits by @mdibaiee in #885 fix(tmpl): preserve escaped JSON string bytes by @fallintoplace in #889 fix(parquet/compress): Brotli CompressBound by @fallintoplace in #884 fix(parquet): align DataPageV2 pages to row boundaries in spaced writes by @truffle-dev in #883 fix(array): guard BinaryViewBuilder size checks by @fallintoplace in #888 fix: clarify StringViewBuilder UnmarshalJSON error message by @fallintoplace in #894 fix: clarify StringViewBuilder.UnmarshalOne expected type by @fallintoplace in #897 fix: guard multiBufferBuilder UnsafeAppend against short copy by @fallintoplace in #896 fix(array): make SetNull idempotent and track null count by @fallintoplace in #905 fix(ci): accept --json and pin gobenchdata by @fallintoplace in #906 build: verify Apache RAT jar checksum before execution by @fallintoplace in #908 fix(parquet): read full bloom filter buffers by @fallintoplace in #892 fix(docs): fix CPU markdown link in README by @fallintoplace in #907 fix(parquet): validate DataPageV2 decoded length by @fallintoplace in #911 fix(arreflect): validate primitive conversion assignment overflow checks by @fallintoplace in #903 fix(parquet): clamp repLevels in byte-array DataPageV2 row-boundary alignment by @zeroshade in #919 fix(parquet/metadata): fix BloomFilter memory recycling, doc contract, and benchmark by @Patzifist in #864 fix(flight/flightsql): recover real error when prepared DoPut stream closes early by @fornwall in #920 fix: guard protobuf oneof/union reflection on nil oneof values by @fallintoplace in #910 fix(parquet): copy ByteArray statistics min/max to prevent use-after-free by @zeroshade in #917 fix(array): bounds check FixedSizeBinary.Value by @fallintoplace in #923 fix(array): reject unsupported type IDs by @fallintoplace in #925 fix(array): validate JSON start offsets by @fallintoplace in #929 fix(scalar): validate dictionary indices before conversion by @fallintoplace in #927 fix(array): retain dictionary before replacement by @fallintoplace in #926 feat(parquet): opt-in streaming reads for large data pages by @joechenrh in #880 fix(parquet): propagate row-group and page-index write errors by @fallintoplace in #921 fix(array): honor struct offsets in constructors and validity masks by @fallintoplace in #922 fix(parquet): make column reader cleanup idempotent by @fallintoplace in #924 fix(scalar): hash list values logically by @fallintoplace in #928 fix(array): recursively validate nested arrays by @fallintoplace in #930 feat(parquet/variant): add Metadata.SizeBytes to split a concatenated variant by @nssalian in #939 fix(cdata): validate imported array sizes by @fallintoplace in #931 fix(avro): decode nullable logical timestamps by @abir-eon in #935 fix(flight/flightsql/example): support dictionary-encoded parameter columns by @fornwall in #936 fix(arrow/scalar): reject negative indices in GetScalar by @fornwall in #940 fix(flight): reject malformed unary bearer headers by @fallintoplace in #941 fix(flight): reject Basic credentials without a separator by @fallintoplace in #942 fix(memory): keep allocator ownership across Buffer.Reset by @fallintoplace in #947 fix(parquet): limit page allocations by @fallintoplace in #953 fix(parquet): return decompression errors by @fallintoplace in #954 fix(parquet): return errors for malformed encrypted payloads by @fallintoplace in #945 fix: repair failing benchmarks and stop bench.sh masking failures by @zeroshade in #938 fix(parquet): handle short input in GetBatchBools by @fallintoplace in #946 fix(arrow): validate timestamp timezones by @fallintoplace in #952 fix(ipc): validate message framing bounds by @fallintoplace in #955 fix(parquet): reject truncated bit-packed batches by @fallintoplace in #943 fix(compute): handle comparison output offsets by @fallintoplace in #956 fix(arrow/array): avoid allocating for ReserveData(0) by @fallintoplace in #895 fix(compute): release temporary arrays in binary view casts by @fallintoplace in #898 fix(arrow/array): validate map builder entry lengths by @fallintoplace in #957 Add TimestampWithOffset canonical extension type by @serramatutu in #558 chore: bump version number by @zeroshade in #959 New Contributors @varun0630 made their first contribution in #779 @twuebi made their first contribution in #786 @SAY-5 made their first contribution in #797 @oakad made their first contribution in #805 @paveon made their first contribution in #817 @lesam made their first contribution in #716 @happydave1 made their first contribution in #844 @qzyu999 made their first contribution in #841 @abir-eon made their first contribution in #832 @matan129 made their first contribution in #850 @nssalian made their first contribution in #859 @fallintoplace made their first contribution in #870 @mdibaiee made their first contribution in #885 @truffle-dev made their first contribution in #883 @Patzifist made their first contribution in #864 @fornwall made their first contribution in #920 Full Changelog: https://github.com/apache/arrow-go/compare/v18.6.0...v18.7.0]]></summary><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://arrow.apache.org/img/arrow-logo_horizontal_black-txt_white-bg.png" /><media:content medium="image" url="https://arrow.apache.org/img/arrow-logo_horizontal_black-txt_white-bg.png" xmlns:media="http://search.yahoo.com/mrss/" /></entry></feed>