0.88.0¶
Released 2026-10-08 — GitHub Release
Important breaks¶
None
Changes¶
⚠️ Breaks¶
Speed up scalar construction, stats reads and aggregate partial merging (#10326) @robert3005
break(arrow): add export options and allow skipping view buffer compaction (#10220) @lorenzhs
Blocked
BitPacked: store block offsets as a child (#10004) @mhk197Delta validity is a top level child (#9970) @robert3005
Move users of map_each_with_validity and map_each_in_place to lane kernels and remove those methods (#10139) @robert3005
🚧 Deprecation¶
✨ Features¶
Add support for reading vortex files through native python readers (#10097) @robert3005
Blocked
BitPacked: encode blocked bitwidths (#10203) @mhk197Scan protocol traits (#10047) @joseph-isaacs
Add combine_chunks option to Python Arrow export (#10279) @robert3005
feat(python): support integer column indices in read_url projection (#10287) @jackylee-ch
🚀 Performance¶
perf(array): expand FixedSizeList filter masks directly into a bitmap (#10274) @robert3005
Make opening a Vortex file with a cached footer cheaper (#10358) @joseph-isaacs
perf: classify canonical arrays by concrete vtable (#10238) @joseph-isaacs
Add OnPair take kernel that shares the pair dictionary (#10363) @joseph-isaacs
perf(arrow): preserve source nullability during export (#10365) @joseph-isaacs
Read integer validity a word per chunk, and fill nulls for runs (#10342) @joseph-isaacs
Read VarBin UTF-8 RowFn inputs through their offsets (#10340) @connortsui20
Execute FSST iteratively (#10209) @joseph-isaacs
perf(array): avoid errors when comparison kernels decline arithmetic (#10301) @connortsui20
Faster ScalarFnArray creation, lazy ArrayStats (#10329) @myrrc
perf(buffer): use fearless_simd for popcount (#10327) @joseph-isaacs
Initialise ArrayParts directly with provided slots (#10072) @robert3005
Execute OnPair iteratively (#10210) @joseph-isaacs
perf(compressor): detect constant primitives before generating stats (#10043) @robert3005
perf: vectorize small u8 table take with AVX2 (#9572) @joseph-isaacs
perf: vectorize small u8 table take with NEON (#9571) @joseph-isaacs
Optimise min/max computation for VarBinView arrays (#9740) @robert3005
Execute DecimalByteParts iteratively (#10207) @joseph-isaacs
Execute RunEnd iteratively (#10206) @joseph-isaacs
perf(zstd): reuse one decompression context per thread (#10270) @joseph-isaacs
feat: add between kernel for DecimalByteParts (#10152) @joseph-isaacs
python: release the GIL only around expensive work (#10280) @robert3005
Convert trivial filters into slices during reduction, MaskValues::last uses BitBuffer::last_set_index (#9831) @robert3005
perf(array): forward probe_scalar through pass-through encodings (#9905) @joseph-isaacs
Perform validation and null view replacement in two passes (#10148) @robert3005
🐛 Bug Fixes¶
13 changes
fix(python): translate Polars time literals (#10186) @danking
fix(buffer): require both halves of a Zip to be TrustedLen (#10293) @jianhe25
fix(buffer): validate offsets in Buffer::into_arrow_offset_buffer (#10292) @jianhe25
fix(buffer): don’t create &mut [u64] over uninitialized memory in collect_words_in (#10294) @jianhe25
duckdb: i16 Decimal on duckdb side may fit in Vortex’s i8 decimal buffer (#10155) @myrrc
Keep the file sum absent when a chunk sum overflows (#10328) @connortsui20
Ignore empty constant chunks in aggregate states (#10254) @connortsui20
fix(parquet-variant): decline all_non_distinct when only one side is shredded (#10275) @jackylee-ch
fix(python): preserve nullability when converting a chunked array with nulls (#10283) @jackylee-ch
fix(buffer): enable and detect bmi1 for _tzcnt_u64 in select_in_word (#10295) @jianhe25
fix(array): preserve result nullability when simplifying and/or/zip (#10286) @jackylee-ch
fix(datafusion): preserve list element field metadata in physical schema (#10285) @jackylee-ch
🧰 Maintenance¶
14 changes
Skip validation when accessing storage scalar of extension scalar (#9875) @robert3005
Validate TPC-DS results against Vortex and Parquet (#9863) @joseph-isaacs
Test IO file sizes in Rust and array sizes in Python doctests (#9792) @robert3005
chore(array): use fearless_simd for AVX2 1- and 2-byte filter compress (#10324) @joseph-isaacs
Add sum oracle to the fuzzer and reject samples that have order dependent overflow behaviour (#9801) @robert3005
Do not build a Sequence array for slices that are not sequences (#10338) @robert3005
perf(buffer): portable fearless_simd kernels for bitmap rank-select (#10319) @joseph-isaacs
fix: reconcile asyncband references in Cargo.lock (#10306) @joseph-isaacs
Fix deprecations flagged by cargo build on new rust version (#10042) @robert3005
Add list_contains membership benchmarks (#10060) @robert3005
Clean up
vortex_ensure_eq!(#10237) @connortsui20Lock file maintenance (#10290) @renovate[bot]
Split SQL PR benchmarks into bench-sql and bench-sql-extended (#10213) @joseph-isaacs