October 2026

Rusty drove a substantial upgrade across the VGI protocol ecosystem, coordinating multiple language implementations and tooling toward v2.0 and v3 compatibility standards.

VGI Language Implementations

Across vgi-java, vgi-cpp, vgi-rust, vgi-csharp, and vgi-go, Rusty synchronized CI pipelines to run against the shared v2.0 test suite (commits dc0cab9 and 39ff88d), ensuring consistent validation across all language bindings. This was paired with two dependency-bump PRs: opened PR #12 on vgi-go and PR #13 on vgi-java to advance Haybarn 2.0 clients from rc8 to rc9, bringing both the engine APIs and HAYBARN_ABI_VERSION 2 support.

Protocol and Validation Work

The vgi-cpp implementation was updated to hold every served table to the v3 ATTACH contract while relaxing nullability constraints, reflecting a refinement in how the protocol handles type safety. Over on vgi-lint-check, a major release cycle completed—bumped to 2.0.0 and configured to run on haybarn 2.0 infrastructure (vgi.v3 workers). The linter now accepts vgi.v3 required-filter paths and DuckDB 2.0 macro arguments, broadening its coverage of the evolving specification.

In vgi-c, Rusty implemented full vgi.v3 protocol support with canonical constraint records, negotiated read profiles, and ATTACH validation checks. The canonical integration suite now runs in CI with comprehensive documentation of v3 validation requirements.

RPC and Telemetry

vgi-rpc-c was updated to follow vgi-rpc 0.52.3, adopting Identity.v2 semantics with timestamp-based grant expiry, while the reference Python client was pinned to v0.52.3 for consistency.

The query-farm-telemetry-client gained new capabilities with QueryFarmSendEventJson, enabling extensions to define and send custom events.

Open Source Community Work

Rusty opened PR #28148 on duckdb/duckdb to add ExtensionCallback::OnParserError, allowing extensions to annotate parser errors and explain callable object parameters during binding—addressing a gap where extensions previously had no hook when SQL parsing failed. He also provided feedback on PR #28120 (WebAssembly idle thread sleep optimization).

Private repositories received 9 commits across 4 repos, reflecting additional ongoing work in the ecosystem.

A productive day focused on upgrading the VGI projects to Haybarn 2.0 and addressing several DuckDB core issues around filter handling and virtual method overrides.

query-farm-astro received two blog-related updates: the /blog index now displays each post's featured figure, and a new post on RapidFuzz for DuckDB 2.0 was published.

vgi-go got a major engine upgrade in PR #11. The embedded DuckDB bindings moved from duckdb-go v2.10505.0 (DuckDB 1.5.5) to the new haybarn-go v2.20000.0-rc.8, which bundles Haybarn 2.0's expression-filter evaluation engine. This brings the entire stack to DuckDB 2.0 compatibility.

Similarly, vgi-java saw PR #12 upgrade its example worker's embedded engine. The ExpressionFilterEvaluator moved from haybarn_jdbc:1.5.4-rc1 to 2.0.0-rc8, aligning the project with Haybarn 2.0 and DuckDB 2.0.

On the DuckDB side, Rusty opened three interconnected PRs addressing filter handling and polymorphism:

• PR #28088 (duckdb) adds a virtual HTTPParams::Clone() method to support proper copying of polymorphic HTTP parameters without slicing, enabling subclasses like HTTPFSParams to override it safely.
• PR #512 (duckdb-httpfs) marks HTTPFSParams::Clone() as override now that the base method will be virtual — a companion fix ensuring type safety.
• PR #28090 (duckdb) lets table functions opt out of Bloom and prefix-range join filters. When a table function can't evaluate these filters locally (since they operate on engine memory), it should still receive min/max bounds to pass downstream.

Rusty also opened issue #28091 documenting a regression: hash joins skip min/max bounds to table functions when choosing a prefix-range filter, a behavior change from v1.5. This complements PR #28089 (duckdb), which adds TableCatalogEntry::GetDeleteReturningVirtualColumns() to distinguish between columns a scan can produce versus those available in DELETE RETURNING contexts—an issue surfaced by storage extensions.

Private repository activity included 2 commits across 2 repos.

A prolific day across the Query Farm ecosystem, with 91 commits spanning 15 repositories and 26 PRs opened. Work centered on releasing protocol v3 updates across multiple language bindings and advancing core data structure and statistical computing libraries.

VGI Protocol v3 Release Cycle

The VGI (Vector Graphics Interface) family of repositories received significant updates to align with protocol version 3. In vgi-java, Rusty released version 0.45.0 with wire 3.0 support and published RPC dependencies. He corrected Javadoc parameter names and exercised SHM (shared memory) pool ownership in subprocess CI lanes using the native vgi.v3 runner. Similar v3 rollout work occurred in vgi-cpp (released 0.13.0), vgi-rust (merged release PR #12 for 0.48.0), vgi-go (merged PR #10 for 0.38.0), and vgi-csharp (released 0.22.0). Across these language bindings, he addressed SHM pool lifetime safety, authenticated call contexts, and FlatBuffers producer fixes for strict Arrow IPC readers.

Stochastic Distribution Library Expansion

The stochastic repository received extensive enhancements, with commits adding Dirichlet and multinomial distributions, 12 distributions from Boost.Math, and a unified spec-based architecture for registration. He refactored the codebase to compile as C++17 and added fitting and hypothesis test aggregates that report log-likelihood, AIC, and BIC metrics. Bug fixes hardened the library against extreme arguments discovered through fuzzing, resolved non-central beta skewness/kurtosis NULL returns, fixed Apple clang linking for non-central t distributions, and standardized NaN spelling across platforms. He also rejected infinite BIGINT quantiles rather than casting them.

Datasketches Extension Overhaul

Major performance and feature work landed in datasketches. The extension now includes Bloom filter, Count-Min, VarOpt, EBPPS, and Density sketches. Performance optimizations moved aggregate states to DuckDB's arena allocator to enable spilling, with separate arena storage for Quantiles, REQ, TDigest, frequent items, VarOpt, EBPPS, Bloom, and Count-Min. He merged multiple PRs including sketch merge operations (datasketch_<kll|quantiles|req|tdigest>_merge), scalar sketch caching improvements, and build refactoring to generate one source file per sketch. vcpkg overlay patches now link to their upstream PRs, and he fixed platform-specific hash seed computation in Count-Min.

Arrow and RPC Updates

In vgi-rpc-csharp, he published QueryFarm.Arrow 23.0.0-queryfarm.4 independently and fixed support for zero-field Arrow structs. The vgi-csharp binding received a patch release (0.22.1) to select correct Arrow assets for NuGet consumers while maintaining v3 protocol compatibility.

Fuzzy Matching Enhancements

The rapidfuzz repository received a breaking change commit introducing Unicode-aware scoring, keyword options, new scorers, and extraction capabilities.

Open Source Contributions

Contributed community support to apache/datasketches-cpp and engaged with duckdb/duckdb development.

Additionally, 2 commits across private repositories indicate ongoing confidential work within the ecosystem.

A productive day spanning multiple repositories, with a focus on data sketching stability and IPC validation across the Query-farm ecosystem and DuckDB community.

datasketches saw significant activity with merges and new PRs addressing correctness and performance. Rusty merged three improvements: a performance optimization that takes or moves CPC sketches when combining partial states (PR #20), a fix for hashing HUGEINT/UHUGEINT values directly rather than converting through DOUBLE (PR #17), and stricter validation that rejects NULL split points and properly reports library errors (PR #18). New PRs continued this momentum with additional ports: PR #15 backports the v2.0 fixes to v1.5, PR #19 ports the split-point validation to v1.5, and PR #21 ports the CPC performance improvement to the C++ side. The refactoring work continues with PR #13 simplifying code generation and PR #14 consolidating hand-written sketches onto shared infrastructure.

vgi-rpc-python and vgi-rpc-go both received IPC validation improvements. In Python, PR #76 and the associated release (0.52.3) now validate complete typed IPC values, ensuring that embedded Arrow messages contain exactly one record batch followed by EOF or an EOS marker, rather than silently accepting extra batches or truncated data. The Go counterpart (PR #7) adds similar validation for wrapped binary requests with the optional Wr parameter. Both changes close a gap where extra or malformed IPC content would previously be ignored.

duckdb received two PRs addressing HTTP and transaction handling. PR #27967 fixes the HTTP PUT request content type to own its string rather than holding a dangling reference to the caller's temporary. Simultaneously, issue #27968 was opened to track transaction callback exceptions that skip commit notifications. These are low-level correctness fixes in the HTTP and transaction subsystems.

apache/datasketches-cpp received community support with PR #552, fixing a type-casting bug in req_sketch where the constructor truncated k to 8 bits when values of 256 or more were provided. Rusty also replied to PR #16 on Query-farm/datasketches, supporting the ongoing upstream alignment work.

query-farm-astro was updated with documentation and product pages: October 26–27 datasketches v2.0 changes were documented, a DuckDB 2.0 upgrade analysis with charts was published, and Crossfield product pages with runnable documentation went live. The site also now discloses analytics, cookies, and contact-form processing.

A productive day focused on dependency maintenance, security updates, and CI optimization across multiple projects.

Dependency updates and releases dominated the work in the core RPC libraries. On vgi-rpc-typescript, Rusty merged multiple Dependabot updates and resolved an OpenTelemetry baggage security advisory, preparing the 0.29.3 release. Similarly, vgi-rpc-python received a series of dependency updates including PyArrow 26 and certifi, with type annotation fixes to address stricter type checking on generator context managers. A critical Black security advisory (GHSA-cg8m-r9f2-5wm2) triggered additional work to restore documentation builds after the pymdown-extensions upgrade, culminating in the 0.52.2 release.

CI workflow optimization became a focus on duckdb-wasm. Three related PRs (#2296, #2294, #2292) address inefficiencies in the JavaScript build pipeline: eliminating unnecessary native source checkouts by using a shallow TPC-H setup, restricting tag-based builds to push events rather than create events, and decoupling Wasm compilation from unused test-data generator prerequisites to enable concurrent execution. He also opened corresponding issues (#2295, #2293, #2291) documenting these problems for the broader team.

haybarn received a fix (PR #6) to stage MinGW engine archives for extension builds, addressing a 404 error when community jobs requested pre-compiled static libraries.

On flatbuffers, Rusty regenerated a 64-bit embedded schema header (PR #9435) to resolve Windows CI failures after alignment changes were introduced.

Two discovery-oriented issues were opened on community extension projects: duckdb-sshfs (#28) and duckdb_webbed (#208) regarding vcpkg baseline alignment, suggesting collaborative stewardship of shared build infrastructure.

A productive day across multiple fronts: WebAssembly optimization, DuckDB SQL parsing robustness, and extension tooling improvements.

Emscripten: Rusty pushed a performance improvement across the chromium mirror and upstream emscripten-core/emscripten repositories that avoids unnecessarily copying all exports in applySignatureConversions. The change addresses issue #27996 and optimizes signature conversion handling in the WebAssembly toolchain.

DuckDB Extension CI Tools: Opened PR #456 to improve cross-platform vcpkg host triplet handling. The change ensures that Linux Docker builds respect the VCPKG_HOST_TRIPLET environment variable, matching the behavior already working on macOS and Windows. This follows up on the distribution matrix work from issue #151.

DuckDB Core: A burst of focused improvements to SQL parsing and formatting:

• PR #27854 fixes Arrow union type handling by mapping sparse union type ids through the format string rather than using raw ids—addressing issue #21842 and reviving work from the stale-closed PR #21843.
• PR #27835 preserves parser error byte offsets across Unicode whitespace normalization, so error positions remain accurate in the original input even when multibyte spaces are normalized (fixing #27831).
• PR #27834 corrects the SQL formatter to preserve parameter spelling, preventing $customer from being mangled into $ customer or having the name changed to match SQL keyword casing (fixing #27830).
• PR #27833 binds named options correctly in correlated table-function calls, ensuring they're validated against the function signature rather than silently becoming input columns (fixing #27829).

Rusty also opened several issues documenting parsing and formatting edge cases: #27857 on Windows MSVC extension crashes, #27832 on parser-affecting SET statements, plus the three issues addressed by the PRs above. He provided community support on extension-ci-tools issue #430, explaining the root cause of a unittest crash involving pooled string literal alignment differences between MSVC-built libraries and clang-cl objects.

Additional private repository work: 4 commits across 2 private repos.

Rusty worked across DuckDB's core repositories and community extensions, opening three issues to track feature gaps and inconsistencies.

Over on community-extensions, he opened issue #3362 highlighting a synchronization problem: v1.5.6 binaries don't yet reflect updates to the variegata extension that were submitted in PRs from October 22. This flags a release or build pipeline issue that needs addressing.

In the main duckdb repository, Rusty identified two separate concerns affecting data ingestion consistency. Issue #27823 requests improved function introspection—specifically exposing parameter kinds and descriptions to enable reliable discovery of DuckDB 2.0 functions. This would help tools and extensions query function signatures programmatically. Meanwhile, issue #27822 highlights an API inconsistency: the CSV and JSON readers currently reject the file_row_number option even though the Parquet reader supports it, limiting row tracking capabilities across format readers.

Beyond the public activity, he made commits across 2 private repositories, continuing work on unreleased projects.

A major maintenance push across the vgi ecosystem today, with 242 commits spanning 113 repositories. The work centered on two parallel efforts: a widespread dependency refresh cycle and a critical haybarn version constraint update.

Dependency Refresh Campaign

Rusty initiated a comprehensive refresh of core dependencies across dozens of repositories. The vgi suite was bumped from versions 0.19–0.24 to 0.43–0.46, with corresponding updates to vgi-rpc (0.14–0.16 → 0.29–0.35), Arrow (59 → 60), and language-specific variants. Affected repositories include vgi-jolt, vgi-etf-invesco, vgi-chart, vgi-bgp, vgi-elasticsearch, vgi-mask, vgi-tika, vgi-azure-signins, vgi-image, vgi-ioc, vgi-compress, vgi-yara, vgi-netflow, vgi-secretscan, vgi-iso20022, vgi-tiktoken, vgi-etf-first-trust, vgi-xslt, vgi-scholar, vgi-docgen, vgi-libpostal, vgi-trading-calendar, and vgi-statsmodels. Python-based repositories like vgi-etf-schwab, vgi-conform, vgi-causal, and vgi-sigma received updates to vgi-python (0.16–0.30 → 0.46.1) and vgi-rpc (0.44 → 0.52). Notably, vgi-tika saw Apache Tika jump from 3.3.1 to 4.1.0, and vgi-sigma received pysigma upgraded to 2.0.

Protocol and Runtime Enhancements

Significant work landed on the core language bindings and protocol layer. In vgi-java, changes improved table-in-out exchange semantics: batches are now refused if they emit nothing, exchange runtime classes are validated, and output continuation logic was refined. The vgi-java 0.44.0 release shipped these updates. Over in vgi-cpp, a v0.12.0 release introduced Protocol 2.2.0, featuring batched catalog_contents and enhanced dictionary FunctionInfo fields. Continued table-in-out output and input_rows_per_call instrumentation were added alongside rigorous continuation review fixes. vgi-rust received parallel improvements with tighter continued table-in-out validation on both exchange sides and shipped 0.47.0.

CI/CD Hardening

Across the ecosystem, Rusty pinned haybarn <2 constraints in CI pipelines across vgi-jolt, vgi-etf-invesco, vgi-chart, vgi-bgp, vgi-elasticsearch, vgi-mask, vgi-tika, vgi-etf-schwab, vgi-conform, vgi-compress, vgi-secretscan, vgi-sigma, vgi-causal, vgi-etf-first-trust, vgi-xslt, vgi-scholar, vgi-docgen, vgi-trading-calendar, vgi-netflow, and vgi-cbor. This appears to be a proactive compatibility guard ahead of a major haybarn 2.0 release. Several Python-based extensions also began lint testing on Python 3.13. Additionally, vgi-etf-invesco gated its live Invesco scan behind an INVESCO_LIVE environment variable, and vgi-mask hardened the vgi extension warm step.

Additional Work

vgi-libpostal expanded E2E and release build coverage to Python 3.13. vgi-cpp also received clang-format 22 reformatting of table-in-out exchange tests. Private repository activity totaled 39 commits across 16 repos, suggesting parallel internal work on related systems.

A sweeping maintenance day across the Query-farm extension ecosystem, with a focus on DuckDB 2.0 compatibility. The primary theme was ensuring function parameters are properly named in the introspection layer—fixing instances where duckdb_functions() was reporting generic column names like col0, col1 instead of actual parameter identifiers.

Parameter naming fixes were applied systematically across quickjs, json_schema, redis, rapidfuzz, tsid, marisa, webmacro, a5, jsonata, evalexpr_rhai, httpserver, and geosilo. This ensures better debuggability and developer experience when inspecting available functions in DuckDB 2.0.

cupola received several quality-of-life updates. Dependency bumps brought in vgi 0.45.0, vgi-rpc 0.29.2, and flechette 2.7.1 for version 0.4.262. Bundle size constraints were relaxed—the hard budget was removed and the total budget raised to 13 MB. The reporting layer gained v2 protocol support for objects and schedules, ownership scopes, and bounded folders (0.4.263), along with a fix to pause schedule polling in hidden browser tabs and stop gracefully on auth errors.

httpserver received critical stability fixes: ensuring the server returns from httpserve_start() only once it accepts connections, preventing database destruction during process exit, improved bind failure error reporting, proper CORS header handling, and distinct auth mode management.

geosilo was enhanced to allow casting geosilo_encode BLOBs back to GEOSILO type, making all examples executable.

redis now handles secrets without passwords correctly, treating "not found" only for genuinely missing secrets rather than empty passwords.

Contributions to upstream projects: fixes to uwdata/flechette addressed FlatBuffer vector encoding for empty cases and preserved native-unit bigint timestamp inputs. The vgi-rpc-typescript package gained support for Flechette server transports in its 0.29.2 release.

All 31+ extensions received coordinated version bumps in duckdb/community-extensions, ensuring the DuckDB community extension registry stays in sync with the latest improvements.

A prolific day across the VGI ecosystem and DuckDB community extensions, with 76 commits and 32 PRs opened.

VGI SDK Releases and Catalog Integration

Rusty coordinated a significant release cycle across multiple VGI language bindings. In vgi-java, he aligned catalog integration checks with the main contract, batched catalog metadata with durable snapshot publication, and released VGI 0.43.0 with published RPC dependencies. Similar work flowed through vgi-rust (releasing 0.46.0), vgi-csharp (releasing 0.20.0), and vgi-go, each implementing batched catalog metadata, snapshot publication validation, and SDK compatibility testing against matching native artifacts. The vgi-rpc-typescript work merged bounded external publication support and added full IPC validation, while vgi-rpc-java and vgi-rpc-csharp both gained support for batched binary dictionaries with proper codec integration and CI formatting.

RPC and Protocol Improvements

Across the RPC layers, the focus was on enhancing binary dictionary support and IPC robustness. vgi-rpc-java introduced idempotent IPC termination, while vgi-rpc-csharp implemented binary dictionaries in schema derivation with aligned formatting helpers. vgi-rpc-typescript refined external result decoding with bounded gzip support, ensuring full IPC validation across the protocol.

DuckDB ADBC Scanner and Extension Compatibility

In adbc_scanner, Rusty fixed validity mask handling for DuckDB 1.5 using ValidityMask::AllValid, updated the extension version, and ensured the driver tests build against the v1.5.6 release tag. A critical fix was added to match target table NOT NULL columns during append operations, improving data integrity.

Community Extension Stewardship

Rusty opened 16 PRs on duckdb/community-extensions to update extensions including webmacro, tsid, tributary, textplot, tera, stochastic, shellfs, redis, rapidfuzz, radio, quickjs, openprompt, and minijinja. Each PR updates extension commits for DuckDB v2.0.0 and requests builds for v1.5.6 (variegata), demonstrating ongoing maintenance of the community extension ecosystem.

Documentation and Web Properties

The query-farm-astro site received significant updates: refreshed the Overture map example for the October data release and added documentation for DuckDB 2.0 changes across six extensions (textplot, bitfilters, tera, datasketches, stochastic, minijinja). A new PR (#13) on query-farm-astro enables installation from the Query.Farm extension repository on DuckDB 2.0, splitting the install process into setup and usage steps.

Upstream Contributions

Rusty added DuckDB integration to the use cases in the MiniJinja documentation (mitsuhiko/minijinja), contributing back to the broader templating ecosystem. He also updated vgi-yfinance and vgi-open-meteo dependencies to the latest VGI SDKs and RPC, gated deployment builds with integration tests, and ensured Node runtime compatibility for Wrangler validation.

Private Activity

An additional 30 commits across 8 private repositories indicate ongoing internal development work.

A high-velocity day across the VGI ecosystem and DuckDB extensions, with 140 commits touching 49 repositories. The work centered on conformance testing parallelization, release coordination, and bug fixes across multiple language bindings and extensions.

Conformance Testing & Parallelization

Rusty pushed parallel execution improvements across the VGI RPC suite. In vgi-rpc-cpp, conformance runners now execute in parallel, and a corresponding 0.11.0 release followed (with an earlier 0.10.0 also tagged). Similar parallel test improvements landed in vgi-rpc-go, vgi-rpc-typescript, and vgi-rpc-java, all using pytest-xdist to speed up conformance validation. These changes represent a systematic effort to reduce CI overhead across the language-specific bindings.

VGI Core Library Releases

Multiple VGI language implementations shipped new releases:

• vgi-rpc-java saw three releases (0.32.0, 0.32.1, 0.33.0) with fixes for stream type casting of strings, decimals, and nested types to match reference behavior
• vgi-rpc-rust released 0.32.0 and vgi-rust shipped 0.45.0, with the latter requiring vgi-rpc 0.32 to pull in Identity.v2 support
• vgi-cpp released v0.11.0 supporting Identity.v2 and attach_tickets 2.0.0, including a rebuild of libsodium from an overlay port
• vgi-java released 0.42.0 and updated to require vgirpc 0.33.0

DuckDB Extension Updates

A broad sweep of extensions received maintenance updates bumping DuckDB and extension-ci-tools to v1.5-variegata tips, with version bumps to 2026102001 across: stochastic, fuzzycomplete, quickjs, json_schema, rapidfuzz, tsid, marisa, textplot, webmacro, a5, jsonata, evalexpr_rhai, httpserver, and airport.

Bug Fixes & Features

redis received significant attention with fixes to properly parse RESP replies, implement per-row arguments, return parsed values, and rename RESP reply type enumerators for Windows compatibility. A work-in-progress fix addresses an issue where table functions never returned rows.

airport had endpoint location and chunk boundary corrections applied, bumping to version 2026102002.

fuzzycomplete includes a work-in-progress backport of the five-column sql_auto_complete output from v2.0, and evalexpr_rhai has backported FFI leak and panic-safety fixes from v2.0.

The stochastic extension also received documentation cleanup (dropping compatibility notes to match standard layout) and a fix naming Cauchy distribution parameters as location and scale.

Private Work

Additionally, 12 commits were made across 4 private repositories.

Overall, this reflects systematic progress on testing infrastructure, dependency management across the extension ecosystem, and targeted bug fixes in critical components.

A productive day spanning multiple Query-farm projects, Arrow ecosystem contributions, and open-source community support.

cupola received performance and reliability improvements. Rusty optimized catalog metadata scoping and report discovery, refactored reporting protocol preferences to reduce redundancy, and reused HTTP capabilities across refresh cycles. OAuth handling was strengthened with fixes to drop dead refresh tokens and prevent sign-in popup conflicts. New tests validate OAuth expiry behavior against both a fake identity provider and real Azure Entra. Two releases shipped: versions 0.4.260 and 0.4.261.

vgi-rpc-c saw substantial expansion of C SDK capabilities. The library now includes nanoarrow C RPC clients and servers with a conformance suite, shared memory transport negotiation and per-connection management, C worker pools, and OpenTelemetry instrumentation. A searchable C documentation site was built from source. Rusty also added Linux socket client support via timeval definitions, updated licensing to match VGI, and contributed governance guidance for contributors.

vgi-rpc-website documentation expanded to cover the new C SDK alongside existing languages. Updates include shared memory and worker pooling documentation, generated C docs integration, and a comprehensive performance comparison of worker artifact size and memory footprint across eight languages. CSV validation was added to ensure data consistency.

vgi-c launched with a complete C VGI worker SDK backed by vgi-rpc-c. This includes documentation for the passing shared-memory integration lane, RPC telemetry exposure in workers, agent guidance, and SDK implementation details. Nanoarrow patching remains isolated to the RPC dependency.

query-farm-astro published a new article on building VGI after Apache Arrow Flight, with abstract Bezier artwork and social card generation. The blog also received a reflective update on DuckDB Quack context in the Arrow Flight article. Additional fixes addressed Cloudflare email decoder errors on SVG links, VGI terminal imports, source-map URLs, and REPL extension prefetching before LOAD. Blog readability improvements included current-section highlighting.

haybarn-community-extensions was kept in sync with upstream developments, pulling in recent vgi and a5 improvements including catalog inventory reuse, scalar/LATERAL dedup optimization, and upgrades to a5 1.0.0 and GEOMETRY integration.

On the open-source front, Rusty contributed five PRs to apache/arrow-nanoarrow:

• PR #982: Validation for binary view and string view values at NANOARROW_VALIDATION_LEVEL_FULL
• PR #981: UTF-8 validation for string types at FULL level (addressing compatibility with Arrow C++)
• PR #980: Writing dictionary deltas in the IPC writer (draft, stacked on earlier work)
• PR #979: Skipping unchanged dictionaries in ArrowIpcWriterWriteArrayViewWithDictionaries
• PR #978: Fixing offset buffer handling for empty variable-size arrays

He also provided community support on duckdb/duckdb-wasm (issue #1916), addressing an extension loading error with Cross-Origin Isolation (COI).

Private repositories also received 4 commits across 2 repos.

A productive day focused on optimizing extension loading in haybarn-wasm and ensuring Node.js compatibility with the latest web-worker release.

haybarn-wasm saw significant performance work. Rusty tackled extension prefetching by implementing parallel downloads of DuckDB extensions, replacing the previous synchronous XHR approach. The new flow uses WebCrypto to hash extensions (eliminating a wasm-compiled SHA-256 bottleneck that accounted for ~25% of load time for common extensions like icu, autocomplete, spatial, iceberg, grainlift, and ducklake) and compileStreaming to compile WASM modules as they download. He also optimized COI (Cross-Origin-Isolation) behavior by batching consecutive extension loads to run single-threaded once rather than repeatedly context-switching. These changes landed across PR #37, alongside complementary fixes for worker reliability and Node.js compatibility.

Compatibility was a key theme. PR #36 addressed a breaking change in web-worker 1.5.0, which now runs classic workers via vm.runInThisContext instead of require(), leaving globals undefined. Rusty updated the Node worker bundles to run as ES modules, and documented the pattern in PR #34 to help users migrate smoothly. He also filed a community reply on the web-worker repo (issue #55) regarding the regression.

Reliability improvements landed in PR #35: AsyncDuckDB now properly rejects pending requests when a worker fails or never starts, fixing a bug where worker errors would leave awaiting promises hanging indefinitely.

These changes were released as 1.5.6-rc5, and cupola was upgraded to track the new version.

A productive day across multiple WebAssembly and data serialization projects, with a focus on performance optimization and correctness fixes.

haybarn-wasm Optimizations

Rusty landed four feature branches improving extension loading and memory usage in haybarn-wasm. The work centered on reducing both time and memory overhead:

• Memory efficiency: Merging PR #30 freed extension bytes immediately after loading, dropping retained buffers from +25.9 MB to +0.03 MB after garbage collection. A follow-up commit in PR #31 further halved load time (from ~2000 ms to ~1000 ms in benchmarks) and reduced memory footprint by ~40%.
• Cache handling: PR #29 fixed a bug where failed extension downloads were cached, causing misleading signature errors on retry. The fix prevents caching when Node's fetch encounters non-2xx responses.
• Chrome code caching: PR #28 enables Chrome's WebAssembly code cache for repeat page loads, cutting db.open() time from ~230 ms to ~87 ms on subsequent visits. This required counting progress on module clones rather than the original buffer.
• Memory-mapped globals: PR #33 optimized global mutability detection by reading bitflags directly from the binary instead of throwing/catching exceptions for each data symbol. Extensions exporting 1,286+ immutable globals see dramatic speedup.

Rusty also pushed a fix to cupola skipping heavyweight extensions during iOS startup, and bumped the workspace to version 1.5.6-rc4.

Upstream Contributions

Three pull requests landed on critical Apache projects addressing memory and correctness issues:

arrow-nanoarrow: Opened PR #974 to fix IPC serialization of null counts—the encoder was writing -1 (unknown count from the C data interface) directly into FieldNodes instead of computing actual counts. Also contributed a validation-level configuration feature and a fix for zero-offset empty arrays with empty offsets buffers (PR #964 and a separate commit).

emscripten: Three PRs tackle WebAssembly module loading efficiency. PR #28007 frees side-module file data after all threads load it (addressing a TODO in load_library_done). PR #28006 reads exported global mutability from the binary instead of testing it via throw/catch per symbol. PR #28002 ensures wasm bytes are released after compilation rather than held for the program's lifetime.

flatbuffers: Opened PR #9414 fixing Go vtable de-duplication to consider object size, preventing smaller objects' vtables from being incorrectly reused by larger objects.

duckdb-wasm: Two PRs mirror the haybarn-wasm work—PR #2279 fixes extension download caching and PR #2278 enables Chrome code caching for repeat loads.

Diagnostics

Opened two issues documenting edge cases: arrow-dotnet #462 (ArrowArrayConcatenator failing on zero-length string arrays with empty offsets) and flatbuffers #9413 (vtable de-duplication not considering object size).

A prolific day spanning infrastructure, format standardization, and performance work across the Apache Arrow ecosystem and DuckDB/Haybarn projects.

Apache Arrow Format & Serialization

Rusty opened a series of coordinated PRs across multiple Arrow implementations to add custom_metadata to DictionaryEncoding (tracking upstream proposal apache/arrow#49704). This solves a long-standing serialization ambiguity where dictionary-encoded extension types and extensions with dictionary storage couldn't be distinguished over IPC. The work spans:

• PR #53994 on arrow for the format specification and C++ implementation
• PR #972 on arrow-nanoarrow to decode and write the new metadata field
• PR #11622 on arrow-rs adding the Rust implementation
• PR #1381 on arrow-go for Go support

Additionally, he opened several focused IPC improvements on arrow-nanoarrow: fixing metadata truncation at NUL bytes (PR #971, issue #970), adding support for dictionary value type metadata in writers (PR #968), and allowing message metadata attachment (PR #967). PR #965 and #964 optimize the stream reader by referencing message bodies in place and making validation configurable.

DuckDB/Haybarn WebAssembly

Multiple commits to haybarn-wasm addressed dynamic linking performance and memory threading issues. PR #26 resolves dynamic-linking exports lazily at startup, cutting initialization from ~39 ms to ~2 ms with 40k exports. PR #27 keeps heap views current when other threads grow memory, fixing intermittent RangeError crashes on the COI (pthreads) bundle. Complementary work landed in PR #2276 on duckdb-wasm applying the same heap view fix. A separate PR #25 refactors JSON insertion to use the engine's native read_json rather than maintaining a parallel JSON reader.

Emscripten Startup Optimization

Two performance PRs opened on emscripten-core/emscripten: PR #27993 creates GOT entries lazily during dynamic linking to speed up MAIN_MODULE startup by up to 40%, and PR #27996 eliminates unnecessary export object copying in signature conversions, dropping that step from 39 ms to 2 ms.

DuckDB Extensions & Integrations

On duckdb-duckdb-nanoarrow, PR #129 adds dictionary encoding to the Arrow IPC stream writer using nanoarrow's DictionaryBatch encoder, and PR #128 bumps nanoarrow to pick up upstream IPC dictionary decoding fixes. PR #3093 on duckdb/community-extensions updates the a5 extension to latest commits. cupola upgraded to Haybarn 1.5.6-rc3, and a5 itself was upgraded to 1.0.0. Documentation work on query-farm-astro covers the new a5 1.0 features (cell IDs, coverings, set operations).

Community Support

Rusty replied to issues on apache/arrow#49704 (dictionary extension roundtripping), apache/arrow-nanoarrow#622 (dictionary IPC support), and emscripten-core/emscripten#17891 (table growth strategy), providing guidance on cross-implementation coordination and performance tradeoffs.

Other Updates

A commit to cloudflare/cloudflare-docs added Grainlift to D1 community projects. Four new issues opened tracking test parallelization (arrow-nanoarrow#969), metadata handling in JSON comparison (#973), and feature tracking across Arrow implementations.

A focused day of release coordination and maintenance across the Query.Farm ecosystem, with attention to component versioning and documentation.

vgi-excel and haybarn-wasm received release-preparation commits. The WASM package bumped to version 1.5.6-rc2, with a fix to report the actual release version rather than git describe's dev suffix. Package README files were also branded with Haybarn and Query.Farm logos and linked back to parent repositories.

cupola was upgraded to consume Haybarn WASM 1.5.6 (0.4.250) and its extensions. Rusty removed an unsupported Haybarn MVP fallback and refactored documentation by splitting CLAUDE.md into path-scoped .claude/rules files for better organization.

haybarn-community-extensions pulled in the latest upstream changes from the main Haybarn branch, incorporating fixes for Arrow schema lifecycle management, performance optimizations for large-string deduplication, and worker pool idle limit tuning (5s → 60s).

query-farm-astro saw three updates: refreshed extension load counts across a 90-day window, updated to Haybarn v1.5.6-rc1, and pinned to the rc2 build after release. An additional fix ensured alias extensions remain distinct in the Pagefind search index.

Rusty also replied to a security notice on vgi-typesafe (issue #1) regarding a potential decision hijacking risk in Jev-based applications, providing community support on the concern.

Private repository work continued across three separate projects.

vgi-polymarket received a commit normalizing repeated price-history observations and rejecting conflicting prices, improving data consistency in the polymarket integration.

vgi-rust shipped version 0.44.0 and added typed pruning and explicit worker filtering contracts, expanding the query engine's capabilities for more precise data filtering and worker task management.

haybarn-rust underwent a comprehensive rebrand and release cycle. The work included:

• Releasing version 1.10506.0-rc.2 rebased onto upstream duckdb-rs
• Maintaining backward compatibility by keeping stream_arrow_auto as a deprecated alias of stream_arrow
• Replacing upstream CI workflows with Haybarn-specific CI and enabling Trusted Publishing for crates.io
• Updating documentation with rebranded README, NOTICE, and CLAUDE.md files plus a smoke test example
• Renaming crates from duckdb-rs to haybarn, libduckdb-sys to libhaybarn-sys, and duckdb-loadable-macros to haybarn-loadable-macros
• Re-pointing DuckDB sources to the Haybarn engine and regenerating the amalgamation

haybarn-go-bindings pushed updated libraries to support the Haybarn rebrand.

haybarn-community-extensions synced upstream community-extensions for the v1.5.6 sweep.

Opened PR #65 on duckdb-zarr leveraging the new VGI 0.44 typed pruning and diagnostics APIs. The PR fixes coordinate pruning cases that discarded valid rows, resolves large integer precision loss, handles strings returning NULL, and treats HTTP 403 responses appropriately rather than assuming absent chunks.

vgi-github generated connected GitHub semantic models and researcher workloads for analysis and integration.

On open-source stewardship, Rusty added Radio to the IXWebSocket users list on machinezone/IXWebSocket (PR #614), opened issue #1 on KLL-Polymer-code requesting a LICENSE.txt file, and replied to issue #38484 on apache/arrow regarding subsidiary logos for Arrow subprojects.

Private repository activity included 3 commits across 1 private repo.

Rusty focused on performance optimization and feature refinement across multiple repositories today, with particular attention to the cupola workspace and supporting infrastructure.

Work on vgi-rust centered on a performance improvement: caching decoded window partitions with bounded retention to reduce redundant computation during query planning.

The haybarn-wasm build system received updates, with a workspace version bump to 1.5.6-rc1 and an upgrade to emscripten 6.0.12. These changes prepare the WebAssembly runtime for release. The haybarn-community-extensions repository was similarly updated with a CI target bump to v1.5.6 to align with the upstream engine version.

Most of the day's effort went into cupola, where Rusty shipped a series of fixes and features across seven commits:

• Access control discovery: Fixed scheduled access policy and caller email resolution (0.4.243)
• Library consolidation: Unified library saving and document recovery (0.4.245), and consolidated reports, notebooks, and queries into a single worker library abstraction (0.4.242)
• Bundle optimization: Removed redundant nested reporting schemas and unused response schemas from the browser bundle, reducing client-side payload
• Editor integration: Fixed library document opening to use their normal editors rather than falling back to generic views (0.4.246)
• Protocol improvements: Adopted the complete Identity protocol with unsupported fallbacks for better compatibility (0.4.244)
• UI polish: Refined the browsing experience for saved objects and worker schedules (0.4.247)

Beyond internal work, Rusty provided community support by replying to PR #22715 on duckdb/duckdb regarding Arrow export support for the VARIANT type.

Private repository activity included 2 commits across 1 private project.

Cloudflare API codegen expansion. Work on vgi-cloudflare pushed significant progress in resource coverage, expanding from 1,142 to 1,684 queryable resources through improved code generation. The commit emphasizes stable naming conventions, a critical foundation for maintaining consistency as the resource set grows.

GitHub API refinements. Two commits landed on vgi-github: one expanded read-only API coverage with newly typed datasets, and a follow-up pinned CI and live workflows to Python 3.14, ensuring compatibility across the project's automation infrastructure.

Community support. Rusty replied to issue #45073 on microsoft/vcpkg, investigating a cache failure scenario in GitHub Actions workflows — contributing troubleshooting insights to the broader vcpkg tooling ecosystem.

A lighter day on October 11th, with activity focused on private work. Rusty made a single commit to a private repository, keeping progress moving on undisclosed projects.

Daily Activity Summary — October 10, 2026

cupola — Report Scheduling and UI Refinements

Rusty shipped multiple improvements to cupola across versions 0.4.239 and 0.4.240. The work focused on scheduled report exports and email delivery, with a new isolated headless renderer added to handle report generation without blocking the main UI thread. Error handling received attention as well—SQL errors in reports now retain full context so users can debug issues while keeping the editing interface accessible. A separate fix prevented native dropdowns from clipping across the application, addressing a recurring UI issue.

adbc_scanner — Schema and Write Capability Fixes

Two significant issues were resolved in adbc_scanner to improve compatibility with autocommit-only ADBC drivers:

• Schema handling: Attached databases now correctly inherit the driver's default schema instead of always using "main". This fix enables drivers like PostgreSQL (which default to the "public" schema) to work properly with attached database references like db.table and USE db statements.
• Write operations: Catalog writes now function for autocommit-only drivers, fixing failures when executing INSERT INTO db.t and CREATE TABLE db.t AS statements even outside transaction blocks.

Rusty fielded community questions on both issues (issue #19 and #20), providing context for the fixes.

adbc_scanner PR and Community Extensions

Opened PR #3016 on duckdb/community-extensions to bring the community extension's adbc_scanner version up to date with the latest commits, ensuring users have access to the schema and write operation improvements.

haybarn-community-extensions — Dependency Bump

A substantial upstream bump was applied to align with main branch commit 74015fc. The changelog reflects deep infrastructure work across the haybarn ecosystem: structured error handling with extra_info vocabularies, execution and transport layer improvements, worker exception reporting, OAuth and auth/config refinements, and catalog reflection fixes. Notably, invocation_id was renamed to execution_id throughout the codebase for consistency.

Private Repository Work

11 commits were pushed across 3 private repositories, continuing parallel development efforts.

A high-volume day focused on open-source stewardship, documentation standardization, and security fixes across multiple projects.

Community Extensions & Documentation

Rusty made substantial contributions to the DuckDB ecosystem by standardizing documentation links across duckdb/community-extensions. This involved updating links for 24 extensions—including airport, bitfilters, cronjob, crypto, datasketches, and many others—to ensure consistent documentation formatting. Beyond documentation, he also took on maintainer roles for five community extensions: webmacro, cronjob, quickjs, redis, and tsid.

URI Credential Security Fixes

A significant security issue was addressed across two related repositories. In vgi-rpc-rust, commits fixed a critical bug where JWKS fetch errors were echoing URL credentials and query parameters in error messages. This fix propagated to grainlift, where he released version 0.6.1 with URI credential redaction. The fix ensures that sensitive authentication information (user:password@ patterns) is stripped from URIs before logging or error reporting, with documentation clarifying that OAuth messages were the source of the leak, not the reqwest HTTP client itself.

Cupola Report Storage & UI Improvements

cupola saw a flurry of activity across multiple releases (0.4.225–0.4.230). Key improvements included:

• Integration with HTTP report stores, unifying report storage browsing and transfers
• Ability to browse and compare worker report revisions
• UI fixes aligning report controls and editor saves with the worker contract
• Resolution of a Pivot catalog table discovery bug (quoted Perspective table IDs)
• Improved HTTPS localhost connection error messaging
• Test enhancements including e2e test isolation and regression testing for protocol 0.5 session schemas

Filter Library Documentation

On FastFilter/xor_singleheader, documentation was added for DuckDB bindings for XOR and binary fuse filters, supporting PR #79.

Routine Maintenance

Other updates included releasing grainlift 0.6.1 with the credential redaction fix and vgi-rpc 0.31.4 dependency bump, CI toolchain repinning on grainlift, and housekeeping across davidgasquez/awesome-duckdb to update Query.Farm extension links.

A prolific day across the Query.Farm ecosystem and upstream community projects. Work focused on bug fixes, extension maintenance, and broad ecosystem documentation.

RPC and observability work. vgi-rpc-go received multiple stability fixes: resolved a conflict where OpenTelemetry and Sentry instrumentation hooks were replacing each other, fixed an issue where only one dispatch hook could be held at a time, and corrected trace correlation loss in access logging. Two patch releases shipped as v0.34.0 and v0.34.1.

Extension updates and releases. quickjs was bumped to DuckDB v1.5.6 and QuickJS-NG v0.17.0, with the extension version incremented to 2026100801. redis merged PR #10 adding TTL support (EXPIRE, TTL, EXPIREAT functions) and updated its DuckDB submodule to the v1.5-variegata branch. cupola released v0.4.223 with multi-catalog inventory and AI prompt caching fixes, with hardening applied across those systems and DDL navigation. airport fixed an issue where Flight exchange lifetime was materializing results unnecessarily. radio fixed VARCHAR message handling—payloads were being escaped, and all messages were sent as binary frames instead of text where appropriate. vgi-csharp corrected run_tests.sh to execute the actual test tree rather than a compiled-in unittest version.

Community extension submissions. Rusty opened 16 PRs submitting Query.Farm extensions to upstream projects' adoption and integration lists: Crypto to BLAKE3, Hashfuncs to xxHash and RapidHash, Tributary to Kafka integrations and librdkafka bindings, Bitfilters to XOR and binary fuse filters, Marisa to marisa-trie bindings, DataSketches adapter to the DataSketches website, Radio to IXWebSocket users, MiniJinja integration to MiniJinja use cases, and Query.Farm projects to Apache Arrow's Powered By page. Additionally, he submitted comprehensive updates to awesome-duckdb covering six existing extension entries and filling in missing projects from the Query.Farm catalog.

Community extension registry updates. Opened 15 PRs against duckdb/community-extensions to sync the latest commits for: tsid, stochastic, shellfs, redis, radio, quickjs, openprompt, minijinja, lindel, json_schema, inflector, httpserver, hashfuncs, fuzzycomplete, cronjob, and geosilo.

Awesome list submissions. Submitted documentation improvements to multiple community curated lists: added DuckDB projects to awesome-typesafe-jev, awesome-adbc, awesome-kafka, awesome-duckdb-spatial, and awesome-iroh. Also opened an issue against json-schema-org/website proposing Query.Farm's JSON Schema for DuckDB in the ecosystem.

Website and documentation. query-farm-astro published an article on Iroh browser SQL with diagrams and metadata, connected product pages and technical guides to published articles, updated VARIANT and Arrow support tracking, and documented new TTL functions in Redis extension documentation. datasketches-website and quickjs-ng received commits documenting their respective DuckDB adapters.

Community support. Replied to issues on a5 regarding overlapping polygon detection and on quickjs about DuckDB community extension details. Also provided community stewardship across upstream projects with targeted, well-documented submissions.

Private activity. 16 commits across 2 private repositories.

Rusty delivered a major coordinated release across the VGI ecosystem today, addressing security, stability, and protocol improvements across multiple language implementations.

VGI RPC Layer

The vgi-rpc-cpp library received several critical fixes bundled into releases 0.8.1 through 0.9.0. A bearer authenticator that refused anonymous callers now supports optional bearer authentication via HttpConfig::bearer_optional. The access log was oversharing request payloads and stream state tokens—this has been corrected. Additionally, serve_unix workers lacked an idle timeout, causing launched processes to hang indefinitely; this is now fixed. The reflection client tests were reformatted to match CI's clang-format 22.1 standards.

Similar fixes landed in vgi-rpc-rust (releases 0.31.1–0.31.3) and vgi-rpc-go (releases 0.33.1–0.33.2), where request payloads were inadvertently reaching access logs and unary records still carried a transitional payload_omitted marker. The vgi-rpc-typescript library received corresponding updates for request data logging at DEBUG level.

VGI Core Implementations

vgi-java saw substantial protocol advancement across releases 0.39.0–0.41.1. Port attach tickets now allow a runner holding only a grant to reattach a user's catalog using a sealed ticket. Opaque values are no longer unsealed with a fallback—they either validate or reject, and secret options no longer appear in the attach ID. The VgiService is now generated from the reference specification rather than hand-written, and the vgi.v2 registry reflects this change with the reference hash reported by reflection. AttachTickets now seal using vgirpc's XChaCha20Poly1305 instead of a local copy. HTTP lane error handling was improved so worker errors are treated as skips, and the full vgi.v2 surface is now hosted.

In vgi-cpp, attach tickets introduce a critical security feature: a user's ATTACH is sealed so only a runner holding their grant can replay it. Opaque attach data is now sealed and bound to the caller, eliminating plaintext secret options. The integration harnesses were fixed to fail on every HTTP-related worker error rather than silently skipping them, and the vgi.v2 hash is pinned to the reference to prevent surface drift. Launched workers now honour the --idle-timeout flag on --unix sockets. The vgi.v2 registry is generated from the reference specification, and optional bearer support surfaces the principal. Releases span v0.8.0 through v0.10.1.

vgi-rust (releases 0.41.0–0.43.0) mirrors these advances: attach tickets seal user ATTACHes for runner replay, opaque data is sealed on HTTP, and integration lanes no longer hide failures as skips. The vgi.v2 surface is now fully reference-generated with a hash matching vgi-python 0.43.0. HTTP test infrastructure was cleaned up—run_http_tests.sh was leaking all five servers on every run. CI now runs unit tests in the staged directory where HTTP workers live.

Grainlift and Integration

grainlift-go now advertises per-connection statistics support via Grainlift protocol 0.5. The grainlift-cloudflare deployment was updated to protocol 0.5 and declares statistics as unsupported; PR #1 merging durable-objects-and-permissions support was integrated.

Auxiliary Work

cupola received a fix to open the default schema and simplify empty sidebar sections. Across the ecosystem, integration test scripts in vgi-java and elsewhere were cleaned up to prevent HTTP worker leaks during CI runs. Overall, Rusty coordinated security hardening (sealed attach tickets, secret option protection), stability improvements (idle timeouts, error visibility), and specification alignment (vgi.v2 generated from reference) across 22 repositories.

A particularly active day across the VGI ecosystem, with 91 commits spanning 18 repositories and focused work on grant-based authentication, catalog capabilities, and cross-language RPC implementations.

Grant Authentication Across the Stack

The dominant theme was rolling out bearer credential support for minted grants across all VGI RPC language bindings. vgi-rpc-cpp (v0.7.0), vgi-rpc-rust (v0.30.0), vgi-rpc-go (v0.32.0), vgi-rpc-typescript (v0.27.0), vgi-rpc-java (v0.29.0), and vgi-rpc-csharp (v0.14.0) all received updates to accept sealed grants from issue_grant and resolved tokens as HTTP bearer credentials. This represents a significant authentication flow improvement across the distributed worker infrastructure.

Core Language Workers

vgi-rust received dual releases (v0.39.0 and v0.40.0) adding catalog_contents fixture catalogs, DDL-capable in-memory catalogs, and client-side load_catalog support. The worker now accepts sealed grants via HTTP bearer authentication configured through --grant-key or VGI_RPC_GRANT_KEYS. A flaky test suite issue was addressed where tampered-grant tests intermittently passed with real grants.

vgi-cpp (v0.7.0) underwent substantial work: ported five missing fixtures to published-extension lanes, reverted and re-applied catalog_contents fixture handling, served six catalog_contents fixture catalogs, and delivered DDL-capable memory catalogs. A critical scalar bug was fixed where result() moved an array before reading its length on x86-64. Build parallelism was capped at 2 to prevent GCC runners from exhausting memory. The worker now properly authenticates incoming grant-based calls.

vgi-java shipped two releases (v0.37.0 and v0.38.0) with catalog_contents fixture support for code-defined and in-memory DDL catalogs, plus grant-based bearer authentication.

vgi-csharp (v0.14.0 and v0.15.0) added catalog_contents fixture catalogs, DDL-capable in-memory catalogs, and full support for sealed-grant and resolve_token authentication over HTTP.

Specialized Workers & Extensions

vgi-polars (v0.9.0–0.9.1) tackled several improvements: migrated to PyPI-hosted vgi-python with dependency pinning to v0.41.0, added Filter Encoding v2 support for pushed-down filters, fixed string/binary is_in needle handling, resolved hanging scans that ended before their streams, and implemented proper cleanup of abandoned table_function streams. Polars 2.0 is now the build target with 1.x still supported.

vgi-lint-check fixed buffering-function argument attachment and improved simulation output to show result columns.

vgi-rpc-go (v0.32.0) extended bearer credential support to HTTP-based sealed grants and resolve_token flows.

Frontend & User-Facing Tools

cupola saw steady improvements across three releases (v0.4.219–0.4.221): bulk catalog content loading with legacy fallback, sidebar context menus for reports and notebooks, file attachment support across AI assistants, HTTP request abort handling on cancellation, Iroh node startup deferred to first use for performance, upgrade notification dismissal, and local document sidebar organization with restored report creation.

haybarn-wasm bumped to v1.5.5-rc8 to support request cancellation in flight.

Open Source & Private Work

Rusty also contributed to the duckdb/duckdb-wasm repository as part of broader community engagement. Additionally, 22 commits across 4 private repositories indicate ongoing internal development.

A prolific day across the Query Farm ecosystem, with 91 commits spanning 17 repositories and significant feature development in both backend services and the Cupola UI.

VGI Language Bindings & Core Updates

Rusty pushed releases and maintenance updates across the VGI stack. The vgi-rust crate received version 0.37.1 with documentation improvements, including fixes to broken intra-doc links in vgi-client and removal of secrets guidance from attach-option docs. Similar maintenance appeared in vgi-java with CI adjustments to handle upstream attach_secrets skips. The vgi-excel binding was prepared for Cupola 0.5.1, pinning VGI releases and adding sign-in recovery support.

Haybarn WASM & HTTP Layer

Work on haybarn-wasm focused on request lifecycle management. He upgraded the engine dependency and implemented abort support for HTTP requests in flight, ensuring httpfs requests properly carry the query's interrupt flag. This honors request cancellation in retry loops, improving responsiveness when queries are stopped mid-execution.

Cupola: Major Multi-Release Sprint

The bulk of today's effort landed in cupola, with a coordinated sequence of 8 releases (0.4.208–0.4.216) delivering substantial UI and workspace features. Early releases tackled inspector and editor polish: sidebar Inspector with call snippets and signature help (0.4.209), per-tab revision history and a redesigned split Run button (0.4.214), distinct summary and documentation display (0.4.210), Inspector sections with view SQL (0.4.211), and toolbar regrouping plus a History diff stack (0.4.212). The inspector also now stringifies LIST columns in Perspective snapshots (0.4.208).

Following these incremental improvements, he merged multi-catalog phases 3A, 3B, and 3C. Phase 3A introduced the workspace manager editor for managing multiple catalogs. Phase 3B added workspace files, DuckDB script export/import, and JSON Schema support. Phase 3C brought alias rename dialogs, report requires with Rebind/Attach, command palette, and a tokenizer for catalog alias references. Supporting these features are structured attach options with a typed options form and consent screen, plus a portable workspace format with aliases and #ws= codec for catalog-qualified routing.

Final releases in the sequence (0.4.215–0.4.216) delivered native SQL notebooks with charts and AI editing, query cancellation consistency across all surfaces, and fixes to arrowLosslessConversion and R2 cache policy handling. Dependencies were updated to vgi 0.37.1 and test workers to vgi-python 0.38.

Ecosystem & Private Work

Additional activity touched vgi-rpc-typescript, vgi-csharp, vgi-go, vgi-rpc-python, vgi-rpc-csharp, vgi-github, adbc_scanner, query-farm-astro, and community extensions. He also contributed 13 commits across 2 private repositories, and engaged in 8 community support interactions—likely reviewing or advising on DuckDB extensions and open-source projects.

Rusty released version 0.4.207 of cupola with a notable feature addition: a resizable value panel and formatted JSON display in Lines mode. The work involved both the feature implementation and expanded test coverage for results handling, as captured in the release commit and preceding feature branch work.

Over on arrow-rs, he opened PR #11374 to address issue #9837, introducing a zero-copy pull-based BufferStreamReader for IPC streams. The motivation is clear: when an entire IPC stream is already loaded in memory—whether from an HTTP body, memory-mapped file, shared-memory segment, or bytes::Bytes from object storage—the existing StreamReader<R: Read> API forces unnecessary copying. This new approach eliminates that overhead by leveraging the data already in hand.

Rusty also weighed in on the DuckDB community with feedback on PR #22715, offering support for the Arrow export enhancement targeting VARIANT type support. His involvement across these three projects reflects ongoing work on columnar data handling, zero-copy optimizations, and interoperability between query execution and Arrow serialization.

Rusty shipped two releases of cupola focused on improving how query results are displayed. The first release (0.4.205) added support for reading full values in query results through a new value panel and Lines layout. He then refined the presentation in 0.4.206 by restyling the Lines results layout to display results as row cards with a key/value table structure, improving readability and visual hierarchy.

Work on vgi-github centered on enhancing the GitHub data integration extension with better user experience and observability. The README was rewritten to focus on user-facing documentation, with internal details moved to a separate DEVELOPMENT.md file. On the technical side, Rusty added a token_source ATTACH option that allows users to supply their own GitHub login credentials rather than relying on a default token. Servers can now opt into this feature via the VGI_GITHUB_ALLOW_TOKEN_SOURCE environment variable. He also implemented logging improvements to report rate-limit waits, retries, and token budget information to the DuckDB log, giving users better visibility into API usage and performance.

A major release day across the Grainlift ecosystem, with significant work on object storage support, request handling, and production readiness across multiple language implementations and supporting libraries.

Core Grainlift Framework

The grainlift repository reached version 0.4.3, introducing object storage for large HTTP requests and results. The driver now accepts bound batches as large as the request itself, enabling more efficient data transfer. Work included SDK object storage implementation for the development Service, shared conformance validation for request limits and object storage contracts, and strict type checking via mypy and pydoclint. CI improvements added haybarn-cli installation for hello-world tests and refined candidate selection, while documentation clarified that connection commands are CALL table functions.

Language Implementations

The Go implementation (grainlift-go) received updates to handle large HTTP requests and results through S3-compatible storage, with result batches now able to fill responses entirely. The TypeScript SDK (grainlift-typescript) and Python driver (grainlift-python) both released versions 0.3.0–0.3.1, implementing the same object storage and request limit improvements. The grainlift-hello-world-python example was bumped to 0.2.0 with proper connection cleanup via CALL adbc_disconnect, while grainlift-hello-world-go and grainlift-hello-world-typescript were updated to use the latest SDK versions. Documentation across examples now highlights object storage flags and DuckDB's attached catalog patterns.

RPC and Transport Layer

vgi-rpc-go and vgi-rpc-typescript received refinements to handle response size constraints more robustly. The Go implementation now declares body size in zstd frames to prevent decode failures under small response limits, while the TypeScript version (0.25.8) fixed type-checking for consumers without the optional iroh-http-node peer dependency.

Turso Integration

The new grainlift-turso driver reached production readiness with support for Turso Cloud's database engine and concurrent transactions. Work included adding deadlines and cancellation semantics, a comprehensive release workflow for binaries and container images, and production host configuration. Documentation was significantly expanded with a user-focused README, quick-start guide from release downloads, SQL examples using DuckDB's attached catalog, and a changelog. CI improvements ensure unique table names across concurrent jobs.

Query Engine and WebAssembly

haybarn-wasm (1.5.5-rc7) and cupola (0.4.204) benefited from query cancellation improvements. The WebAssembly engine now properly interrupts running queries through connection flags and mirrors HTTPFSParams fields for consistency. Cupola's editor now cancels running queries via the Stop button and utilizes every engine thread for cancellable queries. He also added a feature to remove earlier revisions from report history.

Community Extensions and Ecosystem

haybarn-community-extensions was updated to track latest versions: grainlift driver 0.4.3, adbc_scanner from main (with telemetry thread and decimal fixes), and vgi from main (with cancel dispatcher thread safety improvements). The duckdb-grainlift extension bumped grainlift to 0.4.3, while query-farm-astro received related updates.

Supporting Work

The Cloudflare example and hello-world projects were synchronized with the latest dependency versions. Across all repositories, work emphasized production readiness, proper resource management, and comprehensive documentation. A private repository also received 2 commits.

Overall, this represents a coordinated push to ship object storage support, production hardening, and improved concurrent request handling across the entire Grainlift stack.

Daily Activity Summary — October 1, 2026

RPC and Authentication Infrastructure

Rusty made significant progress on OAuth and device-flow authentication across the RPC ecosystem. In vgi-rpc-rust, he advertised a separate OAuth device-flow client and fixed the resource metadata to properly expose OAuth client fields. The TypeScript SDK (vgi-rpc-typescript) gained support for OAuth PKCE on Node ESM and Workers, while also exporting the Arrow facade as a first-class module (./arrow).

The vgi-rpc-website received a substantial redesign and content refresh, including new transport icons, updated SDK capability summaries, documentation of Iroh transports, improved social previews, and a general restyle aligned with the Query.Farm brand. Product logos and technical concepts now link to their respective documentation.

Grainlift Driver and Gateway

The grainlift driver saw major feature work around OAuth: interactive sign-in flows within the driver, device-flow client support on the gateway, and OAuth discovery and token refresh capabilities. A new release (0.4.1) captured these authentication improvements. Separately, Rusty fixed session recovery to recognize lost sessions regardless of wording and implemented fail-fast timeouts.

The duckdb-grainlift extension received extensive updates to support this new authentication layer. Notable changes include preparing iroh:// endpoints for ATTACH operations, reporting catalog type and version 0.4.0, naming attached databases instead of connection handles, removing rowid from remote tables, adding OAuth refresh-token ATTACH options, and fixing type casting for loosely typed scan results. Autocommit writes now work correctly on non-transactional remotes. Documentation for OAuth ATTACH options was also added.

Data Layer and Database Attachments

Work in cupola focused on integrating Grainlift databases into the query interface, including support for Iroh-based connections. Multiple releases (0.4.197 through 0.4.203) delivered incremental improvements: the old block-grid report builder was removed, Evidence sandboxed components now load and render properly, follow-up messages were added to both the AI chat and report agent workflows, and catalog refresh now correctly clears the grainlift extension's schema cache. Users can now open Grainlift gateways using ?service=grainlift+https://…&target=… parameters.

The adbc_scanner fix ensures that remote tables without rowid properly read a real column for count(*) operations.

TypeScript Runtime and Community Extensions

grainlift-typescript can now run on Cloudflare Workers with flechette support, expanding the reach of the toolkit to serverless environments.

In haybarn-community-extensions, Rusty bumped the grainlift extension repeatedly to track the latest driver improvements: OAuth sign-in, timeout handling, session recovery, loosely typed remote support, D1 write operations, database naming, and iroh endpoint preparation. This ensures community-facing DuckDB installations stay current with the core driver development.

Query.Farm Website Updates

The query-farm-astro site received a small but important fix to the churches map in the VGI project to accommodate Overture's updated places taxonomy, along with a responsive layout adjustment for the hero logo at phone widths.

Publishing and Package Naming

Rusty opened PR #29 on grainlift to resolve a PyPI publishing conflict. The server wheel was named grainlift, which conflicts with the Python worker toolkit (grainlift-python, version 0.2.1) and causes PyPI to reject uploads since only grainlift-python is a trusted publisher for that package name. The solution is to publish the server wheel as grainlift-adbc-gateway instead.