Activity

A productive day across multiple WebAssembly and data serialization projects, with a focus on performance optimization and correctness fixes.

haybarn-wasm Optimizations

Rusty landed four feature branches improving extension loading and memory usage in haybarn-wasm. The work centered on reducing both time and memory overhead:

• Memory efficiency: Merging PR #30 freed extension bytes immediately after loading, dropping retained buffers from +25.9 MB to +0.03 MB after garbage collection. A follow-up commit in PR #31 further halved load time (from ~2000 ms to ~1000 ms in benchmarks) and reduced memory footprint by ~40%.
• Cache handling: PR #29 fixed a bug where failed extension downloads were cached, causing misleading signature errors on retry. The fix prevents caching when Node's fetch encounters non-2xx responses.
• Chrome code caching: PR #28 enables Chrome's WebAssembly code cache for repeat page loads, cutting db.open() time from ~230 ms to ~87 ms on subsequent visits. This required counting progress on module clones rather than the original buffer.
• Memory-mapped globals: PR #33 optimized global mutability detection by reading bitflags directly from the binary instead of throwing/catching exceptions for each data symbol. Extensions exporting 1,286+ immutable globals see dramatic speedup.

Rusty also pushed a fix to cupola skipping heavyweight extensions during iOS startup, and bumped the workspace to version 1.5.6-rc4.

Upstream Contributions

Three pull requests landed on critical Apache projects addressing memory and correctness issues:

arrow-nanoarrow: Opened PR #974 to fix IPC serialization of null counts—the encoder was writing -1 (unknown count from the C data interface) directly into FieldNodes instead of computing actual counts. Also contributed a validation-level configuration feature and a fix for zero-offset empty arrays with empty offsets buffers (PR #964 and a separate commit).

emscripten: Three PRs tackle WebAssembly module loading efficiency. PR #28007 frees side-module file data after all threads load it (addressing a TODO in load_library_done). PR #28006 reads exported global mutability from the binary instead of testing it via throw/catch per symbol. PR #28002 ensures wasm bytes are released after compilation rather than held for the program's lifetime.

flatbuffers: Opened PR #9414 fixing Go vtable de-duplication to consider object size, preventing smaller objects' vtables from being incorrectly reused by larger objects.

duckdb-wasm: Two PRs mirror the haybarn-wasm work—PR #2279 fixes extension download caching and PR #2278 enables Chrome code caching for repeat loads.

Diagnostics

Opened two issues documenting edge cases: arrow-dotnet #462 (ArrowArrayConcatenator failing on zero-length string arrays with empty offsets) and flatbuffers #9413 (vtable de-duplication not considering object size).

A prolific day spanning infrastructure, format standardization, and performance work across the Apache Arrow ecosystem and DuckDB/Haybarn projects.

Apache Arrow Format & Serialization

Rusty opened a series of coordinated PRs across multiple Arrow implementations to add custom_metadata to DictionaryEncoding (tracking upstream proposal apache/arrow#49704). This solves a long-standing serialization ambiguity where dictionary-encoded extension types and extensions with dictionary storage couldn't be distinguished over IPC. The work spans:

• PR #53994 on arrow for the format specification and C++ implementation
• PR #972 on arrow-nanoarrow to decode and write the new metadata field
• PR #11622 on arrow-rs adding the Rust implementation
• PR #1381 on arrow-go for Go support

Additionally, he opened several focused IPC improvements on arrow-nanoarrow: fixing metadata truncation at NUL bytes (PR #971, issue #970), adding support for dictionary value type metadata in writers (PR #968), and allowing message metadata attachment (PR #967). PR #965 and #964 optimize the stream reader by referencing message bodies in place and making validation configurable.

DuckDB/Haybarn WebAssembly

Multiple commits to haybarn-wasm addressed dynamic linking performance and memory threading issues. PR #26 resolves dynamic-linking exports lazily at startup, cutting initialization from ~39 ms to ~2 ms with 40k exports. PR #27 keeps heap views current when other threads grow memory, fixing intermittent RangeError crashes on the COI (pthreads) bundle. Complementary work landed in PR #2276 on duckdb-wasm applying the same heap view fix. A separate PR #25 refactors JSON insertion to use the engine's native read_json rather than maintaining a parallel JSON reader.

Emscripten Startup Optimization

Two performance PRs opened on emscripten-core/emscripten: PR #27993 creates GOT entries lazily during dynamic linking to speed up MAIN_MODULE startup by up to 40%, and PR #27996 eliminates unnecessary export object copying in signature conversions, dropping that step from 39 ms to 2 ms.

DuckDB Extensions & Integrations

On duckdb-duckdb-nanoarrow, PR #129 adds dictionary encoding to the Arrow IPC stream writer using nanoarrow's DictionaryBatch encoder, and PR #128 bumps nanoarrow to pick up upstream IPC dictionary decoding fixes. PR #3093 on duckdb/community-extensions updates the a5 extension to latest commits. cupola upgraded to Haybarn 1.5.6-rc3, and a5 itself was upgraded to 1.0.0. Documentation work on query-farm-astro covers the new a5 1.0 features (cell IDs, coverings, set operations).

Community Support

Rusty replied to issues on apache/arrow#49704 (dictionary extension roundtripping), apache/arrow-nanoarrow#622 (dictionary IPC support), and emscripten-core/emscripten#17891 (table growth strategy), providing guidance on cross-implementation coordination and performance tradeoffs.

Other Updates

A commit to cloudflare/cloudflare-docs added Grainlift to D1 community projects. Four new issues opened tracking test parallelization (arrow-nanoarrow#969), metadata handling in JSON comparison (#973), and feature tracking across Arrow implementations.

A focused day of release coordination and maintenance across the Query.Farm ecosystem, with attention to component versioning and documentation.

vgi-excel and haybarn-wasm received release-preparation commits. The WASM package bumped to version 1.5.6-rc2, with a fix to report the actual release version rather than git describe's dev suffix. Package README files were also branded with Haybarn and Query.Farm logos and linked back to parent repositories.

cupola was upgraded to consume Haybarn WASM 1.5.6 (0.4.250) and its extensions. Rusty removed an unsupported Haybarn MVP fallback and refactored documentation by splitting CLAUDE.md into path-scoped .claude/rules files for better organization.

haybarn-community-extensions pulled in the latest upstream changes from the main Haybarn branch, incorporating fixes for Arrow schema lifecycle management, performance optimizations for large-string deduplication, and worker pool idle limit tuning (5s → 60s).

query-farm-astro saw three updates: refreshed extension load counts across a 90-day window, updated to Haybarn v1.5.6-rc1, and pinned to the rc2 build after release. An additional fix ensured alias extensions remain distinct in the Pagefind search index.

Rusty also replied to a security notice on vgi-typesafe (issue #1) regarding a potential decision hijacking risk in Jev-based applications, providing community support on the concern.

Private repository work continued across three separate projects.

vgi-polymarket received a commit normalizing repeated price-history observations and rejecting conflicting prices, improving data consistency in the polymarket integration.

vgi-rust shipped version 0.44.0 and added typed pruning and explicit worker filtering contracts, expanding the query engine's capabilities for more precise data filtering and worker task management.

haybarn-rust underwent a comprehensive rebrand and release cycle. The work included:

• Releasing version 1.10506.0-rc.2 rebased onto upstream duckdb-rs
• Maintaining backward compatibility by keeping stream_arrow_auto as a deprecated alias of stream_arrow
• Replacing upstream CI workflows with Haybarn-specific CI and enabling Trusted Publishing for crates.io
• Updating documentation with rebranded README, NOTICE, and CLAUDE.md files plus a smoke test example
• Renaming crates from duckdb-rs to haybarn, libduckdb-sys to libhaybarn-sys, and duckdb-loadable-macros to haybarn-loadable-macros
• Re-pointing DuckDB sources to the Haybarn engine and regenerating the amalgamation

haybarn-go-bindings pushed updated libraries to support the Haybarn rebrand.

haybarn-community-extensions synced upstream community-extensions for the v1.5.6 sweep.

Opened PR #65 on duckdb-zarr leveraging the new VGI 0.44 typed pruning and diagnostics APIs. The PR fixes coordinate pruning cases that discarded valid rows, resolves large integer precision loss, handles strings returning NULL, and treats HTTP 403 responses appropriately rather than assuming absent chunks.

vgi-github generated connected GitHub semantic models and researcher workloads for analysis and integration.

On open-source stewardship, Rusty added Radio to the IXWebSocket users list on machinezone/IXWebSocket (PR #614), opened issue #1 on KLL-Polymer-code requesting a LICENSE.txt file, and replied to issue #38484 on apache/arrow regarding subsidiary logos for Arrow subprojects.

Private repository activity included 3 commits across 1 private repo.

Rusty focused on performance optimization and feature refinement across multiple repositories today, with particular attention to the cupola workspace and supporting infrastructure.

Work on vgi-rust centered on a performance improvement: caching decoded window partitions with bounded retention to reduce redundant computation during query planning.

The haybarn-wasm build system received updates, with a workspace version bump to 1.5.6-rc1 and an upgrade to emscripten 6.0.12. These changes prepare the WebAssembly runtime for release. The haybarn-community-extensions repository was similarly updated with a CI target bump to v1.5.6 to align with the upstream engine version.

Most of the day's effort went into cupola, where Rusty shipped a series of fixes and features across seven commits:

• Access control discovery: Fixed scheduled access policy and caller email resolution (0.4.243)
• Library consolidation: Unified library saving and document recovery (0.4.245), and consolidated reports, notebooks, and queries into a single worker library abstraction (0.4.242)
• Bundle optimization: Removed redundant nested reporting schemas and unused response schemas from the browser bundle, reducing client-side payload
• Editor integration: Fixed library document opening to use their normal editors rather than falling back to generic views (0.4.246)
• Protocol improvements: Adopted the complete Identity protocol with unsupported fallbacks for better compatibility (0.4.244)
• UI polish: Refined the browsing experience for saved objects and worker schedules (0.4.247)

Beyond internal work, Rusty provided community support by replying to PR #22715 on duckdb/duckdb regarding Arrow export support for the VARIANT type.

Private repository activity included 2 commits across 1 private project.

Cloudflare API codegen expansion. Work on vgi-cloudflare pushed significant progress in resource coverage, expanding from 1,142 to 1,684 queryable resources through improved code generation. The commit emphasizes stable naming conventions, a critical foundation for maintaining consistency as the resource set grows.

GitHub API refinements. Two commits landed on vgi-github: one expanded read-only API coverage with newly typed datasets, and a follow-up pinned CI and live workflows to Python 3.14, ensuring compatibility across the project's automation infrastructure.

Community support. Rusty replied to issue #45073 on microsoft/vcpkg, investigating a cache failure scenario in GitHub Actions workflows — contributing troubleshooting insights to the broader vcpkg tooling ecosystem.

A lighter day on October 11th, with activity focused on private work. Rusty made a single commit to a private repository, keeping progress moving on undisclosed projects.

Daily Activity Summary — October 10, 2026

cupola — Report Scheduling and UI Refinements

Rusty shipped multiple improvements to cupola across versions 0.4.239 and 0.4.240. The work focused on scheduled report exports and email delivery, with a new isolated headless renderer added to handle report generation without blocking the main UI thread. Error handling received attention as well—SQL errors in reports now retain full context so users can debug issues while keeping the editing interface accessible. A separate fix prevented native dropdowns from clipping across the application, addressing a recurring UI issue.

adbc_scanner — Schema and Write Capability Fixes

Two significant issues were resolved in adbc_scanner to improve compatibility with autocommit-only ADBC drivers:

• Schema handling: Attached databases now correctly inherit the driver's default schema instead of always using "main". This fix enables drivers like PostgreSQL (which default to the "public" schema) to work properly with attached database references like db.table and USE db statements.
• Write operations: Catalog writes now function for autocommit-only drivers, fixing failures when executing INSERT INTO db.t and CREATE TABLE db.t AS statements even outside transaction blocks.

Rusty fielded community questions on both issues (issue #19 and #20), providing context for the fixes.

adbc_scanner PR and Community Extensions

Opened PR #3016 on duckdb/community-extensions to bring the community extension's adbc_scanner version up to date with the latest commits, ensuring users have access to the schema and write operation improvements.

haybarn-community-extensions — Dependency Bump

A substantial upstream bump was applied to align with main branch commit 74015fc. The changelog reflects deep infrastructure work across the haybarn ecosystem: structured error handling with extra_info vocabularies, execution and transport layer improvements, worker exception reporting, OAuth and auth/config refinements, and catalog reflection fixes. Notably, invocation_id was renamed to execution_id throughout the codebase for consistency.

Private Repository Work

11 commits were pushed across 3 private repositories, continuing parallel development efforts.

A high-volume day focused on open-source stewardship, documentation standardization, and security fixes across multiple projects.

Community Extensions & Documentation

Rusty made substantial contributions to the DuckDB ecosystem by standardizing documentation links across duckdb/community-extensions. This involved updating links for 24 extensions—including airport, bitfilters, cronjob, crypto, datasketches, and many others—to ensure consistent documentation formatting. Beyond documentation, he also took on maintainer roles for five community extensions: webmacro, cronjob, quickjs, redis, and tsid.

URI Credential Security Fixes

A significant security issue was addressed across two related repositories. In vgi-rpc-rust, commits fixed a critical bug where JWKS fetch errors were echoing URL credentials and query parameters in error messages. This fix propagated to grainlift, where he released version 0.6.1 with URI credential redaction. The fix ensures that sensitive authentication information (user:password@ patterns) is stripped from URIs before logging or error reporting, with documentation clarifying that OAuth messages were the source of the leak, not the reqwest HTTP client itself.

Cupola Report Storage & UI Improvements

cupola saw a flurry of activity across multiple releases (0.4.225–0.4.230). Key improvements included:

• Integration with HTTP report stores, unifying report storage browsing and transfers
• Ability to browse and compare worker report revisions
• UI fixes aligning report controls and editor saves with the worker contract
• Resolution of a Pivot catalog table discovery bug (quoted Perspective table IDs)
• Improved HTTPS localhost connection error messaging
• Test enhancements including e2e test isolation and regression testing for protocol 0.5 session schemas

Filter Library Documentation

On FastFilter/xor_singleheader, documentation was added for DuckDB bindings for XOR and binary fuse filters, supporting PR #79.

Routine Maintenance

Other updates included releasing grainlift 0.6.1 with the credential redaction fix and vgi-rpc 0.31.4 dependency bump, CI toolchain repinning on grainlift, and housekeeping across davidgasquez/awesome-duckdb to update Query.Farm extension links.

A prolific day across the Query.Farm ecosystem and upstream community projects. Work focused on bug fixes, extension maintenance, and broad ecosystem documentation.

RPC and observability work. vgi-rpc-go received multiple stability fixes: resolved a conflict where OpenTelemetry and Sentry instrumentation hooks were replacing each other, fixed an issue where only one dispatch hook could be held at a time, and corrected trace correlation loss in access logging. Two patch releases shipped as v0.34.0 and v0.34.1.

Extension updates and releases. quickjs was bumped to DuckDB v1.5.6 and QuickJS-NG v0.17.0, with the extension version incremented to 2026100801. redis merged PR #10 adding TTL support (EXPIRE, TTL, EXPIREAT functions) and updated its DuckDB submodule to the v1.5-variegata branch. cupola released v0.4.223 with multi-catalog inventory and AI prompt caching fixes, with hardening applied across those systems and DDL navigation. airport fixed an issue where Flight exchange lifetime was materializing results unnecessarily. radio fixed VARCHAR message handling—payloads were being escaped, and all messages were sent as binary frames instead of text where appropriate. vgi-csharp corrected run_tests.sh to execute the actual test tree rather than a compiled-in unittest version.

Community extension submissions. Rusty opened 16 PRs submitting Query.Farm extensions to upstream projects' adoption and integration lists: Crypto to BLAKE3, Hashfuncs to xxHash and RapidHash, Tributary to Kafka integrations and librdkafka bindings, Bitfilters to XOR and binary fuse filters, Marisa to marisa-trie bindings, DataSketches adapter to the DataSketches website, Radio to IXWebSocket users, MiniJinja integration to MiniJinja use cases, and Query.Farm projects to Apache Arrow's Powered By page. Additionally, he submitted comprehensive updates to awesome-duckdb covering six existing extension entries and filling in missing projects from the Query.Farm catalog.

Community extension registry updates. Opened 15 PRs against duckdb/community-extensions to sync the latest commits for: tsid, stochastic, shellfs, redis, radio, quickjs, openprompt, minijinja, lindel, json_schema, inflector, httpserver, hashfuncs, fuzzycomplete, cronjob, and geosilo.

Awesome list submissions. Submitted documentation improvements to multiple community curated lists: added DuckDB projects to awesome-typesafe-jev, awesome-adbc, awesome-kafka, awesome-duckdb-spatial, and awesome-iroh. Also opened an issue against json-schema-org/website proposing Query.Farm's JSON Schema for DuckDB in the ecosystem.

Website and documentation. query-farm-astro published an article on Iroh browser SQL with diagrams and metadata, connected product pages and technical guides to published articles, updated VARIANT and Arrow support tracking, and documented new TTL functions in Redis extension documentation. datasketches-website and quickjs-ng received commits documenting their respective DuckDB adapters.

Community support. Replied to issues on a5 regarding overlapping polygon detection and on quickjs about DuckDB community extension details. Also provided community stewardship across upstream projects with targeted, well-documented submissions.

Private activity. 16 commits across 2 private repositories.

Rusty delivered a major coordinated release across the VGI ecosystem today, addressing security, stability, and protocol improvements across multiple language implementations.

VGI RPC Layer

The vgi-rpc-cpp library received several critical fixes bundled into releases 0.8.1 through 0.9.0. A bearer authenticator that refused anonymous callers now supports optional bearer authentication via HttpConfig::bearer_optional. The access log was oversharing request payloads and stream state tokens—this has been corrected. Additionally, serve_unix workers lacked an idle timeout, causing launched processes to hang indefinitely; this is now fixed. The reflection client tests were reformatted to match CI's clang-format 22.1 standards.

Similar fixes landed in vgi-rpc-rust (releases 0.31.1–0.31.3) and vgi-rpc-go (releases 0.33.1–0.33.2), where request payloads were inadvertently reaching access logs and unary records still carried a transitional payload_omitted marker. The vgi-rpc-typescript library received corresponding updates for request data logging at DEBUG level.

VGI Core Implementations

vgi-java saw substantial protocol advancement across releases 0.39.0–0.41.1. Port attach tickets now allow a runner holding only a grant to reattach a user's catalog using a sealed ticket. Opaque values are no longer unsealed with a fallback—they either validate or reject, and secret options no longer appear in the attach ID. The VgiService is now generated from the reference specification rather than hand-written, and the vgi.v2 registry reflects this change with the reference hash reported by reflection. AttachTickets now seal using vgirpc's XChaCha20Poly1305 instead of a local copy. HTTP lane error handling was improved so worker errors are treated as skips, and the full vgi.v2 surface is now hosted.

In vgi-cpp, attach tickets introduce a critical security feature: a user's ATTACH is sealed so only a runner holding their grant can replay it. Opaque attach data is now sealed and bound to the caller, eliminating plaintext secret options. The integration harnesses were fixed to fail on every HTTP-related worker error rather than silently skipping them, and the vgi.v2 hash is pinned to the reference to prevent surface drift. Launched workers now honour the --idle-timeout flag on --unix sockets. The vgi.v2 registry is generated from the reference specification, and optional bearer support surfaces the principal. Releases span v0.8.0 through v0.10.1.

vgi-rust (releases 0.41.0–0.43.0) mirrors these advances: attach tickets seal user ATTACHes for runner replay, opaque data is sealed on HTTP, and integration lanes no longer hide failures as skips. The vgi.v2 surface is now fully reference-generated with a hash matching vgi-python 0.43.0. HTTP test infrastructure was cleaned up—run_http_tests.sh was leaking all five servers on every run. CI now runs unit tests in the staged directory where HTTP workers live.

Grainlift and Integration

grainlift-go now advertises per-connection statistics support via Grainlift protocol 0.5. The grainlift-cloudflare deployment was updated to protocol 0.5 and declares statistics as unsupported; PR #1 merging durable-objects-and-permissions support was integrated.

Auxiliary Work

cupola received a fix to open the default schema and simplify empty sidebar sections. Across the ecosystem, integration test scripts in vgi-java and elsewhere were cleaned up to prevent HTTP worker leaks during CI runs. Overall, Rusty coordinated security hardening (sealed attach tickets, secret option protection), stability improvements (idle timeouts, error visibility), and specification alignment (vgi.v2 generated from reference) across 22 repositories.

A particularly active day across the VGI ecosystem, with 91 commits spanning 18 repositories and focused work on grant-based authentication, catalog capabilities, and cross-language RPC implementations.

Grant Authentication Across the Stack

The dominant theme was rolling out bearer credential support for minted grants across all VGI RPC language bindings. vgi-rpc-cpp (v0.7.0), vgi-rpc-rust (v0.30.0), vgi-rpc-go (v0.32.0), vgi-rpc-typescript (v0.27.0), vgi-rpc-java (v0.29.0), and vgi-rpc-csharp (v0.14.0) all received updates to accept sealed grants from issue_grant and resolved tokens as HTTP bearer credentials. This represents a significant authentication flow improvement across the distributed worker infrastructure.

Core Language Workers

vgi-rust received dual releases (v0.39.0 and v0.40.0) adding catalog_contents fixture catalogs, DDL-capable in-memory catalogs, and client-side load_catalog support. The worker now accepts sealed grants via HTTP bearer authentication configured through --grant-key or VGI_RPC_GRANT_KEYS. A flaky test suite issue was addressed where tampered-grant tests intermittently passed with real grants.

vgi-cpp (v0.7.0) underwent substantial work: ported five missing fixtures to published-extension lanes, reverted and re-applied catalog_contents fixture handling, served six catalog_contents fixture catalogs, and delivered DDL-capable memory catalogs. A critical scalar bug was fixed where result() moved an array before reading its length on x86-64. Build parallelism was capped at 2 to prevent GCC runners from exhausting memory. The worker now properly authenticates incoming grant-based calls.

vgi-java shipped two releases (v0.37.0 and v0.38.0) with catalog_contents fixture support for code-defined and in-memory DDL catalogs, plus grant-based bearer authentication.

vgi-csharp (v0.14.0 and v0.15.0) added catalog_contents fixture catalogs, DDL-capable in-memory catalogs, and full support for sealed-grant and resolve_token authentication over HTTP.

Specialized Workers & Extensions

vgi-polars (v0.9.0–0.9.1) tackled several improvements: migrated to PyPI-hosted vgi-python with dependency pinning to v0.41.0, added Filter Encoding v2 support for pushed-down filters, fixed string/binary is_in needle handling, resolved hanging scans that ended before their streams, and implemented proper cleanup of abandoned table_function streams. Polars 2.0 is now the build target with 1.x still supported.

vgi-lint-check fixed buffering-function argument attachment and improved simulation output to show result columns.

vgi-rpc-go (v0.32.0) extended bearer credential support to HTTP-based sealed grants and resolve_token flows.

Frontend & User-Facing Tools

cupola saw steady improvements across three releases (v0.4.219–0.4.221): bulk catalog content loading with legacy fallback, sidebar context menus for reports and notebooks, file attachment support across AI assistants, HTTP request abort handling on cancellation, Iroh node startup deferred to first use for performance, upgrade notification dismissal, and local document sidebar organization with restored report creation.

haybarn-wasm bumped to v1.5.5-rc8 to support request cancellation in flight.

Open Source & Private Work

Rusty also contributed to the duckdb/duckdb-wasm repository as part of broader community engagement. Additionally, 22 commits across 4 private repositories indicate ongoing internal development.

A prolific day across the Query Farm ecosystem, with 91 commits spanning 17 repositories and significant feature development in both backend services and the Cupola UI.

VGI Language Bindings & Core Updates

Rusty pushed releases and maintenance updates across the VGI stack. The vgi-rust crate received version 0.37.1 with documentation improvements, including fixes to broken intra-doc links in vgi-client and removal of secrets guidance from attach-option docs. Similar maintenance appeared in vgi-java with CI adjustments to handle upstream attach_secrets skips. The vgi-excel binding was prepared for Cupola 0.5.1, pinning VGI releases and adding sign-in recovery support.

Haybarn WASM & HTTP Layer

Work on haybarn-wasm focused on request lifecycle management. He upgraded the engine dependency and implemented abort support for HTTP requests in flight, ensuring httpfs requests properly carry the query's interrupt flag. This honors request cancellation in retry loops, improving responsiveness when queries are stopped mid-execution.

Cupola: Major Multi-Release Sprint

The bulk of today's effort landed in cupola, with a coordinated sequence of 8 releases (0.4.208–0.4.216) delivering substantial UI and workspace features. Early releases tackled inspector and editor polish: sidebar Inspector with call snippets and signature help (0.4.209), per-tab revision history and a redesigned split Run button (0.4.214), distinct summary and documentation display (0.4.210), Inspector sections with view SQL (0.4.211), and toolbar regrouping plus a History diff stack (0.4.212). The inspector also now stringifies LIST columns in Perspective snapshots (0.4.208).

Following these incremental improvements, he merged multi-catalog phases 3A, 3B, and 3C. Phase 3A introduced the workspace manager editor for managing multiple catalogs. Phase 3B added workspace files, DuckDB script export/import, and JSON Schema support. Phase 3C brought alias rename dialogs, report requires with Rebind/Attach, command palette, and a tokenizer for catalog alias references. Supporting these features are structured attach options with a typed options form and consent screen, plus a portable workspace format with aliases and #ws= codec for catalog-qualified routing.

Final releases in the sequence (0.4.215–0.4.216) delivered native SQL notebooks with charts and AI editing, query cancellation consistency across all surfaces, and fixes to arrowLosslessConversion and R2 cache policy handling. Dependencies were updated to vgi 0.37.1 and test workers to vgi-python 0.38.

Ecosystem & Private Work

Additional activity touched vgi-rpc-typescript, vgi-csharp, vgi-go, vgi-rpc-python, vgi-rpc-csharp, vgi-github, adbc_scanner, query-farm-astro, and community extensions. He also contributed 13 commits across 2 private repositories, and engaged in 8 community support interactions—likely reviewing or advising on DuckDB extensions and open-source projects.

Rusty released version 0.4.207 of cupola with a notable feature addition: a resizable value panel and formatted JSON display in Lines mode. The work involved both the feature implementation and expanded test coverage for results handling, as captured in the release commit and preceding feature branch work.

Over on arrow-rs, he opened PR #11374 to address issue #9837, introducing a zero-copy pull-based BufferStreamReader for IPC streams. The motivation is clear: when an entire IPC stream is already loaded in memory—whether from an HTTP body, memory-mapped file, shared-memory segment, or bytes::Bytes from object storage—the existing StreamReader<R: Read> API forces unnecessary copying. This new approach eliminates that overhead by leveraging the data already in hand.

Rusty also weighed in on the DuckDB community with feedback on PR #22715, offering support for the Arrow export enhancement targeting VARIANT type support. His involvement across these three projects reflects ongoing work on columnar data handling, zero-copy optimizations, and interoperability between query execution and Arrow serialization.

Rusty shipped two releases of cupola focused on improving how query results are displayed. The first release (0.4.205) added support for reading full values in query results through a new value panel and Lines layout. He then refined the presentation in 0.4.206 by restyling the Lines results layout to display results as row cards with a key/value table structure, improving readability and visual hierarchy.

Work on vgi-github centered on enhancing the GitHub data integration extension with better user experience and observability. The README was rewritten to focus on user-facing documentation, with internal details moved to a separate DEVELOPMENT.md file. On the technical side, Rusty added a token_source ATTACH option that allows users to supply their own GitHub login credentials rather than relying on a default token. Servers can now opt into this feature via the VGI_GITHUB_ALLOW_TOKEN_SOURCE environment variable. He also implemented logging improvements to report rate-limit waits, retries, and token budget information to the DuckDB log, giving users better visibility into API usage and performance.

A major release day across the Grainlift ecosystem, with significant work on object storage support, request handling, and production readiness across multiple language implementations and supporting libraries.

Core Grainlift Framework

The grainlift repository reached version 0.4.3, introducing object storage for large HTTP requests and results. The driver now accepts bound batches as large as the request itself, enabling more efficient data transfer. Work included SDK object storage implementation for the development Service, shared conformance validation for request limits and object storage contracts, and strict type checking via mypy and pydoclint. CI improvements added haybarn-cli installation for hello-world tests and refined candidate selection, while documentation clarified that connection commands are CALL table functions.

Language Implementations

The Go implementation (grainlift-go) received updates to handle large HTTP requests and results through S3-compatible storage, with result batches now able to fill responses entirely. The TypeScript SDK (grainlift-typescript) and Python driver (grainlift-python) both released versions 0.3.0–0.3.1, implementing the same object storage and request limit improvements. The grainlift-hello-world-python example was bumped to 0.2.0 with proper connection cleanup via CALL adbc_disconnect, while grainlift-hello-world-go and grainlift-hello-world-typescript were updated to use the latest SDK versions. Documentation across examples now highlights object storage flags and DuckDB's attached catalog patterns.

RPC and Transport Layer

vgi-rpc-go and vgi-rpc-typescript received refinements to handle response size constraints more robustly. The Go implementation now declares body size in zstd frames to prevent decode failures under small response limits, while the TypeScript version (0.25.8) fixed type-checking for consumers without the optional iroh-http-node peer dependency.

Turso Integration

The new grainlift-turso driver reached production readiness with support for Turso Cloud's database engine and concurrent transactions. Work included adding deadlines and cancellation semantics, a comprehensive release workflow for binaries and container images, and production host configuration. Documentation was significantly expanded with a user-focused README, quick-start guide from release downloads, SQL examples using DuckDB's attached catalog, and a changelog. CI improvements ensure unique table names across concurrent jobs.

Query Engine and WebAssembly

haybarn-wasm (1.5.5-rc7) and cupola (0.4.204) benefited from query cancellation improvements. The WebAssembly engine now properly interrupts running queries through connection flags and mirrors HTTPFSParams fields for consistency. Cupola's editor now cancels running queries via the Stop button and utilizes every engine thread for cancellable queries. He also added a feature to remove earlier revisions from report history.

Community Extensions and Ecosystem

haybarn-community-extensions was updated to track latest versions: grainlift driver 0.4.3, adbc_scanner from main (with telemetry thread and decimal fixes), and vgi from main (with cancel dispatcher thread safety improvements). The duckdb-grainlift extension bumped grainlift to 0.4.3, while query-farm-astro received related updates.

Supporting Work

The Cloudflare example and hello-world projects were synchronized with the latest dependency versions. Across all repositories, work emphasized production readiness, proper resource management, and comprehensive documentation. A private repository also received 2 commits.

Overall, this represents a coordinated push to ship object storage support, production hardening, and improved concurrent request handling across the entire Grainlift stack.

Daily Activity Summary — October 1, 2026

RPC and Authentication Infrastructure

Rusty made significant progress on OAuth and device-flow authentication across the RPC ecosystem. In vgi-rpc-rust, he advertised a separate OAuth device-flow client and fixed the resource metadata to properly expose OAuth client fields. The TypeScript SDK (vgi-rpc-typescript) gained support for OAuth PKCE on Node ESM and Workers, while also exporting the Arrow facade as a first-class module (./arrow).

The vgi-rpc-website received a substantial redesign and content refresh, including new transport icons, updated SDK capability summaries, documentation of Iroh transports, improved social previews, and a general restyle aligned with the Query.Farm brand. Product logos and technical concepts now link to their respective documentation.

Grainlift Driver and Gateway

The grainlift driver saw major feature work around OAuth: interactive sign-in flows within the driver, device-flow client support on the gateway, and OAuth discovery and token refresh capabilities. A new release (0.4.1) captured these authentication improvements. Separately, Rusty fixed session recovery to recognize lost sessions regardless of wording and implemented fail-fast timeouts.

The duckdb-grainlift extension received extensive updates to support this new authentication layer. Notable changes include preparing iroh:// endpoints for ATTACH operations, reporting catalog type and version 0.4.0, naming attached databases instead of connection handles, removing rowid from remote tables, adding OAuth refresh-token ATTACH options, and fixing type casting for loosely typed scan results. Autocommit writes now work correctly on non-transactional remotes. Documentation for OAuth ATTACH options was also added.

Data Layer and Database Attachments

Work in cupola focused on integrating Grainlift databases into the query interface, including support for Iroh-based connections. Multiple releases (0.4.197 through 0.4.203) delivered incremental improvements: the old block-grid report builder was removed, Evidence sandboxed components now load and render properly, follow-up messages were added to both the AI chat and report agent workflows, and catalog refresh now correctly clears the grainlift extension's schema cache. Users can now open Grainlift gateways using ?service=grainlift+https://…&target=… parameters.

The adbc_scanner fix ensures that remote tables without rowid properly read a real column for count(*) operations.

TypeScript Runtime and Community Extensions

grainlift-typescript can now run on Cloudflare Workers with flechette support, expanding the reach of the toolkit to serverless environments.

In haybarn-community-extensions, Rusty bumped the grainlift extension repeatedly to track the latest driver improvements: OAuth sign-in, timeout handling, session recovery, loosely typed remote support, D1 write operations, database naming, and iroh endpoint preparation. This ensures community-facing DuckDB installations stay current with the core driver development.

Query.Farm Website Updates

The query-farm-astro site received a small but important fix to the churches map in the VGI project to accommodate Overture's updated places taxonomy, along with a responsive layout adjustment for the hero logo at phone widths.

Publishing and Package Naming

Rusty opened PR #29 on grainlift to resolve a PyPI publishing conflict. The server wheel was named grainlift, which conflicts with the Python worker toolkit (grainlift-python, version 0.2.1) and causes PyPI to reject uploads since only grainlift-python is a trusted publisher for that package name. The solution is to publish the server wheel as grainlift-adbc-gateway instead.