I make DuckDB do things it wasn't built to do.

Query.Farm ships 36 native DuckDB extensions — Kafka topics, Arrow Flight servers, and shell pipes, all queryable from SQL. I write here about what I find in the internals: what breaks, what surprised me, and what I'd do differently.

Rusty Conover GitHub LinkedIn RSS

2M+ Installs

134 VGI repos

1,775 Stars

30 Articles

162 PRs, 90 d

30 Articles Twelve years of notes. Lately: extensions, Arrow, and the seams between them. Full index →

Recent writing

Jul 30, 26 Paging Through a Parquet File in DuckDB: file_row_number or OFFSET? You have a large Parquet file and an API that has to return its contents, but responses have a size ceiling, so the caller pages through it. LIMIT/OFFSET is the obvious way to write that and it is the wrong one. I measured file_row_number against it: 2.53x faster on a file with 163 row groups. Speed is the boring half of the answer. OFFSET will also hand back the wrong rows without telling you. DuckDB Jul 27, 26 A DuckDB Release Isn't a Distribution When DuckDB ships a new version, every community extension has to be rebuilt against it before anyone can install one. I measured how much of the catalog is actually there when a release lands, how long extension updates take to merge, and why the same builds run 2.5x faster on different CI. The answer to all three is distribution work. DuckDB Jun 14, 26 Compiling Isn't Running: Functionally Testing DuckDB-WASM Extensions A DuckDB extension that compiles for WebAssembly has only proven that it compiles. Whether it loads, and whether it actually runs, are separate questions. I built a Node harness to ask them across 124 community extensions. Here's what it found and the fixes that came out of it. DuckDB May 12, 26 Quack and VGI: Two Approaches to Bringing RPC to DuckDB Quack — the new client/server protocol extension for DuckDB — was announced today. It's been built since November 2025, almost exactly as long as I've been building VGI. Here's what's interesting, what's surprising, and where the two projects converge and diverge. DuckDB May 05, 26 DuckDB ↔ Arrow Compatibility: A Status Page A working status page for the Arrow-related issues and PRs I've filed against duckdb/duckdb — the UNION saga, schema/data disagreements in nested appenders, type fidelity through round-trips, and the bigger question of how defensive the C Data Interface should be. Data Engineering Apr 08, 26 GeoSilo vs GeoArrow + Byte Stream Split: Two Approaches to Geometry Compression A detailed comparison of GeoSilo's delta-integer encoding against GeoArrow's columnar float64 layout with Parquet BYTE_STREAM_SPLIT, covering compression ratios, precision tradeoffs, and when each approach wins. Data Engineering Mar 08, 26 VGI Injector: A Tiny HTTPS Download-and-Execute Binary in Zig I needed a self-contained binary that downloads a program over HTTPS and exec's it — small enough to run in a FROM scratch container with nothing else. Go and Rust couldn't get small enough. Zig could. Programming Feb 25, 26 TIME Data Type Compatibility Across Databases A survey of the TIME data type across 14 databases, comparing supported ranges, maximum values, and whether the special value 24:00:00 is accepted. Data Engineering

1,631 Stars, extensions Plus VGI — a protocol for writing DuckDB functions in Python, Rust, Go, Java or TypeScript instead of C++. All projects →

What I build

Name	Function	Stars
airport	Query any Arrow Flight server as if it were a DuckDB table.	345
httpserver	Turns DuckDB itself into a queryable HTTP API server.	284
shellfs	Pipe shell command output straight into a DuckDB scan.	95
clickhouse-sql	ClickHouse SQL dialect compatibility, inside DuckDB.	91
httpclient	HTTP GET and POST from SQL, with no external tooling.	80
lindel	Hilbert and Z-order curves for locality-preserving sort keys.	66
openprompt	Prompt an LLM directly from a SQL query.	60
tributary	Read and write Kafka topics from SQL.	57

3,450 Commits, 90 days Counted nightly from the GitHub API by a Cloudflare Worker, cached in R2. Full activity log →

The record

Fig. 1 — commits per day Peak 220 · Total 3,450

Still building. 3,450 commits and 162 pull requests across the last 90 days , mostly in vgi-java , vgi-rust , haybarn-wasm .

5 Appearances Amsterdam, Paris, and one podcast. All events →

Out loud

DuckDB Extension Development Workshop

Workshop · DuckDB Developer Meeting #1 · Amsterdam, Netherlands · Jan 2026

Building on Flight: Real-World Lessons from the DuckDB Airport Extension

Talk · Apache Arrow Summit 2025 · Paris, France · Oct 2025

Query.Farm: Growing DuckDB Community Extensions

Meetup · DuckDB Amsterdam Meetup #3 · Amsterdam, Netherlands · Sep 2025

DuckDB, Apache Arrow, & the Future of Data Engineering

Podcast · The Hedgineer Podcast (S2E3) · Sep 2025