Data infrastructure engineering, DuckDB extensions, and building tools that scale.https://rusty.today/https://rusty.today/blog/time-data-type-compatibility-across-databases/https://rusty.today/blog/time-data-type-compatibility-across-databases/A survey of the TIME data type across 14 databases, comparing supported ranges, maximum values, and whether the special value 24:00:00 is accepted.Wed, 25 Feb 2026 00:00:00 GMThttps://rusty.today/blog/query-farm-telemetry-client-for-duckdb-extensions/https://rusty.today/blog/query-farm-telemetry-client-for-duckdb-extensions/I open-sourced the telemetry client I use across Query.Farm's DuckDB extensions. It's two files, one function call, and it only collects platform and version info.Mon, 23 Feb 2026 00:00:00 GMThttps://rusty.today/blog/vgi-rpc-transport-agnostic-python-rpc-framework/https://rusty.today/blog/vgi-rpc-transport-agnostic-python-rpc-framework/I built an RPC framework for Python that uses Apache Arrow IPC as the wire format and Python Protocol classes as the interface definition. No .proto files, no codegen — just type annotations.Sat, 21 Feb 2026 00:00:00 GMThttps://rusty.today/blog/inflector-acronym-aware-case-conversions/https://rusty.today/blog/inflector-acronym-aware-case-conversions/The Inflector extension for DuckDB now supports configurable acronyms, so case conversions preserve terms like HTML, API, and URL as fully uppercase — configured through a native DuckDB setting.Fri, 20 Feb 2026 00:00:00 GMThttps://rusty.today/blog/linkedin-2026-02-13-i-built-a-thing-for-runners-and-walkers-who-are-tired-of-che/https://rusty.today/blog/linkedin-2026-02-13-i-built-a-thing-for-runners-and-walkers-who-are-tired-of-che/Fair Weather scores every hour of the next 48 hours on a 0–100 scale based on temperature, humidity, wind, UV, precipitation, air quality, and daylight —...Fri, 13 Feb 2026 00:00:00 GMThttps://rusty.today/blog/linkedin-2026-01-10-i-m-teaching-a-hands-on-duckdb-extension-development-worksho/https://rusty.today/blog/linkedin-2026-01-10-i-m-teaching-a-hands-on-duckdb-extension-development-worksho/If you've ever wanted to extend DuckDB with custom functions, table functions, or new capabilities — this is your chance to learn how. In this free 2.5-hour...Sat, 10 Jan 2026 00:00:00 GMThttps://rusty.today/blog/linkedin-2025-12-10-announcing-the-adbc-scanner-extension-for-duckdb/https://rusty.today/blog/linkedin-2025-12-10-announcing-the-adbc-scanner-extension-for-duckdb/We’re excited to introduce our newest DuckDB community extension: ADBC Scanner — a fast, flexible way to connect DuckDB to external databases using Arrow...Wed, 10 Dec 2025 00:00:00 GMThttps://rusty.today/blog/linkedin-2025-12-08-the-duckdb-extension-workshop-in-amsterdam-this-january-is-a/https://rusty.today/blog/linkedin-2025-12-08-the-duckdb-extension-workshop-in-amsterdam-this-january-is-a/Since I live on the U.S. East Coast, I’d be happy to hold a workshop here in the USA. If you're interested, comment to help decide a location and date. If...Mon, 08 Dec 2025 00:00:00 GMThttps://rusty.today/blog/linkedin-2025-12-03-big-update-to-the-duckdb-crypto-extension/https://rusty.today/blog/linkedin-2025-12-03-big-update-to-the-duckdb-crypto-extension/I'm excited to share today’s release of the crypto extension, bringing improvements and new capabilities: Updated crypto_hash API – This function now...Wed, 03 Dec 2025 00:00:00 GMThttps://rusty.today/blog/linkedin-2025-12-03-deep-dive-into-duckdb-extensions-let-s-workshop-in-person/https://rusty.today/blog/linkedin-2025-12-03-deep-dive-into-duckdb-extensions-let-s-workshop-in-person/I’ll be teaching a free, 2-hour workshop about DuckDB extension development the morning of DuckDB Developer Day. 📅 Date: January 30, 2026 📍 Location:...Wed, 03 Dec 2025 00:00:00 GMThttps://rusty.today/blog/linkedin-2025-12-01-excited-to-share-something-very-cool-with-the-duckdb-arrow-f/https://rusty.today/blog/linkedin-2025-12-01-excited-to-share-something-very-cool-with-the-duckdb-arrow-f/For everyone building on the Airport extension for DuckDB https://airport.query.farm/ - there is now support for Go from Vladimir Gribanov. airport-go — a Go...Mon, 01 Dec 2025 00:00:00 GMThttps://rusty.today/blog/linkedin-2025-11-20-i-ve-been-deep-in-kafka-duckdb-development-the-past-few-week/https://rusty.today/blog/linkedin-2025-11-20-i-ve-been-deep-in-kafka-duckdb-development-the-past-few-week/🆕 Created a Kafka topic with 8 partitions on my M3 MacBook Air (one per core) ⚡ Inserted 50M messages from DuckDB (47 bytes each) in 11.62s (4.3 million...Thu, 20 Nov 2025 00:00:00 GMThttps://rusty.today/blog/linkedin-2025-11-09-excited-to-announce-the-inflector-extension-for-duckdb-by-qu/https://rusty.today/blog/linkedin-2025-11-09-excited-to-announce-the-inflector-extension-for-duckdb-by-qu/Transform string cases and inflect column names directly in SQL—no more wrestling with naming conventions across your data pipeline. What does it do? ✅...Sun, 09 Nov 2025 00:00:00 GMThttps://rusty.today/blog/linkedin-2025-11-09-help-support-query-farm/https://rusty.today/blog/linkedin-2025-11-09-help-support-query-farm/I’m trying to get our DuckDB extensions fully indexed in Google so users can more easily discover what we’re building — especially all the tools that make...Sun, 09 Nov 2025 00:00:00 GMThttps://rusty.today/blog/linkedin-2025-11-09-introducing-the-jsonata-extension-for-duckdb-by-query-farm/https://rusty.today/blog/linkedin-2025-11-09-introducing-the-jsonata-extension-for-duckdb-by-query-farm/Transform and query JSON data with elegant expressions—all within SQL. No more fighting with nested JSON or writing complex extraction logic. What does it...Sun, 09 Nov 2025 00:00:00 GMThttps://rusty.today/blog/linkedin-2025-10-30-kafka-reading-writing-in-duckdb-is-possible/https://rusty.today/blog/linkedin-2025-10-30-kafka-reading-writing-in-duckdb-is-possible/I’ve been working on integrating DuckDB with Apache Kafka, and now you can read from and write to Kafka topics using the SQL you already know. In the demo...Thu, 30 Oct 2025 00:00:00 GMThttps://rusty.today/blog/linkedin-2025-10-28-new-duckdb-extension-launch/https://rusty.today/blog/linkedin-2025-10-28-new-duckdb-extension-launch/I am happy to introduce the MiniJinja Extension for DuckDB, brought to you by Query.Farm! Now you can bring the power of the industry-standard MiniJinja...Tue, 28 Oct 2025 00:00:00 GMThttps://rusty.today/blog/linkedin-2025-10-28-the-duckdb-developer-meeting-2026-has-been-announced/https://rusty.today/blog/linkedin-2025-10-28-the-duckdb-developer-meeting-2026-has-been-announced/#DuckDB Developer Meeting 2026, hosted by DuckDB Labs will be on January 30, 2026, in #Amsterdam. This specialized event is designed for developers working...Tue, 28 Oct 2025 00:00:00 GMThttps://rusty.today/blog/linkedin-2025-10-25-excited-to-announce-the-json-schema-extension-for-duckdb/https://rusty.today/blog/linkedin-2025-10-25-excited-to-announce-the-json-schema-extension-for-duckdb/We've just released a powerful new community extension that brings JSON schema validation and data enrichment directly to your SQL queries. No more context...Sat, 25 Oct 2025 00:00:00 GMThttps://rusty.today/blog/linkedin-2025-10-22-announcing-the-tera-duckdb-extension/https://rusty.today/blog/linkedin-2025-10-22-announcing-the-tera-duckdb-extension/Now you can bring powerful template rendering directly into your SQL queries. Generate dynamic text, HTML, reports, and configuration files without leaving...Wed, 22 Oct 2025 00:00:00 GMThttps://rusty.today/blog/linkedin-2025-10-09-duckdb-lab-s-roadmap-for-the-next-year-2025-2026-has-been-up/https://rusty.today/blog/linkedin-2025-10-09-duckdb-lab-s-roadmap-for-the-next-year-2025-2026-has-been-up/Here are some of the highlights with my commentary - 🧩 Extensions ◆ Updated C API + C extension API - this means extensions will have a stable API build...Thu, 09 Oct 2025 00:00:00 GMThttps://rusty.today/blog/linkedin-2025-09-26-heading-to-paris-today-for-pydata-paris-and-to-speak-at-the-/https://rusty.today/blog/linkedin-2025-09-26-heading-to-paris-today-for-pydata-paris-and-to-speak-at-the-/I’ll be presenting: “Building on Flight: Real-World Lessons from the DuckDB Airport Extension” I'm excited to reconnect with the Arrow community. If you’re...Fri, 26 Sep 2025 00:00:00 GMThttps://rusty.today/blog/linkedin-2025-09-20-fun-friday-afternoon-hack/https://rusty.today/blog/linkedin-2025-09-20-fun-friday-afternoon-hack/👉 Just set your dialect with the dialectic_flavor PRAGMA. This means you can attempt to run SQL built for other systems without rewriting it for DuckDB —...Sat, 20 Sep 2025 00:00:00 GMThttps://rusty.today/blog/linkedin-2025-09-11-it-s-always-great-to-see-the-work-we-re-doing-at-query-farm-/https://rusty.today/blog/linkedin-2025-09-11-it-s-always-great-to-see-the-work-we-re-doing-at-query-farm-/The team at yeet.cx has built an observability platform that makes clever use of DuckDB Wasm together with our Textplot DuckDB extension. The result?...Thu, 11 Sep 2025 00:00:00 GMThttps://rusty.today/blog/linkedin-2025-09-09-i-had-the-chance-to-sit-down-with-michael-watson-on-the-hedg/https://rusty.today/blog/linkedin-2025-09-09-i-had-the-chance-to-sit-down-with-michael-watson-on-the-hedg/We talked about why DuckDB’s in-process, C++-based architecture is changing how people think about data systems, and we dug into the ecosystem that’s...Tue, 09 Sep 2025 00:00:00 GMThttps://rusty.today/blog/linkedin-2025-09-05-any-time-i-wind-up-using-scikit-optimize-differential-evolut/https://rusty.today/blog/linkedin-2025-09-05-any-time-i-wind-up-using-scikit-optimize-differential-evolut/But when I'm dealing with optimization, I'm always left with that feeling, did I find a local or global minima? The only way to know for sure is to gather...Fri, 05 Sep 2025 00:00:00 GMThttps://rusty.today/blog/linkedin-2025-09-03-query-farm-announces-the-integration-of-quackscience-s-duckd/https://rusty.today/blog/linkedin-2025-09-03-query-farm-announces-the-integration-of-quackscience-s-duckd/I am excited to announce Query.Farm is expanding! We've integrated 12 high-quality DuckDB community extensions from the QuackScience project, strengthening...Wed, 03 Sep 2025 00:00:00 GMThttps://rusty.today/blog/linkedin-2025-08-30-lets-look-into-using-duckdb-with-arrow-and-python-generators/https://rusty.today/blog/linkedin-2025-08-30-lets-look-into-using-duckdb-with-arrow-and-python-generators/If you’ve used DuckDB with Pandas, you know how easy it is to query DataFrames directly. But what if your data is too big to fit in memory, or you want to...Sat, 30 Aug 2025 00:00:00 GMThttps://rusty.today/blog/streaming-data-duckdb-arrow-python-generators/https://rusty.today/blog/streaming-data-duckdb-arrow-python-generators/How to use Python generators and Apache Arrow's RecordBatchReader to stream large datasets into DuckDB without loading everything into memory at once.Sat, 30 Aug 2025 00:00:00 GMThttps://rusty.today/blog/linkedin-2025-08-22-announcing-textplot-bringing-beautiful-ascii-unicode-data-vi/https://rusty.today/blog/linkedin-2025-08-22-announcing-textplot-bringing-beautiful-ascii-unicode-data-vi/Create stunning bar charts, density plots, and sparklines without leaving DuckDB. Perfect for quick data exploration, monitoring dashboards, CLI analytics,...Fri, 22 Aug 2025 00:00:00 GMThttps://rusty.today/blog/linkedin-2025-08-16-new-stochastic-extension-for-duckdb/https://rusty.today/blog/linkedin-2025-08-16-new-stochastic-extension-for-duckdb/Excited to share the stochastic extension - bringing comprehensive statistical distributions directly to DuckDB SQL! ✨ What's included: 15+ probability...Sat, 16 Aug 2025 00:00:00 GMThttps://rusty.today/blog/linkedin-2025-08-14-fuzzy-string-matching-in-duckdb-just-got-easier/https://rusty.today/blog/linkedin-2025-08-14-fuzzy-string-matching-in-duckdb-just-got-easier/The RapidFuzz extension is now available as a DuckDB Community Extension! 🦆✨ You can now perform high-performance fuzzy string matching directly in your...Thu, 14 Aug 2025 00:00:00 GMThttps://rusty.today/blog/linkedin-2025-08-11-new-duckdb-extension-bitfilters-probabilistic-data-structure/https://rusty.today/blog/linkedin-2025-08-11-new-duckdb-extension-bitfilters-probabilistic-data-structure/Just launched a DuckDB Community Extension that brings high-performance probabilistic approximate membership filters to SQL! What it does: ✅ Fast set...Mon, 11 Aug 2025 00:00:00 GMThttps://rusty.today/blog/linkedin-2025-08-07-excited-to-announce-the-release-of-the-hashfuncs-extension-f/https://rusty.today/blog/linkedin-2025-08-07-excited-to-announce-the-release-of-the-hashfuncs-extension-f/Whether you’re working on database indexing, data partitioning, caching, or implementing Bloom filters, hashfuncs offers fast and reliable hashing optimized...Thu, 07 Aug 2025 00:00:00 GMThttps://rusty.today/blog/linkedin-2025-08-07-new-duckdb-marisa-extension-by-query-farm/https://rusty.today/blog/linkedin-2025-08-07-new-duckdb-marisa-extension-by-query-farm/🚀 New DuckDB Marisa Extension by Query.Farm Add fast, space-efficient 🧠 MARISA trie support to DuckDB for string lookups, autocomplete 🔍, prefix...Thu, 07 Aug 2025 00:00:00 GMThttps://rusty.today/blog/linkedin-2025-07-25-just-published-a-new-benchmark-report-for-airport-our-duckdb/https://rusty.today/blog/linkedin-2025-07-25-just-published-a-new-benchmark-report-for-airport-our-duckdb/🔍 We tested scalar functions, remote table creation, and selection queries using both BIGINT and 32-byte strings — scaling up to 100 million rows. The...Fri, 25 Jul 2025 00:00:00 GMThttps://rusty.today/blog/linkedin-2025-07-24-thank-you-for-the-incredible-interest-in-airport/https://rusty.today/blog/linkedin-2025-07-24-thank-you-for-the-incredible-interest-in-airport/I’ve been blown away by the enthusiasm and curiosity around Airport (https://airport.query.farm), our DuckDB extension that brings the power of Arrow Flight...Thu, 24 Jul 2025 00:00:00 GMThttps://rusty.today/blog/linkedin-2025-07-22-the-airport-extension-for-duckdb-is-now-live/https://rusty.today/blog/linkedin-2025-07-22-the-airport-extension-for-duckdb-is-now-live/🛫 Airport lets DuckDB use Arrow Flight, unlocking new ways to query remote data seamlessly and quickly. ✨ Bonus: the demo includes built-in UTF-8 tests,...Tue, 22 Jul 2025 00:00:00 GMThttps://rusty.today/blog/linkedin-2025-06-13-kafka-duckdb-now-seamlessly-connected/https://rusty.today/blog/linkedin-2025-06-13-kafka-duckdb-now-seamlessly-connected/The Tributary extension by Query.Farm — enables real-time querying and analysis of Kafka streams directly in DuckDB. With Tributary, you can consume...Fri, 13 Jun 2025 00:00:00 GMThttps://rusty.today/blog/linkedin-2025-06-09-are-you-getting-extension-not-found-errors-in-duckdb/https://rusty.today/blog/linkedin-2025-06-09-are-you-getting-extension-not-found-errors-in-duckdb/Extension availability across versions and platforms isn’t always obvious—and the docs don’t always have the answers. So I built a DuckDB Extension...Mon, 09 Jun 2025 00:00:00 GMThttps://rusty.today/blog/linkedin-2025-06-03-new-duckdb-extension-radio/https://rusty.today/blog/linkedin-2025-06-03-new-duckdb-extension-radio/I'm excited to launch Radio, a new DuckDB extension that enables real-time interaction with event-driven systems like WebSocket and Redis Pub/Sub. With...Tue, 03 Jun 2025 00:00:00 GMThttps://rusty.today/blog/linkedin-2025-05-27-thoughts-on-duckdb-s-ducklake-so-far/https://rusty.today/blog/linkedin-2025-05-27-thoughts-on-duckdb-s-ducklake-so-far/1. As someone's who is familiar with the backend implementations of Delta Lake and Iceberg, the simplicity of DuckLake is admirable. 2. Iceberg table...Tue, 27 May 2025 00:00:00 GMThttps://rusty.today/blog/linkedin-2025-02-06-here-s-a-short-clip-from-my-presentation-at-duckcon-6-in-ams/https://rusty.today/blog/linkedin-2025-02-06-here-s-a-short-clip-from-my-presentation-at-duckcon-6-in-ams/I shared how 🦆DuckDB can seamlessly work with Apache Arrow Flight to provide universal data access. Arrow Flight enables DuckDB to access remote data...Thu, 06 Feb 2025 00:00:00 GMThttps://rusty.today/blog/linkedin-2025-01-31-today-is-duckdb-s-duckcon-6-in-amsterdam/https://rusty.today/blog/linkedin-2025-01-31-today-is-duckdb-s-duckcon-6-in-amsterdam/I'm going to demo of write support for #DeltaLake tables, building a machine learning prediction model with #AutoGluon in only #SQL, and wrap up with some...Fri, 31 Jan 2025 00:00:00 GMThttps://rusty.today/blog/linkedin-2025-01-16-duckcon-6-in-amsterdam-is-just-a-few-weeks-away/https://rusty.today/blog/linkedin-2025-01-16-duckcon-6-in-amsterdam-is-just-a-few-weeks-away/DuckCon 6 in Amsterdam is just a few weeks away! Here’s a sneak peek of a small part of my presentation about the new Airport extension, which integrates...Thu, 16 Jan 2025 00:00:00 GMThttps://rusty.today/blog/linkedin-2025-01-02-i-m-excited-to-announce-the-datasketches-extension-for-duckd/https://rusty.today/blog/linkedin-2025-01-02-i-m-excited-to-announce-the-datasketches-extension-for-duckd/I'm excited to announce the datasketches extension for DuckDB, which integrates the capabilities of Apache DataSketches with DuckDB's aggregate and scalar...Thu, 02 Jan 2025 00:00:00 GMThttps://rusty.today/blog/cron-expressions-duckdb/https://rusty.today/blog/cron-expressions-duckdb/Introducing a DuckDB extension that interprets cron expressions to generate scheduled timestamps directly in SQL, built with the Rust croner crate.Tue, 28 May 2024 00:00:00 GMThttps://rusty.today/blog/introducing-shellfs-duckdb-extension/https://rusty.today/blog/introducing-shellfs-duckdb-extension/Introducing shellfs, a DuckDB extension that enables seamless integration with Unix pipes for both input and output, allowing command-line programs to be used directly within DuckDB queries.Mon, 20 May 2024 00:00:00 GMThttps://rusty.today/blog/turn-tape-recorder-keep-running/https://rusty.today/blog/turn-tape-recorder-keep-running/How to build a kitchen-sink observability event stream using AWS EventBridge, Firehose, and Apache Iceberg to capture and query every event in your distributed system.Wed, 15 May 2024 00:00:00 GMThttps://rusty.today/blog/bare-minimum-metadata-any-data/https://rusty.today/blog/bare-minimum-metadata-any-data/A practical guide to the essential metadata columns every data platform should store alongside its data, enabling traceability back to source files and loader versions using Apache Parquet's efficient encoding.Sat, 27 Apr 2024 00:00:00 GMThttps://rusty.today/blog/duckdb-strategically-important/https://rusty.today/blog/duckdb-strategically-important/Why DuckDB is becoming the go-to tool for data tasks in 2024, along with key areas where improvements are still needed including Iceberg support, Parquet encodings, and remote data throughput.Thu, 25 Apr 2024 00:00:00 GMThttps://rusty.today/blog/images-emails-content-addressable-storage/https://rusty.today/blog/images-emails-content-addressable-storage/An investigation about how to store a large number of referenced URLs while considering WebP and AVIF recompression for images.Fri, 22 Jan 2021 00:00:00 GMThttps://rusty.today/blog/cost-effective-s3-for-subscription-data-distribution/https://rusty.today/blog/cost-effective-s3-for-subscription-data-distribution/Solving the problem where you have a lot of data you want to allow customers to obtain (provided they purchased access), but you don't want to pay for their internet transfer fees can be tricky. This architecture makes it easy.Wed, 13 Jan 2021 00:00:00 GMThttps://rusty.today/blog/anomaly-detection-for-cities-and-airports/https://rusty.today/blog/anomaly-detection-for-cities-and-airports/Cities and airports stream their radio traffic to the internet, you can listen to police or firetrucks being dispatched or airplanes being given headings and altitudes. By analyzing the duration of transmissions over time with machine learning, we make be able to achieve automated detection of emergencies or abnormal events.Tue, 12 Jan 2021 00:00:00 GMThttps://rusty.today/blog/web-based-chat-user-interfaces/https://rusty.today/blog/web-based-chat-user-interfaces/Looking into various web chat front end implementationsTue, 12 Jan 2021 00:00:00 GMThttps://rusty.today/blog/thoughts-on-purgeable-caching/https://rusty.today/blog/thoughts-on-purgeable-caching/A summary of thoughts about purgeable memory allocations and distributed caching.Sat, 02 Jan 2021 00:00:00 GMThttps://rusty.today/blog/predicting-ny-electricity-with-julia-and-tensorflowjs/https://rusty.today/blog/predicting-ny-electricity-with-julia-and-tensorflowjs/A short adventure in prediction for electrical generation and demand in New York State.Fri, 01 Jan 2021 00:00:00 GMThttps://rusty.today/blog/multi-dimensional-indexing/https://rusty.today/blog/multi-dimensional-indexing/My personal research notes into multi-dimensional indexing strategies for data, involving Z-order curves and Hilbert curves.Mon, 14 Sep 2020 00:00:00 GMThttps://rusty.today/blog/bridger-foothills-fire/https://rusty.today/blog/bridger-foothills-fire/A fire started on the Bridger Range last Friday. These mountains are iconic if you've ever lived in Bozeman. They aren't going to look the same for a long time.Mon, 07 Sep 2020 00:00:00 GMThttps://rusty.today/blog/preferred-pictures/https://rusty.today/blog/preferred-pictures/Preferred Pictures is a software-as-a-service that algorithmically determines which image from many that will result in the most sales and conversions in a way that is arguably that is better than A/B tests.Sun, 23 Aug 2020 00:00:00 GMThttps://rusty.today/blog/tulum-mexico-travel-guide/https://rusty.today/blog/tulum-mexico-travel-guide/My short guide to adventures and happiness in Tulum, MexicoMon, 15 Apr 2019 00:00:00 GMThttps://rusty.today/blog/finding-available-short-domain-names/https://rusty.today/blog/finding-available-short-domain-names/Find short domain names that are able to be registeredFri, 07 Aug 2015 00:00:00 GMThttps://rusty.today/blog/largest-integer-consisting-of-unique-triple-primes/https://rusty.today/blog/largest-integer-consisting-of-unique-triple-primes/Goal: Find the largest integer where all triples are unique and prime.Sun, 07 Jun 2015 00:00:00 GMThttps://rusty.today/blog/ffmpeg-scene-change-detector/https://rusty.today/blog/ffmpeg-scene-change-detector/ffmpeg is a very versatile piece of software for working with video. At its core there are video filters, and one of these filters is called select. The select filter allows a user to only write certain frames that pass thresholds to the output, in other words you can pull out interesting pieces of video using a select filter. A common use of the select filter is to produce thumbnails for each scene in a video.Thu, 27 Feb 2014 00:00:00 GMT