Reading list

DuckDB performance and Parquet storage

Performance decisions depend on how data is laid out and how queries reach it. These articles connect a measured pagination problem to column encodings and storage design, with attention to correctness as well as speed.

1. 01

   Paging Through a Parquet File in DuckDB: file_row_number or OFFSET?

   Compare file_row_number with LIMIT/OFFSET, including the row-ordering traps behind a faster query.

2. 02

   GeoSilo vs GeoArrow + Byte Stream Split: Two Approaches to Geometry Compression

   Compare geometry layouts, compression ratios, and precision tradeoffs using Parquet BYTE_STREAM_SPLIT.

3. 03

   The Bare Minimum of Metadata For Any Data

   Keep data traceable to source files and loader versions without overlooking Parquet encoding efficiency.

4. 04

   DuckDB is Strategically Important

   Read the 2024 perspective on DuckDB's role, including remote throughput and Parquet encoding gaps at the time.