Utilities
Run in-process SQL on CSV, JSON, or Parquet
In-process SQL on CSV, JSON, and Parquet. No server.
overview
What duckdb does
You have a query, and maybe a CSV, JSON, or Parquet file, and the next node needs the result as a file. Set -c to the SQL. The wrapper runs duckdb and redirects stdout to the last argv (the Hive output path). A database file on the positional database port is optional. SELECT 1 AS n works with no file.
-read and -write pick csv, json, or parquet when you want those helpers. -o is a file output inside duckdb; prefer the Hive path the wrapper already tees. -readonly is a boolean; leave it disconnected unless you want that flag.
duckdb is in-process. It does not start a server and it does not reach a remote warehouse. Use csvkit when the job is one CSV and csvsql is enough. A query that references a missing file fails the node.
source github.com/duckdb/duckdb
use cases
Where duckdb fits
Run a constant query
Set -c to SELECT 1 AS n. The output file holds the result. No database file required.
Query a CSV
Connect the file to database or read it in SQL. Set -read csv when you use that helper.
Write JSON or Parquet
Set -write json or parquet when you want that format from the helper flags.
Open a file read-only
Connect -readonly only when you mean to send that flag.
reference
duckdb inputs and flags
| Name | Type | Flag | Description |
|---|---|---|---|
| query | STRING | -c | SQL query to execute. |
| database | FILE | · | Database file. Optional for a constant query. |
| read_format | STRING | -read | Input format: csv, json, or parquet. |
| output_format | STRING | -write | Output format: csv, json, or parquet. |
Showing key inputs. duckdb exposes 6 inputs in total.
Full flag reference (6 inputs)
| Name | Type | Flag | Description |
|---|---|---|---|
| database | FILE | · | Database file or memory database |
| query | STRING | -c | SQL query to execute |
| read_format | STRING | -read | Input format (csv, json, parquet) |
| output_format | STRING | -write | Output format (csv, json, parquet) |
| output | FILE | -o | Output file path |
| readonly | BOOLEAN | -readonly | Open database in read-only mode |
example
Run duckdb
duckdb -c "SELECT 1 AS n"┌───────┐│ n ││ int32 │├───────┤│ 1 │└───────┘guidance
Choosing duckdb
Use duckdb when you want SQL on CSV, JSON, or Parquet without a server. Use csvkit when the job is one CSV and csvsql is enough.
csvkit
csvsql on one CSV. Narrower than duckdb, enough for a single table filter.
jq
Filters JSON documents. It is not SQL.
wget
Downloads a data file. It does not query it.
faq
duckdb questions
related
More Utilities tools
browser-fetch
Headless Chromium render: requested URL, final URL, status, and title next to the page.
cewl
Spider a URL and return a wordlist for password crackers.
csvkit
SQL on one CSV. The table name is the file stem.
dnsgen
Wordlist and mined-word subdomain permutation.
edge-tts
Microsoft Edge neural speech from text or a script file. No API key.
exiftool
Read embedded metadata from hundreds of image, media, and document formats.
Run duckdb yourself
SELECT 1 AS n feeds duckdb. The table below is from a completed run.
Facts on this page come from the live Trickest tool library.