Repository navigation
feat: add DuckDB backend - #307
Draft
vincentsarago wants to merge 24 commits into
Draft
vincentsarago wants to merge 24 commits into
vincentsarago wants to merge 24 commits into
Conversation
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Mirrors PgCollection: same query structure, validation and output. Queries run in a threadpool on a per-call cursor of the collection's DuckDB connection; CQL geometry literals are tagged/reprojected with DuckDB's ST_SetCRS / ST_Transform; MVT properties DuckDB can't encode are cast. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
duck_get_collection_index lists the tables and views of all attached databases (incl. file sources), normalizes DuckDB types to the names tipg.collections.Column understands, reads the SRID from the geometry column type and computes spatial/temporal extents in one scan per table. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Cover catalog/extents, items filters (ids, bbox, datetime, properties, cql2 incl. projected CRS), sortby/properties, output formats, item, vector tiles and an app mixing PostgreSQL and DuckDB collections. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
DuckDBSettings.database now defaults to None: the database file (path or URL) is only attached when set, otherwise only file sources are served. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Remove the read_only setting: tipg only reads data. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ion_index keyword-only connect_to_db / close_db_connection → connect_to_duckdb / close_duckdb_connection, duck_get_collection_index(*, connection, settings=None); update docs and tests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
vincentsarago
force-pushed
the
feat/duckdb-backend
branch
from
November 10, 2026 14:22
291bdc9 to
88b3f22
Compare
vincentsarago
changed the base branch from
main
to
refactor/pg-catalog-settings
November 10, 2026 14:23
Member
Author
This was referenced Nov 12, 2026
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Note
Claude Code made this PR under the supervision of vincentsarago.
Adds a
tipg.duckbackend that serves DuckDB tables, views and files (GeoParquet, or anyST_Readformat) as OGC Features and Tiles collections. They can be served alone or mixed with PostgreSQL collections in the same catalog. This builds on #304, where each collection holds its own connection.Stacked on #313, which holds the non-DuckDB changes split out of this PR:
PostgresCatalogSettings, the newpg_get_collection_indexsignature, and the geometry literal hooks. Merge #313 first; this PR's base will then move tomain.How it works
tipg/duck/database.py,connect_to_duckdb/close_duckdb_connection, stored onapp.state.duckdb_connection):TIPG_DUCKDB_DATABASEfile (path or URL) is attached read-only under its file name. Without it, only filesourcesare served.sourcesare registered as views inmemory.main.connection.cursor(), which opens a new connection. So the views are regular (notTEMP) and the settings useSET GLOBAL(geometry_always_xy = true,TimeZone = 'UTC').DuckCollection(tipg/duck/collections.py):PgCollection: same validation and query structure, sameItemList/ MVT output.PgCollection.connection_pool.duck_get_collection_index(*, connection, settings=None)):duckdb_tables()/duckdb_views()/duckdb_columns(), with primary keys fromduckdb_constraints().Columnalready understands.GEOMETRY('EPSG:26915')).TableSettings/TableConfigfromtipg.settings.DuckCollectionimplements the abstract hooks added in refactor: PostgresCatalogSettings, simpler pg_get_collection_index and backend-specific geometry literal hooks #313 (_tag_geometry_literal/_reproject_geometry_literal) withST_SetCRS/ST_Transform(g, 'EPSG:4326', 'EPSG:n'). This matters because DuckDB doesn't raise an error on mixed-CRS comparisons, it silently matches nothing.ST_AsMVTGeom/ST_AsMVT, with anORDER BYinside the aggregate. Property typesST_AsMVTcan't encode are cast:DECIMALtoDOUBLE, and dates, timestamps, lists, etc. toVARCHAR.Packaging
The new
tipg[duckdb]extra installsduckdb>=1.4andpytz. DuckDB needspytzto returnTIMESTAMP WITH TIME ZONEvalues to Python. Tested with DuckDB 2.0.0.Not included
tipg/main.pyis unchanged. The new docs page shows a DuckDB-only app and a mixed PG + DuckDB app.ST_EstimatedExtent); it can be disabled withTIPG_DUCKDB_SPATIAL_EXTENT=FALSE.Also in this PR
plans/duckdb-backend.mdis the implementation plan. A final "Implementation notes" section lists where the code differs from it.Testing
tests/duck/: 17 tests against a generated.duckdbfile plus a GeoParquet source:ids,bbox,datetime, properties, cql2 (IN,LIKE,S_INTERSECTS, including on the projected table);sortby/properties, CSV / NDJSON / GeoJSON-seq / HTML output, single item (including a 404 for a non-integer id);file://URL.pre-commit run --all-filesandmypy -p tipg --ignore-missing-imports(CI command) pass.docs/src/advanced/duckdb.mdwere run against generated data and the test PostGIS database (not committed).🤖 Generated with Claude Code