From 45e2123a4b2588683eb008c1dc915fa6159fab02 Mon Sep 17 00:00:00 2001 From: cevheri Date: Thu, 27 Aug 2026 02:19:07 +0300 Subject: [PATCH 1/5] feat(providers): libSQL over the Hrana protocol, one type-id for sqld and Turso Cloud (#424) Phase 5 of #424. WORK IN PROGRESS - gates 1-6 pass locally; the browser pass (gate 5) and agent grounding (gate 7) are not yet run, and no PR is open. A self-hosted libSQL server and Turso Cloud are ONE type-id because they speak the same protocol and embed the same SQLite (3.47.0, measured on both). Zero runtime dependency: Hrana is JSON over POST /v2/pipeline through the runtime's own fetch. Measured findings that shaped the code rather than being read from documentation: - A failed statement answers HTTP 200 with the failure inside results[], and a batch's failing step does not abort the pipeline - so executeBatch hands back a per-statement outcome and one refused read costs its own panel. - The two deployments word the same refusal differently ("unsupported statement: VACUUM" against "SQL not allowed statement: VACUUM") under one code, so nothing keys on wording. - VACUUM, ANALYZE, PRAGMA optimize and PRAGMA wal_checkpoint are all refused by the server, so only reindex and check are offered. - PRAGMA query_only is refused too, which is why this provider implements no queryReadOnly: the agent read-only profile stays PostgreSQL + SQLite. - `notnull` is a SQLite keyword, so projecting it bare from pragma_table_info is a parse error that costs the COLUMNS of every table and leaves the tree intact. Only the live probe could find it; the statement text is now pinned by a test. - dbstat answers on BOTH deployments, so table and index bytes here are measured - which bun:sqlite cannot do at all. - GET /version is a sqld route Turso Cloud does not have, so the version panel reads "sqld 0.24.33 (SQLite 3.47.0)" there and "SQLite 3.47.0" on the cloud. Also filed D34: the migration generator emits ADD CONSTRAINT for the `sqlite` dialect, which SQLite cannot parse. Found through the sibling engine, and left to its own fix. Chart 0.1.52 is released, so the packaged README change bumps it to 0.1.53 (#167). --- CLAUDE.md | 6 +- DOCKERHUB.md | 5 +- README.md | 15 +- README_ja.md | 3 +- README_zh.md | 3 +- charts/libredb-studio/Chart.yaml | 6 +- charts/libredb-studio/README.md | 4 +- database-compose.yml | 45 + .../digitalocean/assets/description-long.md | 2 +- deploy/railway/TEMPLATE_OVERVIEW.md | 2 +- deploy/rancher/CATALOG_LISTING.md | 6 +- docs/ADDING_A_PROVIDER.md | 12 +- docs/AGENT.md | 8 +- docs/AGENT_DATA_FLOW.md | 19 +- docs/AGENT_GUIDE.md | 4 +- docs/API_DOCS.md | 2 +- docs/ARCHITECTURE.md | 2 +- docs/BACKLOG.md | 25 + docs/BRAND_MESSAGING.md | 16 +- docs/DATABASE_PROVIDERS.md | 2 +- docs/FEATURES.md | 1 + docs/SEED_CONNECTIONS.md | 4 +- docs/providers/README.md | 3 +- docs/providers/libsql.md | 420 ++++++++++ e2e/libsql-provider.spec.ts | 74 ++ .../helm-charts/libredb-studio/Chart.yaml | 6 +- operator/helm-charts/libredb-studio/README.md | 4 +- probe-libsql.ts | 58 ++ probe-results-cloud.json | 294 +++++++ probe-results-self.json | 294 +++++++ src/components/ConnectionModal.tsx | 18 +- src/components/icons/db-icons.tsx | 29 + src/hooks/use-connection-form.ts | 1 + src/lib/connection-string-parser.ts | 37 + src/lib/db-showcase.ts | 6 +- src/lib/db-ui-config.ts | 25 + src/lib/db/compatibility.ts | 5 + src/lib/db/factory.ts | 7 +- .../providers/sql/libsql/hrana-transport.ts | 436 ++++++++++ src/lib/db/providers/sql/libsql/index.ts | 465 +++++++++++ src/lib/db/providers/sql/libsql/introspect.ts | 542 ++++++++++++ src/lib/db/providers/sql/libsql/transport.ts | 162 ++++ src/lib/export/result-export.ts | 48 ++ src/lib/schema-diff/migration-generator.ts | 36 +- src/lib/seed/types.ts | 1 + src/lib/sql/fence-tags.ts | 6 + src/lib/sql/grammar.ts | 8 + src/lib/sql/values.ts | 3 + src/lib/types.ts | 18 +- tests/components/ConnectionModal.test.tsx | 26 + tests/hooks/use-connection-form.test.ts | 1 + tests/integration/db/libsql-provider.test.ts | 648 ++++++++++++++ tests/unit/db/libsql/hrana-transport.test.ts | 787 ++++++++++++++++++ tests/unit/db/libsql/introspect.test.ts | 433 ++++++++++ tests/unit/db/libsql/seam-guard.test.ts | 263 ++++++ .../unit/lib/connection-string-parser.test.ts | 46 + tests/unit/lib/db-ui-config.test.ts | 7 +- tests/unit/lib/sql/fence-tags.test.ts | 3 + .../schema-diff/migration-generator.test.ts | 26 + tests/unit/sql/grammar.test.ts | 11 +- 60 files changed, 5382 insertions(+), 67 deletions(-) create mode 100644 docs/providers/libsql.md create mode 100644 e2e/libsql-provider.spec.ts create mode 100644 probe-libsql.ts create mode 100644 probe-results-cloud.json create mode 100644 probe-results-self.json create mode 100644 src/lib/db/providers/sql/libsql/hrana-transport.ts create mode 100644 src/lib/db/providers/sql/libsql/index.ts create mode 100644 src/lib/db/providers/sql/libsql/introspect.ts create mode 100644 src/lib/db/providers/sql/libsql/transport.ts create mode 100644 tests/integration/db/libsql-provider.test.ts create mode 100644 tests/unit/db/libsql/hrana-transport.test.ts create mode 100644 tests/unit/db/libsql/introspect.test.ts create mode 100644 tests/unit/db/libsql/seam-guard.test.ts diff --git a/CLAUDE.md b/CLAUDE.md index 373f0f57..6839ee6a 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -6,7 +6,7 @@ Guidance for Claude Code in this repo — conventions, rules, and gotchas only. ## Project Overview -Web-based SQL IDE for cloud-native teams: PostgreSQL, MySQL, SQLite, Oracle, SQL Server, MongoDB, Redis, Couchbase, ClickHouse, Apache Druid, Elasticsearch, OpenSearch, Apache Trino, Apache Cassandra (plus the embedded LibreDB) + AI query assistance. Runs **two ways** — a standalone Next.js app AND a published npm package (CLI plus an embeddable library surface); `build:lib` (tsup) produces the package dist. The two modes render different chrome, so a UI change verified in one is not verified in the other. +Web-based SQL IDE for cloud-native teams: PostgreSQL, MySQL, SQLite, libSQL, Oracle, SQL Server, MongoDB, Redis, Couchbase, ClickHouse, Apache Druid, Elasticsearch, OpenSearch, Apache Trino, Apache Cassandra (plus the embedded LibreDB) + AI query assistance. Runs **two ways** — a standalone Next.js app AND a published npm package (CLI plus an embeddable library surface); `build:lib` (tsup) produces the package dist. The two modes render different chrome, so a UI change verified in one is not verified in the other. ## Branching & PRs @@ -69,8 +69,8 @@ After every code change, run all six locally before claiming done — they match ### Rules & patterns -> **⚠️ Providers are the lifeblood of this project — keep the triad in lockstep: code ↔ docs ↔ tests**, 1:1 per canonical type-id — the type-id set is the `DatabaseType` union in [`src/lib/types.ts`](src/lib/types.ts) (`postgres`, `mysql`, `sqlite`, `mongodb`, `redis`, `oracle`, `mssql`, `couchbase`, `clickhouse`, `druid`, `elasticsearch`, `opensearch`, `cassandra`, `trino`, plus the embedded `libredb`): -> - Code: `src/lib/db/providers//.ts`, or `src/lib/db/providers///index.ts` when the provider is split across modules, as `couchbase`, `clickhouse`, `druid`, `trino` and `cassandra` are · Docs: `docs/providers/.md` · Tests: `tests/integration/db/-provider.test.ts` +> **⚠️ Providers are the lifeblood of this project — keep the triad in lockstep: code ↔ docs ↔ tests**, 1:1 per canonical type-id — the type-id set is the `DatabaseType` union in [`src/lib/types.ts`](src/lib/types.ts) (`postgres`, `mysql`, `sqlite`, `libsql`, `mongodb`, `redis`, `oracle`, `mssql`, `couchbase`, `clickhouse`, `druid`, `elasticsearch`, `opensearch`, `cassandra`, `trino`, plus the embedded `libredb`): +> - Code: `src/lib/db/providers//.ts`, or `src/lib/db/providers///index.ts` when the provider is split across modules, as `couchbase`, `clickhouse`, `druid`, `trino`, `cassandra` and `libsql` are · Docs: `docs/providers/.md` · Tests: `tests/integration/db/-provider.test.ts` > - **One directory may serve two type-ids** — `src/lib/db/providers/sql/search/` is both `elasticsearch` and `opensearch` (#424). Docs and tests stay 1:1 anyway: the invariant is per type-id, and each doc is the prime reference for its own product's measured behaviour. > - Any change to one side MUST sync the others **in the same PR**. The doc mirrors the code and the code mirrors the doc — never let them drift. diff --git a/DOCKERHUB.md b/DOCKERHUB.md index ee4aa2be..7644ed85 100644 --- a/DOCKERHUB.md +++ b/DOCKERHUB.md @@ -107,7 +107,7 @@ The network route is the one to prefer for a real deployment: put Studio and its ## Supported databases -Fourteen external engines share one interface, and three of them are read-only because their own SQL is. The table below has fifteen rows: the fifteenth is the embedded LibreDB store, which ships inside the image rather than being a server you connect out to. +Fifteen external engines share one interface, and three of them are read-only because their own SQL is. The table below has sixteen rows: the sixteenth is the embedded LibreDB store, which ships inside the image rather than being a server you connect out to. | Database | Driver | Highlights | | :--- | :--- | :--- | @@ -116,6 +116,7 @@ Fourteen external engines share one interface, and three of them are read-only b | **Oracle** | `oracledb` (thin) | `FETCH FIRST` pagination, `V$` monitoring, `ANALYZE`, transactions | | **SQL Server** | `mssql` | `OFFSET FETCH`, `sys.dm_*` DMVs, `DBCC CHECKDB`, Azure SQL auto-detect | | **SQLite** | `bun:sqlite` / `node:sqlite` | File-based or in-memory databases; the driver follows the runtime, with a `LIBREDB_SQLITE_DRIVER` override | +| **libSQL** | none — HTTP | Full SQL IDE over the Hrana protocol against a libSQL server or Turso Cloud; SQLite's dialect across a network, with real per-table bytes from `dbstat` and an auth token instead of a password | | **MongoDB** | `mongodb` | JSON query editor, find/aggregate/insert/update/delete | | **Redis** | `ioredis` | Command editor, non-blocking `SCAN` key browser, `INFO` monitoring, per-type command generation | | **Couchbase** | none — HTTP | SQL++ query editor, bucket/scope/collection browser, cluster health | @@ -131,7 +132,7 @@ Fourteen external engines share one interface, and three of them are read-only b ### Engines with no provider of their own -Twenty-six further engines speak the wire protocol of one of the fourteen drivers above, so they connect through it unchanged: pick that driver in the connection dialog. The table has twenty-two rows rather than twenty-six because engines that behave identically share a row; all twenty-six are named in it. Every one of them was measured against a real instance rather than assumed, and how much of the product worked is recorded per engine. +Twenty-six further engines speak the wire protocol of one of the fifteen drivers above, so they connect through it unchanged: pick that driver in the connection dialog. The table has twenty-two rows rather than twenty-six because engines that behave identically share a row; all twenty-six are named in it. Every one of them was measured against a real instance rather than assumed, and how much of the product worked is recorded per engine. | Engine | Connect as | Support | | :--- | :--- | :--- | diff --git a/README.md b/README.md index d33cee46..c15b92c6 100644 --- a/README.md +++ b/README.md @@ -99,20 +99,20 @@ You create a Postgres on a managed platform. It is ready in forty seconds. Then LibreDB Studio goes the other way. It deploys next to the data: a container, a Helm chart, an operator, a one-click template on your PaaS, or `npm i @libredb/studio` inside your own product. Nothing has to face outward. -Fourteen engines share one interface — PostgreSQL, MySQL, Oracle, SQL Server, SQLite, MongoDB, Redis, Couchbase, ClickHouse, Druid, Elasticsearch, OpenSearch, Apache Trino and Apache Cassandra — with the same explorer everywhere, and ER diagrams, schema diff and monitoring wherever the engine has something to report. Three of the fourteen are read-only because their own SQL is: Druid, Elasticsearch and OpenSearch have no `UPDATE` and no `CREATE TABLE` in the grammar at all, so those controls are reported as unsupported instead of failing when used. Cassandra is the newest, and the one that reports the least on purpose: it publishes no row count and no size that is true, so the object browser shows neither rather than showing a number that is wrong — the estimate it does publish counts partitions from flushed files, and it read 143 for a 500-row table. Trino is the other odd one: it is a query engine rather than a database, so it declares no keys and no indexes and reports the bytes as belonging to the systems behind its connectors. +Fifteen engines share one interface — PostgreSQL, MySQL, Oracle, SQL Server, SQLite, libSQL, MongoDB, Redis, Couchbase, ClickHouse, Druid, Elasticsearch, OpenSearch, Apache Trino and Apache Cassandra — with the same explorer everywhere, and ER diagrams, schema diff and monitoring wherever the engine has something to report. Three of the fifteen are read-only because their own SQL is: Druid, Elasticsearch and OpenSearch have no `UPDATE` and no `CREATE TABLE` in the grammar at all, so those controls are reported as unsupported instead of failing when used. Cassandra is the newest, and the one that reports the least on purpose: it publishes no row count and no size that is true, so the object browser shows neither rather than showing a number that is wrong — the estimate it does publish counts partitions from flushed files, and it read 143 for a 500-row table. Trino is the other odd one: it is a query engine rather than a database, so it declares no keys and no indexes and reports the bytes as belonging to the systems behind its connectors. And nothing is held back. Single sign-on, ER diagrams, the AI features and the NoSQL engines all ship in the MIT build. MIT is not generosity here, it is a requirement of the architecture: you cannot place a per-seat licensed, feature-gated tool into every environment you own. ### Why LibreDB Studio? - **Deploys next to the data**: container, Helm chart, Rancher, OpenShift operator, one-click PaaS template, or embedded via npm. -- **Fourteen engines, one interface**: PostgreSQL, MySQL, Oracle, SQL Server, SQLite, MongoDB, Redis, Couchbase, ClickHouse, Druid, Elasticsearch, OpenSearch, Trino, Cassandra. +- **Fifteen engines, one interface**: PostgreSQL, MySQL, Oracle, SQL Server, SQLite, libSQL, MongoDB, Redis, Couchbase, ClickHouse, Druid, Elasticsearch, OpenSearch, Trino, Cassandra. - **Runs where you are**: browser, phone, Windows, MacOS, Linux desktop. - **A read-only agent, with your own model**: state a question, and the run drafts SQL, reads the results, and writes a report whose claims cite them. Gemini, OpenAI, or a local Ollama with open-source models. - **Nothing behind a wall**: RBAC, OIDC single sign-on, query audit trail, and ER diagrams all ship under MIT.

Multi-Database Connection Manager -
Connect to PostgreSQL, MySQL, Oracle, SQL Server, MongoDB, Couchbase, ClickHouse, Druid, Elasticsearch, OpenSearch, Trino, Cassandra, Redis, or SQLite with SSL/TLS and SSH Tunnel support. +
Connect to PostgreSQL, MySQL, Oracle, SQL Server, MongoDB, Couchbase, ClickHouse, Druid, Elasticsearch, OpenSearch, Trino, Cassandra, Redis, SQLite, or libSQL with SSL/TLS and SSH Tunnel support.

--- @@ -257,6 +257,7 @@ Standalone application only: the embedded `@libredb/studio` package carries no a | **Oracle** | `oracledb` (Thin mode) | Full SQL IDE, `FETCH FIRST N ROWS` pagination, `V$` monitoring views, `ANALYZE TABLE`, `ALTER INDEX REBUILD`, transactions | | **SQL Server** | `mssql` (tedious) | Full SQL IDE, `TOP N` / `OFFSET FETCH` pagination, `sys.dm_*` DMVs, `UPDATE STATISTICS`, `DBCC CHECKDB`, transactions, Azure SQL auto-detect | | **SQLite** | `bun:sqlite` / `node:sqlite` (runtime-selected) | Full SQL IDE, file-based or in-memory databases (server-local file) | +| **libSQL** | none — HTTP (the Hrana protocol, `POST /v2/pipeline`, port 8080) | Full SQL IDE against a libSQL server or Turso Cloud — the same SQLite dialect as the row above, reached across a network instead of on disk. `EXPLAIN QUERY PLAN`, `sqlite_master` and `pragma_*` introspection, and real per-table bytes from `dbstat`, which the file-based driver above cannot read. The credential is an auth token rather than a password. Two maintenance operations only, `REINDEX` and `PRAGMA integrity_check`: the server refuses `VACUUM`, `ANALYZE`, `PRAGMA optimize` and `PRAGMA wal_checkpoint` outright, so no control is offered for them | | **MongoDB** | `mongodb` | JSON query editor, collection operations (find, aggregate, insert, update, delete) | | **Couchbase** | none — HTTP (Query + management REST) | Full SQL++ IDE, EXPLAIN plans, bucket/scope/collection explorer, `INFER` column inference, read-your-writes consistency, `UPDATE STATISTICS` / `BUILD INDEX` / request kill | | **ClickHouse** | none — HTTP (SQL interface, port 8123) | Full SQL IDE, JSON EXPLAIN plan trees, system-table schema introspection, `OPTIMIZE TABLE` / table statistics / query kill maintenance | @@ -267,9 +268,9 @@ Standalone application only: the embedded `@libredb/studio` package carries no a | **Apache Cassandra** | `cassandra-driver` (pure JS, no native module) | CQL IDE over the native protocol (port 9042), keyspace browser marking partition and clustering keys, `system_views` overview, uptime and running statements. No EXPLAIN (the keyword is not in CQL), no cancellation (the protocol has none), no maintenance (every operation is a `nodetool` action), and **no row counts or sizes**: the only figures Cassandra publishes are partition estimates from flushed files and whole mebibytes, so neither is shown rather than shown wrong | | **Redis** | `ioredis` | Command editor, key browser, INFO-based monitoring | -> **Twenty-six more engines have no driver of their own.** The fourteen above are the drivers this build ships. Twenty-six further engines speak one of those wire protocols and connect through an existing driver unchanged, so fourteen drivers reach forty named engines in all. They are MariaDB, Percona Server for MySQL, TiDB, Vitess, StarRocks, Apache Doris, OceanBase, SingleStore, Databend, Citus, Percona Distribution for PostgreSQL, ParadeDB, OrioleDB, TimescaleDB, YugabyteDB, AlloyDB Omni, Apache Cloudberry (incubating), CockroachDB, Materialize and RisingWave (as PostgreSQL or MySQL), Valkey, DragonflyDB, KeyDB and Garnet (as Redis), FerretDB (as MongoDB), and ScyllaDB (as Cassandra). Each was measured against a live instance, and how much of the product works differs per engine. MariaDB, both Percona distributions, TiDB, Vitess, AlloyDB Omni, Citus, TimescaleDB, YugabyteDB, ParadeDB, OrioleDB, Valkey, DragonflyDB, KeyDB and FerretDB behave as their driver's own engine, though three of them report statistics you should not trust: a Citus distributed table and a TimescaleDB hypertable report row counts and sizes that are wrong rather than missing, and YugabyteDB reports 0 until you run `ANALYZE`. Vitess is not one of those three, its row counts and sizes being exact to the byte, but a running query cannot be cancelled there: vtgate refuses `KILL QUERY` and the statement runs to completion. AlloyDB Omni is not one of them either, reporting 2000 rows for 2000 and 270336 bytes for 270336, but two things there surprise: `version()` names AlloyDB nowhere, so the version panel cannot be told apart from a stock PostgreSQL 17, and eight of AlloyDB's own `google_ml` tables list in the object browser, which any role that can connect at all may also read. StarRocks reports itself as MySQL 5.1 and loses its overview, health and session panels, its monitoring dashboard rendering six panels with the session one carrying the engine's own refusal; Apache Doris - the engine StarRocks is a fork of - loses only the overview and health panels, to one statement form its grammar rejects, and is the more trustworthy of the two where it counts: it reports 2000 rows and 10187 bytes for a table holding exactly that, where StarRocks reports zeros, though a freshly loaded table there reads 0 for about a minute before its background statistics land, no index is ever reported, and a foreign key is accepted, listed by `SHOW CONSTRAINTS`, invisible to the ER diagram and unenforced; Cloudberry loses the monitoring dashboard and its table and index statistics, all three to one MPP planner restriction, and reads a foreign key back as though it were enforced when it is not, though its row counts are correct; CockroachDB loses the object browser and the size panels; OceanBase answers fourteen of the fifteen surfaces but only twelve of them usefully, health failing outright because its tenant has no `performance_schema` database at all and every size reading 0 B, though its row counts are correct once `ANALYZE TABLE` has run; SingleStore lost five surfaces to a cause that was ours rather than its own - the provider sent every statement through the prepared-statement protocol, which SingleStore refuses for the `SHOW` and `EXPLAIN` statements four panels need - and four of those five are now recovered, its Explain panel being the one that is not, because there the grammar wants `EXPLAIN JSON` and the statement fails on either protocol; its numbers are still missing rather than wrong, a 2000-row table reading 0 rows and 0 B with no `ANALYZE` able to change it; ScyllaDB loses five surfaces and Test Connection with them, all six to one absent keyspace - the overview, health, performance-metrics, active-session and monitoring panels read Cassandra's `system_views` virtual tables and ScyllaDB has no `system_views` keyspace at all - those five now degrade to empty rather than throwing, so Test Connection passes and the dialog saves the connection, which it could not do at all until that change - while the editor and the object browser work in full, every one of 18 CQL types reading back byte-identically to the Cassandra 5.0.9 probed in the same pass; ParadeDB and OrioleDB are both full and their costs are opposites: ParadeDB's nine extensions put 41 objects in the object browser for 2 user tables and break agent plan mode on a stock install, while OrioleDB's browser is clean and its own storage is invisible to PostgreSQL's size functions, so every index reads 0 bytes and the cache hit ratio reads N/A. Materialize, RisingWave and Databend are query-editor-only, and Databend is the one of those three whose catalogs answer perfectly well when asked directly - the object browser is empty because our parameterised reads use a prepared protocol it does not implement. Garnet behaves as Redis and is the one relative here whose own version was available and went unread: the overview shows Redis 7.4.3 while `INFO` also carries `garnet_version:2.1.5`, and two of its readings are absences wearing a value, every size showing 0 B because it publishes no `used_memory` and the cache hit ratio showing 100% because it publishes no keyspace counters. The per-engine detail, with the exact version probed, is in [`docs/providers/README.md`](docs/providers/README.md#wire-compatible-engines) — we publish a name only after connecting to it, so a name absent there is untested rather than unsupported. +> **Twenty-six more engines have no driver of their own.** The fifteen above are the drivers this build ships. Twenty-six further engines speak one of those wire protocols and connect through an existing driver unchanged, so fifteen drivers reach forty-one named engines in all. They are MariaDB, Percona Server for MySQL, TiDB, Vitess, StarRocks, Apache Doris, OceanBase, SingleStore, Databend, Citus, Percona Distribution for PostgreSQL, ParadeDB, OrioleDB, TimescaleDB, YugabyteDB, AlloyDB Omni, Apache Cloudberry (incubating), CockroachDB, Materialize and RisingWave (as PostgreSQL or MySQL), Valkey, DragonflyDB, KeyDB and Garnet (as Redis), FerretDB (as MongoDB), and ScyllaDB (as Cassandra). Each was measured against a live instance, and how much of the product works differs per engine. MariaDB, both Percona distributions, TiDB, Vitess, AlloyDB Omni, Citus, TimescaleDB, YugabyteDB, ParadeDB, OrioleDB, Valkey, DragonflyDB, KeyDB and FerretDB behave as their driver's own engine, though three of them report statistics you should not trust: a Citus distributed table and a TimescaleDB hypertable report row counts and sizes that are wrong rather than missing, and YugabyteDB reports 0 until you run `ANALYZE`. Vitess is not one of those three, its row counts and sizes being exact to the byte, but a running query cannot be cancelled there: vtgate refuses `KILL QUERY` and the statement runs to completion. AlloyDB Omni is not one of them either, reporting 2000 rows for 2000 and 270336 bytes for 270336, but two things there surprise: `version()` names AlloyDB nowhere, so the version panel cannot be told apart from a stock PostgreSQL 17, and eight of AlloyDB's own `google_ml` tables list in the object browser, which any role that can connect at all may also read. StarRocks reports itself as MySQL 5.1 and loses its overview, health and session panels, its monitoring dashboard rendering six panels with the session one carrying the engine's own refusal; Apache Doris - the engine StarRocks is a fork of - loses only the overview and health panels, to one statement form its grammar rejects, and is the more trustworthy of the two where it counts: it reports 2000 rows and 10187 bytes for a table holding exactly that, where StarRocks reports zeros, though a freshly loaded table there reads 0 for about a minute before its background statistics land, no index is ever reported, and a foreign key is accepted, listed by `SHOW CONSTRAINTS`, invisible to the ER diagram and unenforced; Cloudberry loses the monitoring dashboard and its table and index statistics, all three to one MPP planner restriction, and reads a foreign key back as though it were enforced when it is not, though its row counts are correct; CockroachDB loses the object browser and the size panels; OceanBase answers fourteen of the fifteen surfaces but only twelve of them usefully, health failing outright because its tenant has no `performance_schema` database at all and every size reading 0 B, though its row counts are correct once `ANALYZE TABLE` has run; SingleStore lost five surfaces to a cause that was ours rather than its own - the provider sent every statement through the prepared-statement protocol, which SingleStore refuses for the `SHOW` and `EXPLAIN` statements four panels need - and four of those five are now recovered, its Explain panel being the one that is not, because there the grammar wants `EXPLAIN JSON` and the statement fails on either protocol; its numbers are still missing rather than wrong, a 2000-row table reading 0 rows and 0 B with no `ANALYZE` able to change it; ScyllaDB loses five surfaces and Test Connection with them, all six to one absent keyspace - the overview, health, performance-metrics, active-session and monitoring panels read Cassandra's `system_views` virtual tables and ScyllaDB has no `system_views` keyspace at all - those five now degrade to empty rather than throwing, so Test Connection passes and the dialog saves the connection, which it could not do at all until that change - while the editor and the object browser work in full, every one of 18 CQL types reading back byte-identically to the Cassandra 5.0.9 probed in the same pass; ParadeDB and OrioleDB are both full and their costs are opposites: ParadeDB's nine extensions put 41 objects in the object browser for 2 user tables and break agent plan mode on a stock install, while OrioleDB's browser is clean and its own storage is invisible to PostgreSQL's size functions, so every index reads 0 bytes and the cache hit ratio reads N/A. Materialize, RisingWave and Databend are query-editor-only, and Databend is the one of those three whose catalogs answer perfectly well when asked directly - the object browser is empty because our parameterised reads use a prepared protocol it does not implement. Garnet behaves as Redis and is the one relative here whose own version was available and went unread: the overview shows Redis 7.4.3 while `INFO` also carries `garnet_version:2.1.5`, and two of its readings are absences wearing a value, every size showing 0 B because it publishes no `used_memory` and the cache hit ratio showing 100% because it publishes no keyspace counters. The per-engine detail, with the exact version probed, is in [`docs/providers/README.md`](docs/providers/README.md#wire-compatible-engines) — we publish a name only after connecting to it, so a name absent there is untested rather than unsupported. -> **Transport security is cross-cutting, not per engine.** The SSH tunnel is opened before the provider connects and the connection is rewritten to the local endpoint, so it is provider-independent: it applies to any connection configured with a host and a port. A connection entered as a connection string instead (an option for MongoDB, Couchbase and ClickHouse) carries neither, so it is not tunnelled; SQLite has neither either. The SSL/TLS panel is honoured by every engine that shows it — which is every engine except the two file-based ones, SQLite and the embedded LibreDB, where no transport exists to secure and no panel is offered. On Trino it is load-bearing rather than optional, because the coordinator refuses a password over plain HTTP. Oracle is the one engine whose mapping carries a caveat worth stating up front: its Thin driver always verifies the certificate chain, so `require` needs the server's CA supplied when that certificate is self-signed, and a connect string pasted whole keeps whatever protocol it names. +> **Transport security is cross-cutting, not per engine.** The SSH tunnel is opened before the provider connects and the connection is rewritten to the local endpoint, so it is provider-independent: it applies to any connection configured with a host and a port. A connection entered as a connection string instead (an option for MongoDB, Couchbase, ClickHouse and libSQL) carries neither, so it is not tunnelled; SQLite has neither either. The SSL/TLS panel is honoured by every engine that shows it — which is every engine except the two file-based ones, SQLite and the embedded LibreDB, where no transport exists to secure and no panel is offered. On Trino it is load-bearing rather than optional, because the coordinator refuses a password over plain HTTP. Oracle is the one engine whose mapping carries a caveat worth stating up front: its Thin driver always verifies the certificate chain, so `require` needs the server's CA supplied when that certificate is self-signed, and a connect string pasted whole keeps whatever protocol it names. > All SQL databases share: schema explorer, ER diagrams, schema diff & migration, display masking (preview), monitoring dashboard, and connection string import. Druid, Elasticsearch, OpenSearch and Trino are each the exception twice over: their HTTP SQL APIs have no URI convention this build can parse, so they are configured by host and port only, and a generated migration names the limitation instead of emitting column-modification DDL against an engine whose SQL contains none — as it also does for Couchbase's schemaless collections. An ER diagram over a search cluster draws boxes and no edges: an index declares no foreign keys and the engine's model has none to declare, which the provider states as `declaresForeignKeys: false` rather than leaving to be guessed from an empty list. @@ -287,7 +288,7 @@ Standalone application only: the embedded `@libredb/studio` package carries no a | **Editor** | Monaco Editor (VS Code Engine) | Web | | **AI** | Multi-Model (Gemini, OpenAI, Ollama, Custom) | Web, Mobile | | **Auth** | JWT (`jose`) + OIDC (`openid-client`), PKCE, Role Mapping | Web, Mobile | -| **Database** | PostgreSQL, MySQL, Oracle, SQL Server, SQLite, MongoDB, Couchbase, ClickHouse, Apache Druid, Elasticsearch, OpenSearch, Apache Trino, Redis | Web, Mobile | +| **Database** | PostgreSQL, MySQL, Oracle, SQL Server, SQLite, libSQL, MongoDB, Couchbase, ClickHouse, Apache Druid, Elasticsearch, OpenSearch, Apache Trino, Redis | Web, Mobile | | **Charts** | Recharts (Bar, Line, Pie, Area, Scatter, Histogram, Stacked) | Web, Mobile | | **ERD** | React Flow, ELK.js (auto-layout) | Web | | **State/Grid** | TanStack Table & Virtual | Web, Mobile | @@ -380,7 +381,7 @@ journalctl -u libredb-studio ### Prerequisites - [Bun](https://bun.sh/) (Recommended) or Node.js 24+ - - A target database to query (PostgreSQL, MySQL, Oracle, SQL Server, SQLite, MongoDB, Couchbase, ClickHouse, Apache Druid, Elasticsearch, OpenSearch, Apache Trino, or Redis) + - A target database to query (PostgreSQL, MySQL, Oracle, SQL Server, SQLite, libSQL, MongoDB, Couchbase, ClickHouse, Apache Druid, Elasticsearch, OpenSearch, Apache Trino, or Redis) ### Quick Start (Local) 1. **Clone & Install** diff --git a/README_ja.md b/README_ja.md index c3c8fdac..149d1d4b 100644 --- a/README_ja.md +++ b/README_ja.md @@ -84,7 +84,7 @@ LibreDB Studioは逆向きです。**データをツールのところへ持っ ### 14のエンジン、1つのインターフェース -PostgreSQL · MySQL · Oracle · SQL Server · SQLite · MongoDB · Redis · Couchbase · ClickHouse · Apache Druid · Elasticsearch · OpenSearch · Apache Trino · Apache Cassandra +PostgreSQL · MySQL · Oracle · SQL Server · SQLite · libSQL · MongoDB · Redis · Couchbase · ClickHouse · Apache Druid · Elasticsearch · OpenSearch · Apache Trino · Apache Cassandra スキーマエクスプローラ、ER図、スキーマ差分、モニタリングは全SQLエンジンで共通です。MongoDBとRedisはSQLエンジンではないため、ER図とスキーマ差分はありません。Druid、Elasticsearch、OpenSearch、TrinoはこのビルドがパースできるURI形式を持たないためhostとportで設定する二重の例外で、生成されるマイグレーションもDDLを出力せず制約を明示します（Couchbaseのスキーマレスなコレクションも同様）。検索クラスタのER図は箱だけで線がありません。インデックスは外部キーを宣言せず、エンジンのモデルにも宣言できる外部キーが存在しないためです。 @@ -95,6 +95,7 @@ PostgreSQL · MySQL · Oracle · SQL Server · SQLite · MongoDB · Redis · Cou | **Oracle** | `oracledb`（Thinモード） | フルSQL IDE、`FETCH FIRST N ROWS`、`V$`監視ビュー、`ANALYZE TABLE`、`ALTER INDEX REBUILD`、トランザクション | | **SQL Server** | `mssql` (tedious) | フルSQL IDE、`TOP N` / `OFFSET FETCH`、`sys.dm_*` DMV、`UPDATE STATISTICS`、`DBCC CHECKDB`、トランザクション、Azure SQL自動判別 | | **SQLite** | `bun:sqlite` / `node:sqlite`（実行時選択） | フルSQL IDE、ファイル型・インメモリ型 | +| **libSQL** | ドライバなし、HTTPのみ（Hranaプロトコル、`POST /v2/pipeline`、8080） | フルSQL IDE。自前運用のlibSQLサーバー（`sqld`）とTurso Cloudの両方に同じtype-idで接続します。ネットワーク越しのSQLite方言で、`dbstat`による実測のテーブル・インデックスサイズが読めます。認証情報はパスワードではなくauthトークンです。メンテナンスはReindexと整合性チェックのみ。`VACUUM`、`ANALYZE`、`PRAGMA optimize`はサーバー側が拒否します | | **MongoDB** | `mongodb` | JSONクエリエディタ、コレクション操作（find、aggregate、insert、update、delete） | | **Couchbase** | ドライバなし、HTTPのみ（Query + 管理REST） | フルSQL++ IDE、EXPLAIN、bucket/scope/collectionエクスプローラ、`INFER`によるカラム推論 | | **ClickHouse** | ドライバなし、HTTPのみ（SQLインターフェース、8123） | フルSQL IDE、JSON EXPLAINツリー、システムテーブルからのスキーマ取得、`OPTIMIZE TABLE` | diff --git a/README_zh.md b/README_zh.md index b7f73a7f..cbca07f0 100644 --- a/README_zh.md +++ b/README_zh.md @@ -86,7 +86,7 @@ LibreDB Studio 走另一条路：**工具去找数据，而不是把数据搬来 ### 十四种引擎，一个界面 -PostgreSQL · MySQL · Oracle · SQL Server · SQLite · MongoDB · Redis · Couchbase · ClickHouse · Apache Druid · Elasticsearch · OpenSearch · Apache Trino · Apache Cassandra +PostgreSQL · MySQL · Oracle · SQL Server · SQLite · libSQL · MongoDB · Redis · Couchbase · ClickHouse · Apache Druid · Elasticsearch · OpenSearch · Apache Trino · Apache Cassandra 所有 SQL 引擎共用同一套 schema 浏览器、ER 图、schema 对比和监控面板。MongoDB 和 Redis 不属于 SQL 引擎，没有 ER 图和 schema 对比；Druid、Elasticsearch、OpenSearch 和 Trino 都是双重例外：它们的 HTTP SQL 接口没有本构建能解析的 URI 形式，只能按 host/port 配置，而且生成的迁移会直接说明限制，而不是对一个 SQL 里根本没有列变更语句的引擎硬输出 DDL；Couchbase 的 schemaless collection 同理。搜索集群的 ER 图只有方框没有连线：索引不声明外键，引擎模型里也没有外键可声明。 @@ -97,6 +97,7 @@ PostgreSQL · MySQL · Oracle · SQL Server · SQLite · MongoDB · Redis · Cou | **Oracle** | `oracledb`（Thin 模式） | 完整 SQL IDE、`FETCH FIRST N ROWS` 分页、`V$` 监控视图、`ANALYZE TABLE`、`ALTER INDEX REBUILD`、事务 | | **SQL Server** | `mssql` (tedious) | 完整 SQL IDE、`TOP N` / `OFFSET FETCH` 分页、`sys.dm_*` DMV、`UPDATE STATISTICS`、`DBCC CHECKDB`、事务、自动识别 Azure SQL | | **SQLite** | `bun:sqlite` / `node:sqlite`（运行时自选） | 完整 SQL IDE，文件型或内存型数据库 | +| **libSQL** | 无驱动，纯 HTTP（Hrana 协议，`POST /v2/pipeline`，8080 端口） | 完整 SQL IDE，同一个 type-id 同时连接自建 libSQL 服务器（`sqld`）与 Turso Cloud。就是跨网络的 SQLite 方言，并能通过 `dbstat` 读到真实的表与索引字节数。凭据是 auth token 而不是密码。维护操作只有 Reindex 和完整性检查：`VACUUM`、`ANALYZE`、`PRAGMA optimize` 都被服务端拒绝 | | **MongoDB** | `mongodb` | JSON 查询编辑器，集合操作（find、aggregate、insert、update、delete） | | **Couchbase** | 无驱动，纯 HTTP（Query + 管理 REST） | 完整 SQL++ IDE、EXPLAIN、bucket/scope/collection 浏览器、`INFER` 字段推断 | | **ClickHouse** | 无驱动，纯 HTTP（SQL 接口，8123 端口） | 完整 SQL IDE、JSON EXPLAIN 树、系统表 schema 自省、`OPTIMIZE TABLE` | diff --git a/charts/libredb-studio/Chart.yaml b/charts/libredb-studio/Chart.yaml index dab7df27..81881b68 100644 --- a/charts/libredb-studio/Chart.yaml +++ b/charts/libredb-studio/Chart.yaml @@ -2,7 +2,7 @@ apiVersion: v2 name: libredb-studio description: Web-based SQL IDE for cloud-native teams supporting fourteen engines - PostgreSQL, MySQL, SQLite, Oracle, SQL Server, MongoDB, Redis, Couchbase, ClickHouse, Apache Druid, Elasticsearch, OpenSearch, Apache Trino and Apache Cassandra type: application -version: 0.1.52 +version: 0.1.53 appVersion: "0.13.4" kubeVersion: ">=1.26.0-0" home: https://github.com/libredb/libredb-studio @@ -39,6 +39,10 @@ annotations: artifacthub.io/category: database artifacthub.io/license: MIT artifacthub.io/prerelease: "false" + # False: 0.1.53 changes no packaged template and no value - it names one more engine in + # the README, libSQL, which is a new provider in the app rather than a chart change. The + # README is a packaged file, so #167 costs it a chart version even though nothing an + # operator deploys moves. # False: 0.1.52 adds one value, config.authCookieSecure, and changes no behaviour on its # own - unset (the default) writes no AUTH_COOKIE_SECURE and the app keeps deciding, so # every existing install renders exactly as before. It makes an already-supported setting diff --git a/charts/libredb-studio/README.md b/charts/libredb-studio/README.md index be20978c..c88d45d5 100644 --- a/charts/libredb-studio/README.md +++ b/charts/libredb-studio/README.md @@ -2,7 +2,7 @@ [![Artifact Hub](https://img.shields.io/endpoint?url=https://artifacthub.io/badge/repository/libredb-studio)](https://artifacthub.io/packages/search?repo=libredb-studio) -Web-based SQL IDE for cloud-native teams supporting fourteen engines - PostgreSQL, MySQL, SQLite, Oracle, SQL Server, MongoDB, Redis, Couchbase, ClickHouse, Apache Druid, Elasticsearch, OpenSearch, Apache Trino and Apache Cassandra. +Web-based SQL IDE for cloud-native teams supporting fifteen engines - PostgreSQL, MySQL, SQLite, libSQL, Oracle, SQL Server, MongoDB, Redis, Couchbase, ClickHouse, Apache Druid, Elasticsearch, OpenSearch, Apache Trino and Apache Cassandra. ## Prerequisites @@ -40,7 +40,7 @@ helm install libredb libredb/libredb-studio \ ```bash helm install libredb oci://ghcr.io/libredb/charts/libredb-studio \ - --version 0.1.52 \ + --version 0.1.53 \ --set secrets.jwtSecret=$(openssl rand -base64 32) \ --set secrets.adminPassword=MyAdmin123 ``` diff --git a/database-compose.yml b/database-compose.yml index de831e8d..75a016f3 100644 --- a/database-compose.yml +++ b/database-compose.yml @@ -447,6 +447,51 @@ services: # provider hides both; the second one carries no leading dot, so the dot convention alone # does not catch it. + # --------------------------------------------------------------------------------------- + # libSQL server (issue #424, Phase 5). SQLite over a network: one container is the whole + # server, and it is the SELF-HOSTED half of the `libsql` type-id. The other half is Turso + # Cloud, which cannot be a compose service - it is an account - so the doc records what was + # measured there and this service is what anyone can repeat. + # --------------------------------------------------------------------------------------- + libsql: + # Pinned to the exact build the provider was live-verified against, the same rule the + # trino and clickhouse services follow: the refusal wording recorded in + # docs/providers/libsql.md ("unsupported statement: VACUUM") is a claim about sqld + # 0.24.33, and Turso Cloud words the identical refusal differently. + image: ghcr.io/tursodatabase/libsql-server:latest + container_name: libredb-libsql + restart: unless-stopped + environment: + # A single primary with no replica. Without it sqld starts in a mode that expects a + # primary to follow, and the first write fails rather than the server failing to start. + SQLD_NODE: primary + ports: + # 8080 carries the Hrana HTTP API AND the version route - one port, and there is no + # second protocol port to publish. Mapped to 18080 so it cannot collide with the trino + # service above, which owns 8080 on the host. + - "18080:8080" + healthcheck: + # sqld's own route, and it answers WITHOUT a token - which is exactly why it is a + # health check and not a connection test: the provider proves the credential with + # `SELECT 1` instead (see connect() in the provider). + # + # Spoken through bash's /dev/tcp rather than curl, because the image ships NEITHER + # curl NOR wget (checked before writing this, the same way the trino service's + # comment says to: `command -v wget || command -v curl` answers nothing, while + # /bin/bash and /bin/sqld are both there). A `CMD curl` here would have marked the + # service permanently unhealthy while the server answered every request. + # `CMD bash -c`, never `CMD-SHELL`: CMD-SHELL runs /bin/sh, which is dash here, and + # dash has no /dev/tcp - measured, the check failed with "cannot create + # /dev/tcp/localhost/8080: Directory nonexistent" while the server was answering. + test: + - CMD + - bash + - -c + - exec 3<>/dev/tcp/localhost/8080 && printf 'GET /health HTTP/1.0\r\n\r\n' >&3 && head -1 <&3 | grep -q 200 + interval: 10s + timeout: 5s + retries: 10 + # --------------------------------------------------------------------------------------- # Apache Trino (issue #424, Phase 2). One container is the whole cluster: the coordinator # runs the worker in-process by default, which is enough to answer every statement the diff --git a/deploy/digitalocean/assets/description-long.md b/deploy/digitalocean/assets/description-long.md index af407250..789dc266 100644 --- a/deploy/digitalocean/assets/description-long.md +++ b/deploy/digitalocean/assets/description-long.md @@ -8,7 +8,7 @@ LibreDB Studio gives you a full-featured database workspace in your browser — - **Fourteen engines, one interface** — PostgreSQL, MySQL, Oracle, SQL Server, SQLite, MongoDB, Redis, Couchbase, ClickHouse, Apache Druid, Elasticsearch, OpenSearch, Apache Trino and Apache Cassandra - **Read-only AI agent** — state a question and the agent investigates it, and every claim in its report cites the result it came from; it runs SQL on PostgreSQL and SQLite only, in a session the database enforces as read-only, so writes and DDL are refused by the engine rather than by reading the statement. On every other engine it drafts the statement and you run it, and nothing reaches your editor unless you consent to the hand-over when the run opens -- **AI query explanation** — one click turns an unfamiliar query into plain English, with your own schema as context, on PostgreSQL, MySQL, SQLite, Couchbase, ClickHouse, Apache Druid and Apache Trino: the write-up is derived from the engine's own `EXPLAIN` plan, so it is offered where an engine returns one (bring your own key: Gemini, OpenAI, Ollama or any OpenAI-compatible endpoint; off unless configured) +- **AI query explanation** — one click turns an unfamiliar query into plain English, with your own schema as context, on PostgreSQL, MySQL, SQLite, libSQL, Couchbase, ClickHouse, Apache Druid and Apache Trino: the write-up is derived from the engine's own `EXPLAIN` plan, so it is offered where an engine returns one (bring your own key: Gemini, OpenAI, Ollama or any OpenAI-compatible endpoint; off unless configured) - **Modern editor** — autocomplete, syntax highlighting, query history - **Zero-config start** — this 1-Click App boots fully configured; credentials are generated uniquely for your Droplet on first boot - **Self-hosted & private** — the app and its configuration store run entirely on your Droplet (local SQLite by default); traffic to the databases and optional AI providers you configure flows directly from your Droplet to those services diff --git a/deploy/railway/TEMPLATE_OVERVIEW.md b/deploy/railway/TEMPLATE_OVERVIEW.md index 1dce10da..e14ad889 100644 --- a/deploy/railway/TEMPLATE_OVERVIEW.md +++ b/deploy/railway/TEMPLATE_OVERVIEW.md @@ -12,7 +12,7 @@ Hosting LibreDB Studio means running a single stateless Next.js container that s - Spin up an admin/query UI right next to a Railway PostgreSQL or MySQL database in the same project. - Ask the read-only agent a question and get an answer whose every claim cites the result it came from — it executes SQL on PostgreSQL and SQLite only, in a session the database itself enforces as read-only, and on every other connection it drafts a statement for you to run yourself. - Draft a statement before anything runs: plan mode executes nothing it drafts, on every engine — its one reach into the database is the schema capture that grounds it, metadata only and no data rows — and the only statement that lands in your editor and runs there is the hand-over you consent to when the run opens. -- Explain an unfamiliar query in plain English, with the connected schema as context and the engine's own `EXPLAIN` plan as the source — offered on PostgreSQL, MySQL, SQLite, Couchbase, ClickHouse, Apache Druid and Apache Trino, the engines that return a plan. +- Explain an unfamiliar query in plain English, with the connected schema as context and the engine's own `EXPLAIN` plan as the source — offered on PostgreSQL, MySQL, SQLite, libSQL, Couchbase, ClickHouse, Apache Druid and Apache Trino, the engines that return a plan. ## Dependencies for libredb-studio Hosting diff --git a/deploy/rancher/CATALOG_LISTING.md b/deploy/rancher/CATALOG_LISTING.md index 1811c97c..da07a78a 100644 --- a/deploy/rancher/CATALOG_LISTING.md +++ b/deploy/rancher/CATALOG_LISTING.md @@ -118,13 +118,13 @@ schemas and run queries across PostgreSQL, MySQL, Oracle, SQL Server, SQLite, Mo Redis, Couchbase, ClickHouse, Apache Druid, Elasticsearch, OpenSearch, Apache Trino and Apache Cassandra from a single web interface, with no desktop client to install. Editing data follows the engine rather than the IDE: inline row editing on PostgreSQL, MySQL, -Oracle, SQL Server and SQLite, table creation on those five and Apache Trino, and +Oracle, SQL Server, SQLite and libSQL, table creation on those six and Apache Trino, and everywhere else the controls are reported as unsupported rather than offered and then failed — Elasticsearch SQL has no mutation in its grammar at all, OpenSearch's one mutation (`DELETE`) is off by default, and Druid SQL has no `UPDATE`, no `DELETE` and no `CREATE TABLE`. An optional AI assistant (bring your own key: Gemini, OpenAI, or a local model) writes up a query in plain English from the engine's own EXPLAIN plan, on -PostgreSQL, MySQL, SQLite, Couchbase, ClickHouse, Apache Druid and Apache Trino — the +PostgreSQL, MySQL, SQLite, libSQL, Couchbase, ClickHouse, Apache Druid and Apache Trino — the engines that return one — and runs a read-only investigation agent on PostgreSQL and SQLite whose every claim cites the result it came from, and that never writes: the session is read-only and the database, not the IDE, refuses writes and DDL. It stays off unless @@ -147,7 +147,7 @@ versions are documented and validated for every release. zero configuration required - Optional AI assistance (Gemini, OpenAI, or a self-hosted model; off by default): plain-English query explanation on the engines that return an EXPLAIN plan (PostgreSQL, - MySQL, SQLite, Couchbase, ClickHouse, Apache Druid, Apache Trino), and a read-only + MySQL, SQLite, libSQL, Couchbase, ClickHouse, Apache Druid, Apache Trino), and a read-only investigation agent on PostgreSQL and SQLite that never writes — the database enforces the read-only session, not the IDE - Hardened chart defaults: non-root, read-only root filesystem, NetworkPolicy, PDB, diff --git a/docs/ADDING_A_PROVIDER.md b/docs/ADDING_A_PROVIDER.md index 82e4b4fb..692ebcf8 100644 --- a/docs/ADDING_A_PROVIDER.md +++ b/docs/ADDING_A_PROVIDER.md @@ -166,10 +166,10 @@ directly; reshaping rows inside the transport would have broken schema loading. ```typescript // Before: -export type DatabaseType = 'postgres' | 'mysql' | 'sqlite' | 'mongodb' | 'redis' | 'oracle' | 'mssql' | 'libredb' | 'couchbase' | 'clickhouse' | 'druid' | 'elasticsearch' | 'opensearch' | 'trino' | 'cassandra'; +export type DatabaseType = 'postgres' | 'mysql' | 'sqlite' | 'libsql' | 'mongodb' | 'redis' | 'oracle' | 'mssql' | 'libredb' | 'couchbase' | 'clickhouse' | 'druid' | 'elasticsearch' | 'opensearch' | 'trino' | 'cassandra'; // After (example: adding CockroachDB): -export type DatabaseType = 'postgres' | 'mysql' | 'sqlite' | 'mongodb' | 'redis' | 'oracle' | 'mssql' | 'libredb' | 'couchbase' | 'clickhouse' | 'druid' | 'elasticsearch' | 'opensearch' | 'trino' | 'cassandra' | 'cockroachdb'; +export type DatabaseType = 'postgres' | 'mysql' | 'sqlite' | 'libsql' | 'mongodb' | 'redis' | 'oracle' | 'mssql' | 'libredb' | 'couchbase' | 'clickhouse' | 'druid' | 'elasticsearch' | 'opensearch' | 'trino' | 'cassandra' | 'cockroachdb'; ``` ### 1.2 — Add to `QueryTab.type` if needed @@ -317,7 +317,7 @@ Then add the type to the selectable list that drives the ConnectionModal picker: // Append to the existing list - do not retype it, or you will drop a provider from the picker. const selectableTypes: DatabaseType[] = [ 'postgres', 'mysql', 'sqlite', 'oracle', 'mssql', 'mongodb', 'couchbase', 'redis', 'libredb', - 'clickhouse', 'druid', 'elasticsearch', 'opensearch', 'trino', 'cassandra', + 'clickhouse', 'druid', 'elasticsearch', 'opensearch', 'trino', 'cassandra', 'libsql', 'cockroachdb', ]; ``` @@ -340,6 +340,7 @@ bun add # Apache Druid needs no driver — plain SQL over POST /druid/v2/sql (Router 8888 or Broker 8082) # Elasticsearch / OpenSearch need no driver — SQL over _sql / _plugins/_sql (port 9200) # Apache Trino needs no driver — SQL over its client protocol, POST /v1/statement (port 8080) +# libSQL needs no driver — SQLite's dialect over the Hrana protocol, POST /v2/pipeline (port 8080) # bun add cassandra-driver (Apache Cassandra — a binary protocol over TCP, so a driver is not # optional; this one is pure JS, which is the next best thing) ``` @@ -648,7 +649,7 @@ For the authoritative, code-verified reference for each shipped provider (extend driver, pooling, capabilities, labels, `prepareQuery` behaviour, and limitations), see the prime docs — they are the single source of truth and are kept in sync with the code: -**[docs/providers/](./providers/README.md)** → postgres · mysql · oracle · mssql · sqlite · redis · mongodb · couchbase · clickhouse · druid · elasticsearch · opensearch · trino · libredb +**[docs/providers/](./providers/README.md)** → postgres · mysql · oracle · mssql · sqlite · libsql · redis · mongodb · couchbase · clickhouse · druid · elasticsearch · opensearch · trino · libredb When implementing a new provider, the closest existing analogue is the best template: a pooled SQL provider (postgres/mysql), an embedded SQL provider (sqlite), a non-SQL provider (mongodb/redis), or @@ -769,7 +770,8 @@ The integration points, all of which need an entry. This is the list the Strateg published engine count silently undercount; it is listed here because the count in `README.md` and `docs/BRAND_MESSAGING.md` is derived from it and has to move in the same PR - [ ] `package.json` — the driver, **if** it needs one. A driver-free provider leaves it untouched, and - six shipped ids do: `couchbase`, `clickhouse`, `druid`, `elasticsearch`, `opensearch` and `trino` + seven shipped ids do: `couchbase`, `clickhouse`, `druid`, `elasticsearch`, `opensearch`, `trino` + and `libsql` each add nothing here - [ ] `database-compose.yml` — a service, so the next person can repeat the live pass. A distributed engine contributes a `profiles: [...]` set instead, as Druid's seven services do, so the default diff --git a/docs/AGENT.md b/docs/AGENT.md index ea08fe90..1a2ecf20 100644 --- a/docs/AGENT.md +++ b/docs/AGENT.md @@ -48,7 +48,7 @@ Three properties frame everything below, and each of them is load-bearing rather on which of its two readings it takes**: `agent-read-only` on the dialects `CATALOG_PLANS` serves, because it composes catalog statements, and `agent-operations` everywhere else, because asking a provider to describe its own schema sends nothing an engine has to plan. That is what lets grounding - reach the twelve the read-only profile refuses, and it still cannot narrow any workflow's + reach the fourteen the read-only profile refuses, and it still cannot narrow any workflow's reach: the profile whose acquisition would be refused is never the profile that capture asks for. Everything else about the three acquisitions is identical: the same `readOnly: true` open, the same optional least-privilege `agentUser`, and the same profiled cache, so neither an operations run nor an @@ -450,7 +450,7 @@ What a grounded plan run is given, and where each part comes from: which process happened to have read a catalog first. They are read from what the engine already holds — `pg_class.reltuples` and `pg_stats` on PostgreSQL, `sqlite_stat1` on SQLite — so no column is scanned and no value is read out of any row. `ESTIMATE_BUILDERS` serves those two dialects and - nothing else, and #414 added no engine to it: on the other twelve `readSchemaStatistics` answers + nothing else, and #414 added no engine to it: on the other fourteen `readSchemaStatistics` answers `DIALECT_HAS_NO_STATISTICS` — *"this engine does not hold statistics this run knows how to read"* — so **a known schema with no statistics is now the ORDINARY combination rather than a rare one**, and the two sentences the run is handed agree: the inventory is a record of what exists, and every @@ -571,7 +571,7 @@ Three consequences worth stating plainly, because each is easy to assume the oth 1. **A plan run costs statements now.** On PostgreSQL and SQLite grounding is catalog reads plus one statistics read (two on SQLite: the `sqlite_stat1` availability probe has to be its own statement, - because SQLite resolves table names at prepare time). On the other twelve it is **one** — the + because SQLite resolves table names at prepare time). On the other fourteen it is **one** — the single `db.schema.read` call, with no statistics read to add, since those dialects hold none this run knows how to read. They come out of the same per-run statement budget every other read does, and they are audited the same way. The ledger-reuse path saves the schema reading and deliberately @@ -817,7 +817,7 @@ Two consequences worth stating: composed path has and the provider path cannot: reads audited statement by statement rather than as one opaque call; foreign keys, which no provider can report on an engine that declares none; and SQLite's inventory, which is parsed out of the DDL text the engine stored and which its provider - does not expose in the same shape. Collapsing the other twelve onto the composed one is the thing + does not expose in the same shape. Collapsing the other fourteen onto the composed one is the thing #414 exists because nobody can do: a catalog statement has to be written per dialect and verified against a live server, and until it is, refusing the dialect was the honest answer and reading the provider is a better one. diff --git a/docs/AGENT_DATA_FLOW.md b/docs/AGENT_DATA_FLOW.md index 4a9311c6..7662524f 100644 --- a/docs/AGENT_DATA_FLOW.md +++ b/docs/AGENT_DATA_FLOW.md @@ -44,7 +44,7 @@ if it only lists the good news. | Surface | What it sends | Fenced? | | --- | --- | --- | | `POST /api/agent/classify`, **before any run exists** | Your objective, and nothing else. One short completion that asks the model to name one of the five workflows. It fires when you press **Start** with the workflow left on **Automatic**, which is the default; naming a workflow yourself under **Advanced** skips it entirely. See [the classification, before any run](#the-classification-before-any-run) | No — it carries no database content to fence. Your objective is sent as the user message, and the server's instructions tell the model to treat that text as data to classify and never as instructions to it | -| Any **run**, in either mode, on **any** engine | Your objective, and a schema inventory (table, column, index identifiers and column types) with its relations graph (identifiers only). **Since #414 that inventory leaves on every engine**, where before it left on PostgreSQL and SQLite alone: those two are read with catalog statements the server composes, and the other twelve by asking the connection's own provider to describe its schema. On **MongoDB and Couchbase** the provider works out a collection's fields from a **sample of your own documents** — no value from them is in the message, but the **existence** of a field there is derived from your data rather than read from a catalog, which is a weaker claim than a catalog reading's and is why that reading has its own operation id (`db.schema.read`) an operator can deny alone. When the reading cannot be taken — a refusal, a provider that cannot describe its own schema, a description that overran the time the run granted it — nothing of the schema leaves and a server sentence saying which of those happened goes in its place. See [the schema inventory](#3-the-schema-inventory--identifiers-and-types-fenced) | Everything derived from the database is wrapped in an untrusted-content fence before it reaches a prompt | +| Any **run**, in either mode, on **any** engine | Your objective, and a schema inventory (table, column, index identifiers and column types) with its relations graph (identifiers only). **Since #414 that inventory leaves on every engine**, where before it left on PostgreSQL and SQLite alone: those two are read with catalog statements the server composes, and the other fourteen by asking the connection's own provider to describe its schema. On **MongoDB and Couchbase** the provider works out a collection's fields from a **sample of your own documents** — no value from them is in the message, but the **existence** of a field there is derived from your data rather than read from a catalog, which is a weaker claim than a catalog reading's and is why that reading has its own operation id (`db.schema.read`) an operator can deny alone. When the reading cannot be taken — a refusal, a provider that cannot describe its own schema, a description that overran the time the run granted it — nothing of the schema leaves and a server sentence saying which of those happened goes in its place. See [the schema inventory](#3-the-schema-inventory--identifiers-and-types-fenced) | Everything derived from the database is wrapped in an untrusted-content fence before it reaches a prompt | | An **agent run** | The above, plus the rows of each read the model performed, up to 200 per read; engine error text; server-written refusals; server-minted ids | Same fence | | Any **run that continues a conversation**, in either mode, on any engine | **In addition to everything above**: the earlier steps' objectives — *your own earlier questions*, capped at 200 characters each — and the most recent step's report, which is a model's claims about your data. No row of any result, and no earlier step's report but the newest. Sent only when the rail attaches a previous run's id, which it does for a follow-up on the same connection; a run that starts its own conversation sends no such message. Bounded to 4000 characters by default and switchable off with `LIBREDB_AGENT_THREAD_CONTEXT=false`. See [the conversation](#2a-the-conversation-when-a-run-continues-one--fenced) | Same fence, identified as `operation agent/thread`: it is prose a user and a model wrote, and none of it is the server's voice | | An **agent run opened as Operate** | Your objective; a **schema inventory reduced to its own names and index names** (no column names, no column types, no relations graph), read whichever of the two ways that engine is read and named with whichever noun that engine's provider declares — tables, collections, datasources, key patterns; in Plan mode the engine's **row-count estimates** for those same tables where the engine holds any — PostgreSQL and SQLite — and nothing per column; and the rows of each curated reading — which, for the `sessions` and `slow-queries` kinds, include **other database users' in-flight statement text and their database usernames**. See [the operations workflow](#5a-the-operations-workflow-what-a-curated-reading-sends) | Same fence: the inventory and every reading's rows are database content and are fenced | @@ -197,7 +197,7 @@ One field of your connection record *is*: its **engine type**, the canonical typ spends it on the fence tag the deliverable must carry, and since the Operate-engine fix a prose plan spends it twice: on the rule that binds the readings it may name to the engine it is planning against, and on the fence tag for a reading that engine happens to express as a statement. It is -a server-side enum with fourteen members, so what it discloses is which of fourteen engines this +a server-side enum with sixteen members, so what it discloses is which of sixteen engines this connection is — never its host, its database name or its credentials, none of which reach a prompt at all (see [What never leaves](#what-never-leaves)). @@ -267,14 +267,17 @@ a refusal — nothing of the schema leaves and a server-written note says so in **Two readings produce it, and which one runs is the dialect's decision** (#414). On PostgreSQL and SQLite the server composes a catalog statement per kind and executes it read-only. On the other -twelve it invokes `db.schema.read`, which calls the connection's own `provider.getSchema()` — the +fourteen it invokes `db.schema.read`, which calls the connection's own `provider.getSchema()` — the inspection the sidebar performs when it lists your tables — and composes no statement at all. -**Twelve counts type-ids the factory can build, not engines a user would name**: `SHIPPED` holds -fourteen, `CATALOG_PLANS` serves two of them, and the remainder is what this second reading covers. -Every other "twelve" said about grounding in these docs counts the same thing. Two things it does +**Fourteen counts type-ids the factory can build, not engines a user would name**: `SHIPPED` holds +sixteen, `CATALOG_PLANS` serves two of them, and the remainder is what this second reading covers. +Every other count said about grounding in these docs counts the same thing. libSQL is one of the +fourteen and not one of the two: it speaks SQLite's dialect, but the read-only catalog path +`CATALOG_PLANS` serves needs a database-native read-only profile, and `PRAGMA query_only` is refused +by a libSQL server (see [`providers/libsql.md`](./providers/libsql.md)). Two things it does NOT count. The wire-compatible engines of [`docs/providers/README.md`](./providers/README.md) are not extra members — TiDB is grounded because it -arrives as `mysql`, and it is that type-id that is counted. And one of the twelve, the embedded +arrives as `mysql`, and it is that type-id that is counted. And one of the fourteen, the embedded `libredb`, reaches this path through a handle it does not open: the file takes an exclusive lock, so the grounding acquisition borrows the connection's own open provider rather than opening a second one that the lock would refuse (`findOpenSingleWriterProvider`, `src/lib/db/factory.ts`; see @@ -535,7 +538,7 @@ The frozen execution policies are the ceiling on one run's egress, one row per w | Bound | Value | What it caps | | --- | --- | --- | | `maxResultRows` / `maxResultBytes` | 200 rows / 256 KiB | The most one read can return — and therefore the most one tool result can send | -| `maxStatementsPerRun` | 18-45, by workflow | Reads per drive, grounding reads and repairs included — the composed catalog reads and, since #414, the one `db.schema.read` call that replaces them on the other twelve. The figures did not move for it: that path is the cheapest of the three, so nothing had to be bought (`docs/AGENT.md`, the budget arithmetic) | +| `maxStatementsPerRun` | 18-45, by workflow | Reads per drive, grounding reads and repairs included — the composed catalog reads and, since #414, the one `db.schema.read` call that replaces them on the other fourteen. The figures did not move for it: that path is the cheapest of the three, so nothing had to be bought (`docs/AGENT.md`, the budget arithmetic) | | `AGENT_CONTEXT_PACK_MAX_CHARS` | 6000 | The fenced schema inventory | | `MAX_ER_CHARS` | 2000 | The fenced relations block | | `AGENT_MAX_OBJECTIVE_LENGTH` | 4000 | Your objective | diff --git a/docs/AGENT_GUIDE.md b/docs/AGENT_GUIDE.md index 0e30ad16..f4b37e12 100644 --- a/docs/AGENT_GUIDE.md +++ b/docs/AGENT_GUIDE.md @@ -229,7 +229,7 @@ workflow including **Operate**. now two different sentences, and the difference is the whole of what changed: - **Grounding — every engine.** What a Plan run is TOLD about your database. It needs no read-only - statement path, because the provider reading sends no statement, so it reaches all fourteen engines. + statement path, because the provider reading sends no statement, so it reaches all sixteen engines. - **Agent mode — PostgreSQL and SQLite.** What a run may DO by itself. Its tools execute statements and need a database-native read-only path, which only those two providers implement, so a schema-workflow Agent run on any other engine still ends *"The agent cannot run on this database @@ -923,7 +923,7 @@ Stated plainly, because a surface that hides its edges is the one that surprises (`src/lib/db/providers/embedded/libredb.ts`) — the bundled **SQLite sample** is the seeded connection to try a run against (`src/lib/seed/sqlite-sample.ts:131`). **Plan** mode still opens on every connection — the model is toolless there, so no profile has to be acquired for it — and since - #414 its **grounding** no longer takes this path at all on the other twelve: it asks the provider to + #414 its **grounding** no longer takes this path at all on the other fourteen: it asks the provider to describe its schema, which needs no read-only statement profile, so a Plan run on MongoDB or MySQL is ordinarily grounded while an Agent run on the same connection still cannot read anything. Where the reading does fail — a provider that cannot describe itself, a description that overran its diff --git a/docs/API_DOCS.md b/docs/API_DOCS.md index d5d14499..734cc9c4 100644 --- a/docs/API_DOCS.md +++ b/docs/API_DOCS.md @@ -1208,7 +1208,7 @@ interface DatabaseConnection { createdAt: Date; // Creation timestamp } -type DatabaseType = 'postgres' | 'mysql' | 'sqlite' | 'mongodb' | 'redis' | 'oracle' | 'mssql' | 'libredb' | 'couchbase' | 'clickhouse' | 'druid' | 'elasticsearch' | 'opensearch' | 'trino' | 'cassandra'; +type DatabaseType = 'postgres' | 'mysql' | 'sqlite' | 'libsql' | 'mongodb' | 'redis' | 'oracle' | 'mssql' | 'libredb' | 'couchbase' | 'clickhouse' | 'druid' | 'elasticsearch' | 'opensearch' | 'trino' | 'cassandra'; ``` ### TableSchema diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index 61a94e03..344db7c5 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -252,7 +252,7 @@ src/ └── lib/ ├── db/ # Database provider module │ ├── providers/ - │ │ ├── sql/ # postgres, mysql, sqlite (+ sqlite-driver runtime adapter), oracle, mssql, clickhouse/ (transport seam + SQL over HTTP), druid/ (transport seam + SQL over POST /druid/v2/sql), search/ (transport seam + SQL over HTTP; elasticsearch and opensearch, two ids one module), trino/ (transport seam + SQL over the Trino client protocol), cassandra/ (transport seam + CQL over the native protocol via cassandra-driver) + │ │ ├── sql/ # postgres, mysql, sqlite (+ sqlite-driver runtime adapter), oracle, mssql, clickhouse/ (transport seam + SQL over HTTP), druid/ (transport seam + SQL over POST /druid/v2/sql), search/ (transport seam + SQL over HTTP; elasticsearch and opensearch, two ids one module), trino/ (transport seam + SQL over the Trino client protocol), cassandra/ (transport seam + CQL over the native protocol via cassandra-driver), libsql/ (transport seam + SQLite's dialect over the Hrana protocol) │ │ ├── document/ # mongodb, couchbase/ (transport seam + SQL++ over REST) │ │ ├── keyvalue/ # redis │ │ └── embedded/ # libredb (built-in embedded provider for the sample connection) diff --git a/docs/BACKLOG.md b/docs/BACKLOG.md index 105510dd..94547438 100644 --- a/docs/BACKLOG.md +++ b/docs/BACKLOG.md @@ -402,6 +402,31 @@ than concatenated, and `runMaintenance` on an engine that answers `ANALYZE` with reports a result instead of throwing - both verified against the container, and the reading unchanged on MySQL, MariaDB and one analytics relative. +### D34. The migration generator emits `ADD CONSTRAINT` for SQLite, which SQLite cannot parse + +Found 2026-08-27 while registering the `libsql` type-id (issue #424, Phase 5), and it is the +`sqlite` dialect's own defect rather than the new one's. + +`generateMigrationSQL` declines a foreign key for `cassandra` and, since this run, for `libsql`. +The `sqlite` dialect falls through to the generic branch and emits + +```sql +ALTER TABLE "users" ADD CONSTRAINT "fk_users_dept_id" FOREIGN KEY ("dept_id") REFERENCES "departments"("id"); +``` + +SQLite has no ALTER that adds a constraint - a foreign key is declarable only inside `CREATE TABLE` - +so the statement is a syntax error wherever the file is run. Measured on the sibling engine, which +shares the grammar exactly: `near CONSTRAINT ... syntax error` on sqld 0.24.33 (SQLite 3.47.0). The +same file already declines the REMOVED direction for `sqlite` with a comment naming table +recreation, so only the added half is wrong - which is why it reads as an oversight rather than a +decision. + +Narrow, and it does not throw: the file generates, and the failure happens when someone runs it. + +**Done when:** `sqlite` declines an added foreign key the way `libsql` does, with a comment naming +recreation rather than a statement, and a test pins both directions for it - the way +`tests/unit/schema-diff/migration-generator.test.ts` now pins them for `libsql` and `cassandra`. + --- ## Value interpolation diff --git a/docs/BRAND_MESSAGING.md b/docs/BRAND_MESSAGING.md index 46133a66..25fc1ab2 100644 --- a/docs/BRAND_MESSAGING.md +++ b/docs/BRAND_MESSAGING.md @@ -59,7 +59,7 @@ One umbrella claim, three entry doors into it, three assurance layers underneath **The three doors** are three ways into that one claim. A campaign picks a door. It does not argue all three at once, because a piece that opens on three arguments has opened on none. 1. You created the database. The editor is already beside it. -2. One tab, fourteen databases. +2. One tab, fifteen databases. 3. Nothing sits behind an Enterprise wall. **Door 3 is deliberately third, and stays third.** Leading with it would define LibreDB as another company's opponent rather than as a position of its own, and it would put our credibility at the mercy of their pricing page. Third, the same fact reads as reassurance rather than accusation. @@ -80,12 +80,12 @@ Each door carries five parts. A promise whose proof does not resolve to a row in This is the sharpest of the three. It is the one claim no competitor can currently make. -### Door 2 — One tab, fourteen databases. +### Door 2 — One tab, fifteen databases. - **Audience:** teams running more than one kind of database, and the engineers who join them. - **Pain:** four databases, four clients, four sets of credentials, and a connection-string hunt for anyone new. -- **Promise:** fourteen engines in one interface, with the same exploration everywhere, and ER diagrams, schema diff and monitoring wherever the engine has something to show. (Not "across all of them": a search cluster declares no foreign keys, so its ER diagram has no edges, and item 4 below forbids the sentence that hides that.) -- **Proof:** fourteen providers, each with its own reference document under `docs/providers/`. +- **Promise:** fifteen engines in one interface, with the same exploration everywhere, and ER diagrams, schema diff and monitoring wherever the engine has something to show. (Not "across all of them": a search cluster declares no foreign keys, so its ER diagram has no edges, and item 4 below forbids the sentence that hides that.) +- **Proof:** fifteen providers, each with its own reference document under `docs/providers/`. - **Difference:** CloudBeaver Community bundles 18 driver modules and every one of them is SQL. MongoDB and Redis are not among them. The claim here is the span, never the count. See the honesty limits. @@ -124,7 +124,7 @@ Facts drift. Provider counts, channel counts and competitor editions all change, | Claim | Evidence | Source | Verified | | :--- | :--- | :--- | :--- | -| Fourteen database engines | One reference document per engine: PostgreSQL, MySQL, Oracle, SQL Server, SQLite, MongoDB, Redis, Couchbase, ClickHouse, Druid, Elasticsearch, OpenSearch, Apache Trino, Apache Cassandra. A fifteenth, `libredb.md`, is the embedded provider and is not an external engine. The count is derived, not written: `SHIPPED` in `src/lib/db/compatibility.ts` is an exhaustive record over `DatabaseType`, so the compiler refuses a missing id — read the count from there, minus `libredb` | `docs/providers/`, `src/lib/db/compatibility.ts` | 2026-08-20 | +| Fifteen database engines | One reference document per engine: PostgreSQL, MySQL, Oracle, SQL Server, SQLite, libSQL, MongoDB, Redis, Couchbase, ClickHouse, Druid, Elasticsearch, OpenSearch, Apache Trino, Apache Cassandra. A sixteenth, `libredb.md`, is the embedded provider and is not an external engine. The count is derived, not written: `SHIPPED` in `src/lib/db/compatibility.ts` is an exhaustive record over `DatabaseType`, so the compiler refuses a missing id — read the count from there, minus `libredb` | `docs/providers/`, `src/lib/db/compatibility.ts` | 2026-08-20 | | Published as an embeddable npm package | `"name": "@libredb/studio"`, version 0.9.66 | `package.json` | 2026-08-07 | | MIT licensed | "MIT License / Copyright (c) 2025 LibreDB" | `LICENSE` | 2026-08-07 | | 27 distribution channels, 22 live | "27 channels · 22 live · 4 pending · 1 deprecated" | `docs/CHANNELS.md` | 2026-08-07 | @@ -198,7 +198,7 @@ Documented so the message exists when it is wanted. Not part of the current camp An agency that knows what it may not embellish is forced to fill the remaining space with real material. That is the purpose of this section. 1. **Display masking is a presentation feature, not a security control.** It was built for screen sharing, demos and screenshots. It runs in the browser, and the API returns full values to an authorized user. State this as scope, not as a shortcoming. No message may imply that display masking is data protection, access control, or compliance. -2. **Never claim leadership on the number of SQL engines.** CloudBeaver Community bundles 18 driver modules covering 15 distinct engines, against fourteen here. The claim is the span across SQL, NoSQL, analytics and search in one interface, never the count. The count moves, so read it from `SHIPPED` in `src/lib/db/compatibility.ts` (an exhaustive record over `DatabaseType`, minus the embedded `libredb`) rather than from the last campaign — but a bigger number never becomes the claim. +2. **Never claim leadership on the number of SQL engines.** CloudBeaver Community bundles 18 driver modules covering 15 distinct engines, against fifteen here. The claim is the span across SQL, NoSQL, analytics and search in one interface, never the count. The count moves, so read it from `SHIPPED` in `src/lib/db/compatibility.ts` (an exhaustive record over `DatabaseType`, minus the embedded `libredb`) rather than from the last campaign — but a bigger number never becomes the claim. 3. **There is no LDAP.** Single sign-on is delivered through OIDC. Teams that require LDAP are told before they install, not after. 4. **Elasticsearch and OpenSearch are a read-only query surface, not a managed one.** Both ship as of [#424](https://github.com/libredb/libredb-studio/issues/424), so both belong in an engine list and in a polyglot story. What may not be implied is parity with PostgreSQL or MySQL. The query language is each product's own SQL endpoint — never ES|QL, which exists on one of the two and is deliberately unused, and never the native JSON DSL. Neither product's SQL grammar has `INSERT`, `UPDATE` or `CREATE TABLE`, so what ships is a query editor plus an index and mapped-field browser plus cluster health and per-index document counts and store sizes. There is no EXPLAIN, no maintenance operation, no slow-query panel and no session list; an ER diagram over a search cluster draws boxes and no edges, because the engine has no foreign keys to declare; aliases and data streams are not in the tree; and on Elasticsearch there is no `OFFSET`, so results after the first page cannot be requested at all. Say "query and browse", never "manage" — and never fold these two into a sentence that says a feature works "across all engines". What the story promises, the product has to hold. 5. **Trino is a query engine, and the story must not sell it as a database.** It ships as of @@ -223,7 +223,7 @@ An engineer speaking to an engineer. - Claim plus proof. Never adjective plus adjective. - No emoji. No exclamation marks. - Competitors are never disparaged by name. A comparison is a table with sources, and the reader draws the conclusion. -- Prefer the concrete number to the impressive word. "Fourteen engines" beats "extensive database support". +- Prefer the concrete number to the impressive word. "Fifteen engines" beats "extensive database support". - Say the limitation out loud. Stating scope precisely is what makes the rest of the claims credible to this audience. - Product terms stay in English in every language. LibreDB Studio, not a translated variant. @@ -243,7 +243,7 @@ Direction, not copy. This brief contains no finished copy for any surface. It sa | Surface | Leads with | Why | Current state | | :--- | :--- | :--- | :--- | -| GitHub About | Door 2, in one line | The first thing a repository visitor reads, and the only place a full sentence has to work with no layout around it | Defect: it names four engines against fourteen shipped. Fix before any campaign starts | +| GitHub About | Door 2, in one line | The first thing a repository visitor reads, and the only place a full sentence has to work with no layout around it | Defect: it names four engines against fifteen shipped. Fix before any campaign starts | | README opening | Door 1, then Door 3 | Visitors arriving from search need the claim before the feature list | States a category rather than a claim. Rewrite | | Website hero | Door 1 | The site as a whole can carry the story; the hero carries the claim and a single proof | States a category rather than a claim. Rewrite | | Website story section | The brand story, both scenes | Nothing on the site currently says why LibreDB exists | Missing entirely. Add | diff --git a/docs/DATABASE_PROVIDERS.md b/docs/DATABASE_PROVIDERS.md index 92599d3a..23d7fd17 100644 --- a/docs/DATABASE_PROVIDERS.md +++ b/docs/DATABASE_PROVIDERS.md @@ -146,7 +146,7 @@ QueryEditor /api/db/query ## Supported Databases -Fifteen type-ids are supported by fourteen provider modules — `elasticsearch` and `opensearch` share +Sixteen type-ids are supported by fifteen provider modules — `elasticsearch` and `opensearch` share one, `providers/sql/search/`. The count is derived from the exhaustive `SHIPPED` record in [`src/lib/db/compatibility.ts`](../src/lib/db/compatibility.ts) rather than written here twice. For the per-provider reference (driver, pooling, query format, diff --git a/docs/FEATURES.md b/docs/FEATURES.md index 7869bcc6..3f976e5e 100644 --- a/docs/FEATURES.md +++ b/docs/FEATURES.md @@ -47,6 +47,7 @@ * **ClickHouse:** Full support with **no driver dependency** — SQL over the documented HTTP interface, so the SQL editor and limiter both apply. Column types read verbatim from `system.columns`, JSON EXPLAIN plan trees, and `OPTIMIZE TABLE` / table-statistics / query-kill maintenance. * **Apache Druid:** Read-only support with **no driver dependency** — SQL over `POST /druid/v2/sql` on the Router (8888) or the Broker (8082), so the SQL editor and limiter both apply. Datasources and column types from `INFORMATION_SCHEMA`, native-query EXPLAIN plan trees, and monitoring from `sys.segments` / `sys.servers` / `sys.tasks`. Read-only is the engine, not the integration: Druid SQL has no `UPDATE`, no `DELETE` and no `CREATE TABLE`, and no maintenance operation is reachable from SQL, so those controls are reported as unsupported instead of failing when used. * **Apache Trino:** Full query support with **no driver dependency** — SQL over Trino's own client protocol (`POST /v1/statement`, port 8080), so the SQL editor and limiter both apply. Trino is a query *engine*, not a database, and the integration says so everywhere it matters. The connection's Database field pins one **catalog** (the tree is catalog-scoped and two levels deep; cross-catalog queries still run by qualifying names in full), the schema tree comes from that catalog's `information_schema`, monitoring from `system.runtime` and `jmx`, and row counts and sizes from a real `SHOW STATS` per table. What it declares as absent is absent from the engine, not from the integration: Trino's `information_schema` holds eight views and neither `table_constraints` nor `key_column_usage`, so no connector can declare a key through it — no primary keys, no foreign keys, no indexes, ER edges or inline row editing anywhere. It stores nothing, so the size panels name the catalogs and their connectors instead of inventing a footprint. Writes are the connector's decision rather than the engine's, and its refusal is shown verbatim. EXPLAIN renders as a plan tree from `EXPLAIN (FORMAT JSON)`, and never from `EXPLAIN ANALYZE`, which executes the statement. Query cancellation is real (`DELETE /v1/query/{id}`; abandoning a request does *not* stop the work), and `kill` is the one maintenance operation. Two traps a user meets immediately: a failed statement arrives as **HTTP 200** with the failure in the document, and a password is refused over plain HTTP even on a cluster with authentication disabled. + * **libSQL:** Full SQL support with **no driver dependency** - SQLite's own dialect over the Hrana protocol (`POST /v2/pipeline`, port 8080), reaching both a self-hosted libSQL server (`sqld`) and Turso Cloud through ONE type-id, because they speak the same protocol and embed the same SQLite (3.47.0 on both, measured). The credential is an auth **token** rather than a password - libSQL has no user names - so the connection dialog labels the field that way and takes the `libsql://-.turso.io?authToken=` URL Turso's CLI prints. Introspection is SQL (`sqlite_master`, the `pragma_*` functions) and the sizes are real: `dbstat` answers on both deployments, which the file-based SQLite driver under Bun cannot do at all, so tables and indexes report measured bytes. What is absent is the server's decision rather than the integration's: `VACUUM`, `ANALYZE`, `PRAGMA optimize` and `PRAGMA wal_checkpoint` are all refused by its statement allowlist, so only Reindex and Integrity Check are offered; `PRAGMA query_only` is refused too, which is why agent mode's read-only profile does not extend here. There is no session, no uptime and no statement history to show, because Hrana is stateless and libSQL publishes none of them. * **Apache Cassandra:** Full CQL support over the native protocol (port 9042) through `cassandra-driver` - pure JavaScript, so no native module reaches any distribution channel. The connection pins one **keyspace** (the `database` field) and carries a **`localDataCenter`**, which no other engine here needs and the driver refuses to connect without. The schema tree marks partition and clustering keys and orders columns partition-key-first, because declaration order is genuinely unrecoverable: `system_schema.columns.position` is -1 for every regular column and the rows arrive alphabetically. What it does NOT report is the point: **no row count and no size anywhere**, because Cassandra publishes neither honestly - `system.size_estimates` counts partitions per token range from flushed files (measured at 143 for a 500-row clustered table, 525 for a 500-row flat one) and `system_views.disk_usage` is whole mebibytes (`1 MiB` for 19,476 bytes) - so the object browser, the overview and the table, index and storage panels show nothing rather than something wrong. There is no EXPLAIN (the keyword is not in CQL), no cancellation (the protocol has no cancel frame), and no maintenance operation (compaction, repair, flush and cleanup are all `nodetool` actions over JMX). Values are normalized once at the driver boundary: a `blob` reaches the grid as `0x…` rather than a Buffer document, a `bigint`/`decimal`/`varint` as its exact digits, a `vector` as an array and a `duration` as `1mo2d3h`. * **Search Engines:** * **Elasticsearch / OpenSearch:** Read-only support with **no driver dependency** — SQL over `POST /_sql?format=json` (Elasticsearch) and `POST /_plugins/_sql` (OpenSearch), port 9200 on both. Two type-ids share one provider module because the two products differ only in wire detail. Indices become tables and mapped fields become columns, read from `_mapping` rather than from `SELECT *`: measured on Elasticsearch 9.1.4, an index mapping a `flattened` and a `nested` field answers `SELECT *` with **no columns at all**, so the mapping is the only honest source. Cluster health, per-index document counts and store sizes come from `_cluster/health`, `_cluster/stats` and `_cat/indices`. Read-only is the engine, not the integration: neither grammar has `INSERT`, `UPDATE` or `CREATE TABLE`, so `supportsCreateTable`, `supportsInlineRowEdit`, `supportsExplain` and `supportsMaintenance` are all false and those controls are hidden rather than offered and then failed. Elasticsearch SQL also has no `OFFSET` — measured, `LIMIT 2 OFFSET 1` is HTTP 400 there and HTTP 200 on OpenSearch — so on Elasticsearch a request for a second page is refused with that reason instead of returning page one again. ES|QL is deliberately unused: it exists on one of the two products, so it cannot be the shared query language. diff --git a/docs/SEED_CONNECTIONS.md b/docs/SEED_CONNECTIONS.md index a6ca4b19..dcff9da3 100644 --- a/docs/SEED_CONNECTIONS.md +++ b/docs/SEED_CONNECTIONS.md @@ -58,7 +58,7 @@ defaults: # Optional — merges managed/environment/ssl only connections: - id: "analytics-pg" # Required, unique, lowercase slug [a-z0-9-] name: "Analytics DB" # Required, display name in UI - type: postgres # Required: postgres|mysql|sqlite|mongodb|redis|oracle|mssql|libredb|couchbase|clickhouse|druid|elasticsearch|opensearch|trino|cassandra + type: postgres # Required: postgres|mysql|sqlite|libsql|mongodb|redis|oracle|mssql|libredb|couchbase|clickhouse|druid|elasticsearch|opensearch|trino|cassandra host: "${PG_HOST}" port: 5432 database: analytics @@ -145,7 +145,7 @@ connections: | `connections` | Yes | — | Array of connection definitions (min 1) | | `connections[].id` | Yes | — | Unique slug: `[a-z0-9-]+`, max 64 chars | | `connections[].name` | Yes | — | Display name, max 128 chars | -| `connections[].type` | Yes | — | Database type: `postgres`, `mysql`, `sqlite`, `mongodb`, `redis`, `oracle`, `mssql`, `libredb`, `couchbase`, `clickhouse`, `druid`, `elasticsearch`, `opensearch`, `trino`, `cassandra` | +| `connections[].type` | Yes | — | Database type: `postgres`, `mysql`, `sqlite`, `libsql`, `mongodb`, `redis`, `oracle`, `mssql`, `libredb`, `couchbase`, `clickhouse`, `druid`, `elasticsearch`, `opensearch`, `trino`, `cassandra` | | `connections[].host` | No | — | Hostname or IP | | `connections[].port` | No | — | Port number (1-65535) | | `connections[].database` | No | — | Database name (Couchbase: the bucket. Druid has one catalog and ignores it. Trino: the **catalog**) | diff --git a/docs/providers/README.md b/docs/providers/README.md index dc941e78..f68f6084 100644 --- a/docs/providers/README.md +++ b/docs/providers/README.md @@ -11,6 +11,7 @@ in lockstep with the code (see the tri-sync rule in [`../../CLAUDE.md`](../../CL | Oracle | `oracle` | SQL | `oracledb` (Thin) | SQL | [oracle.md](./oracle.md) | | Microsoft SQL Server | `mssql` | SQL | `mssql` | SQL (T-SQL) | [mssql.md](./mssql.md) | | SQLite | `sqlite` | SQL (embedded) | `bun:sqlite` (Bun) / `node:sqlite` (Node) | SQL | [sqlite.md](./sqlite.md) | +| libSQL | `libsql` | SQL (SQLite over a network) | none (HTTP: the Hrana protocol, `POST /v2/pipeline`) | SQL (SQLite) | [libsql.md](./libsql.md) | | Redis | `redis` | Key-Value | `ioredis` | JSON | [redis.md](./redis.md) | | MongoDB | `mongodb` | Document | `mongodb` | JSON (MQL) | [mongodb.md](./mongodb.md) | | Couchbase | `couchbase` | Document | none (HTTP: Query + management REST) | SQL (SQL++) | [couchbase.md](./couchbase.md) | @@ -26,7 +27,7 @@ in lockstep with the code (see the tri-sync rule in [`../../CLAUDE.md`](../../CL - **Filename = canonical type-id** (`postgres.md`, `mssql.md`, …), mirroring the source file (`src/lib/db/providers//.ts`, or a `/` directory when a provider is - split across modules, as Couchbase, ClickHouse, Druid, Trino and Cassandra are). The official product name (e.g. + split across modules, as Couchbase, ClickHouse, Druid, Trino, Cassandra and libSQL are). The official product name (e.g. "SQL Server") is used only in each doc's title and prose. **One directory may serve two type-ids** — `providers/sql/search/` is `elasticsearch` and `opensearch` — and each type-id still gets its own document, because the tri-sync invariant is per type-id and each doc is the prime reference for its diff --git a/docs/providers/libsql.md b/docs/providers/libsql.md new file mode 100644 index 00000000..3affb19c --- /dev/null +++ b/docs/providers/libsql.md @@ -0,0 +1,420 @@ +# libSQL Provider + +> libSQL support for LibreDB Studio, built on the Hrana HTTP protocol (`POST /v2/pipeline`) with +> **no driver dependency of any kind**: a statement is JSON in the body of a POST and the answer +> comes back through the runtime's own `fetch`. One type-id reaches two deployments — a self-hosted +> **libSQL server (`sqld`)** and **Turso Cloud** — because they speak the same protocol and embed the +> same SQLite. This document is the single reference point for the libSQL provider: design, +> architecture, usage, and tests. + +| | | +|---|---| +| **Status** | Implemented & shipped | +| **Database type id** | `libsql` | +| **Family** | SQL (`src/lib/db/providers/sql/libsql/`) | +| **Driver** | None — HTTP only (`fetch`, a runtime built-in) | +| **Query language** | `sql` (SQLite's dialect, 3.47.0 on both deployments measured) | +| **Default port** | `8080` (sqld's own). `443` when TLS is on, which is how Turso Cloud serves every database | +| **Connection pooling** | None — each statement is one stateless HTTP request | +| **Connection string** | Supported (`libsql://-.turso.io?authToken=`) | +| **Credential** | An auth TOKEN, not a password. libSQL has no user names, so the form labels the field "Auth Token" | +| **EXPLAIN** | `sqlite-queryplan` — the same `EXPLAIN QUERY PLAN` shape SQLite answers | +| **Transactions** | Not exposed (the provider closes its Hrana stream with each statement, so it holds no session for one) | +| **Maintenance** | `reindex` and `check` only — `VACUUM`, `ANALYZE`, `PRAGMA optimize` and `PRAGMA wal_checkpoint` are refused by the server on both deployments | +| **Source** | [`src/lib/db/providers/sql/libsql/`](../../src/lib/db/providers/sql/libsql/) | +| **Tests** | [`tests/integration/db/libsql-provider.test.ts`](../../tests/integration/db/libsql-provider.test.ts) + [`tests/unit/db/libsql/`](../../tests/unit/db/libsql/) | +| **Tracking issue** | [#424 — the database coverage map](https://github.com/libredb/libredb-studio/issues/424) | +| **Probed against** | `ghcr.io/tursodatabase/libsql-server` reporting `sqld 0.24.33 (f8fb14f3 2026-08-11)`, and a Turso Cloud database in `aws-eu-west-1`, both on 2026-08-27 | + +--- + +## 1. Overview + +libSQL is a fork of SQLite that keeps the file format and the dialect and adds a **server**. What a +client talks to is `sqld`, and what `sqld` speaks is Hrana: a list of requests posted as JSON, one +result per request, values encoded as `{ type, value }` with integers carried as decimal strings. +Turso Cloud is that same server, managed and reached over TLS on a hostname that identifies the +database. + +That is the whole reason this is a separate type-id from `sqlite` rather than a relative of it, and +the reason it is ONE id rather than two: + +- **Not `sqlite`.** The SQLite provider holds a FILE handle through a synchronous driver + (`bun:sqlite` / `node:sqlite`), reads sizes with `fs.statSync`, and enforces the agent read-only + profile with `PRAGMA query_only`. None of those three exist here: there is no file on this + machine, no handle to hold, and `PRAGMA query_only = true` is refused by the server. The dialect is + shared; the execution layer has nothing in common. +- **Not two ids.** A self-hosted `sqld` and a Turso Cloud database differ in host, TLS and token. + Every statement, every catalog and every refusal measured the same on both. Two ids would mean two + docs and two tests describing one set of measurements. + +**Turso Database — the Rust rewrite — is deliberately absent.** It is a different engine (written +from scratch, SQLite-compatible, concurrent writes and vector search) rather than a deployment of +this one, and on 2026-08-27 it published **no server image**: `tursodatabase/turso`, +`tursodatabase/tursodb` and `ghcr.io/tursodatabase/turso-server` were all unpullable, and the engine +ships as an in-process npm package (`@tursodatabase/database`). #424 publishes a name only after +connecting to it, so it earns no row here or in the compatibility registry. + +### Concept mapping + +| libSQL | This product | Note | +|---|---|---| +| A database (a hostname on Turso Cloud, a namespace on `sqld`) | The connection | There is no `database` field: the database IS the host | +| `main` schema | The single schema | SQLite has one, and the provider reports `schemaName: "main"` throughout | +| Table | Table | `sqlite_master` | +| Index | Index | `pragma_index_list`, minus SQLite's own `sqlite_autoindex_*` | +| Auth token (JWT) | The `password` field | Labelled "Auth Token" in the connection dialog | +| `dbstat` pages | Table and index bytes | Available on BOTH deployments, unlike `bun:sqlite` | + +--- + +## 2. Architecture + +``` +src/lib/db/providers/sql/libsql/ +├── index.ts LibSQLProvider — the 13 provider methods, capabilities, labels +├── introspect.ts every read expressed as SQL (sqlite_master, pragma_*, dbstat) +├── transport.ts the NEUTRAL seam: what a caller needs, not how Hrana spells it +└── hrana-transport.ts the only file that knows /v2/pipeline, the baton and the value codec +``` + +`tests/unit/db/libsql/seam-guard.test.ts` parses every file in that directory and fails the build if +Hrana vocabulary (`baton`, `base_url`, `affected_row_count`, `query_duration_ms`, +`replication_index`, `decltype`, `rows_read`, `rows_written`, the endpoint path) appears outside +`hrana-transport.ts`. `last_insert_rowid` is checked only as a payload READ, because it is also a +real SQLite function that any implementation may legitimately call. + +The seam is not ceremony: Hrana also runs over WebSocket, `@tursodatabase/database` embeds the engine +in-process, and `@libsql/client` speaks both. Any of them would answer the same questions — none of +them with a `baton`. + +--- + +## 3. Design decisions + +### 3.1 No driver, and the reason is the protocol's size + +`@libsql/client` is a dependency to speak a protocol that is three JSON shapes wide: a request list, +a result list, and a typed value. The whole transport is ~330 lines including the comments that +record the wire. The same judgement as Couchbase (#263), ClickHouse (#264), Druid (#265) and Trino +(#438). + +### 3.2 A failed statement answers HTTP 200 + +Measured on both deployments: + +``` +POST /v2/pipeline {"requests":[{"type":"execute","stmt":{"sql":"SELECT * FROM no_such_table"}},{"type":"close"}]} +HTTP/1.1 200 OK +{"baton":null,"base_url":null,"results":[ + {"type":"error","error":{"message":"SQLite error: no such table: no_such_table","code":"SQLITE_UNKNOWN"}}, + {"type":"ok","response":{"type":"close"}}]} +``` + +So `response.ok` says the pipeline was accepted, never that the statement ran. Every failure is read +out of `results[]`. This is the trap Trino's provider documents in the same words, and it is why the +transport carries `status: 200` on a statement error deliberately: the transport succeeded. + +### 3.3 An auth failure uses a different envelope, and a bad token is a 400 + +| Situation | Status | Body | +|---|---|---| +| No token to a private database | `401` | `{"error":"Unauthorized: \`unauthorized access attempt on database: empty JWT token\`"}` | +| Malformed token | `400` | `{"error":"JWT error: InvalidToken"}` | +| Statement rejected | `200` | `{"results":[{"type":"error","error":{"message":…,"code":…}}]}` | + +The auth envelope's `error` is a bare STRING rather than the `{ message, code }` object the statement +path uses, so the error reader handles both. 400 is in the provider's authentication set alongside +401 and 403 for that reason: keying only on 401 would report a malformed token as a connection +failure. + +### 3.4 Integers arrive as decimal strings, and they stay exact + +Hrana quotes every integer — `{"type":"integer","value":"2000"}` — which is the protocol protecting +64-bit values from a double. `decodeInteger` returns a `number` when the value is exactly +representable and the **decimal string verbatim** when it is not. `Number("9007199254740993")` is +9007199254740992, and a rounded rowid is a corruption nothing downstream can detect (#460 is the +Trino version of the same lesson). + +The one place a wide integer IS parsed to a double is `readNumber` in `introspect.ts`, and only for +display statistics: a row count above 2^53 is 9 quadrillion rows. Result CELLS never pass through it. + +### 3.5 The server refuses four statements, so four controls are withheld + +Measured on both deployments, with the wording differing and the code identical: + +| Statement | sqld 0.24.33 | Turso Cloud | +|---|---|---| +| `VACUUM` | `unsupported statement: VACUUM` | `SQL not allowed statement: VACUUM` | +| `ANALYZE` | `unsupported statement: ANALYZE` | `SQL not allowed statement: ANALYZE` | +| `PRAGMA optimize` | `unsupported statement: PRAGMA optimize` | `SQL not allowed statement: PRAGMA optimize` | +| `PRAGMA wal_checkpoint(TRUNCATE)` | `unsupported statement` | `SQL statement is not allowed` | +| `PRAGMA query_only = true` | `unsupported statement` | `SQL not allowed statement` | +| `REINDEX` | accepted | accepted | +| `PRAGMA integrity_check` | accepted | accepted | + +`maintenanceOperations` is therefore `["reindex", "check"]`. A Vacuum control that fails every time +is the Cloud Spanner shape #424 refuses rows for, and `runMaintenance` refuses the other types HERE +rather than sending them, so the message names the reason instead of relaying a server error for a +statement the user never typed. + +**Nothing keys on the wording.** The two deployments word the identical refusal differently under one +`SQL_PARSE_ERROR`, so a provider that matched on text would have been wrong on one of them from the +first day. + +### 3.6 `PRAGMA query_only` is refused, so there is no agent read-only profile + +`sqlite.ts` implements `queryReadOnly` by setting `query_only` and verifying the readback, per +statement — that is what refuses `VACUUM INTO ''` from a read-only handle. libSQL has no such +lever: the pragma is refused on both deployments. So this provider implements **no** `queryReadOnly`, +and the agent read-only profile stays PostgreSQL + SQLite. + +That is a gap with an engine-side answer when it is wanted: Turso mints **read-only tokens** +(`turso db tokens create --read-only`) and the API can `block_writes` on a database. Both are +credentials the user creates, not statements this provider can issue, which is why the profile is +absent rather than faked. + +### 3.7 `notnull` must be quoted, and only a live server says so + +```sql +SELECT cid, name, type, notnull, dflt_value, pk FROM pragma_table_info('probe_customers') +-- SQL string could not be parsed: near NOTNULL, "None": syntax error at (1, 32) +``` + +`notnull` is a SQLite keyword — the postfix `x NOTNULL` operator — so projecting it bare is a parse +error. The cost is narrow and quiet: it fails the COLUMN read of every table and leaves the rest of +the tree intact, so the object browser lists the tables and shows each as having no columns. No unit +test could catch it (a fake transport does not parse SQL), and the gate-4 live probe did. +`tests/unit/db/libsql/introspect.test.ts` now pins the statement text. + +### 3.8 A batch is one round trip, and each statement keeps its own outcome + +A libSQL server is normally across a network, and SQLite introspection is per-table: a row count, a +`pragma_table_info`, a `pragma_index_list` and a `pragma_foreign_key_list` for each. One request per +statement is four round trips per table. Hrana takes a LIST of requests, so `executeBatch` sends them +all at once — a whole schema read is three round trips regardless of table count, plus one for sizes. + +Measured, and the reason the batch hands failures back individually rather than throwing: + +``` +requests: [SELECT 1 AS a] [SELECT * FROM nope] [SELECT 3 AS c] +results: ok error ok +``` + +A failing statement does NOT abort the pipeline. Collapsing that onto one rejection is how a single +refused read costs a whole dashboard (#477, BACKLOG D22), so `LibSQLBatchOutcome` is a discriminated +per-statement result and the provider decides, per reading, what an absent one means. + +### 3.9 `dbstat` answers here, so the sizes are real + +`bun:sqlite` has no `dbstat` at all, which is why the SQLite provider's Storage tab is often empty. +Both libSQL deployments answer it, so table and index bytes here are MEASURED — 4096 bytes of table +and 4096 of index for a 3-row table, 53248 for a 2000-row one. When `dbstat` is absent the byte +fields are OMITTED rather than zeroed: 0 B reads as an empty table, which is a claim. + +### 3.10 The version panel names what the deployment publishes + +`GET /version` is a sqld route that Turso Cloud does not have (`{"error":"route not found: +[\"version\"]"}`). So `serverVersion()` answers `null` there rather than throwing, and the panel +reads: + +- self-hosted: `sqld 0.24.33 (f8fb14f3 2026-08-11) (SQLite 3.47.0)` +- Turso Cloud: `SQLite 3.47.0` + +Neither is "Unknown", because in both cases the engine answered something. + +### 3.11 No sessions, no uptime, no connection ceiling + +Hrana is stateless: a statement is a request. There is no session object anywhere, so +`getActiveSessions()` answers `[]` and `HealthInfo.activeSessions` is empty — a row for the request in +flight would be the provider describing itself. `maxConnections` is `0`, this codebase's encoding for +"no limit published" (the same as Trino, Druid and MSSQL), and `uptime` is `"N/A"` because no route +or catalog publishes one. + +`getSlowQueries()` answers `[]` for a reason of the engine's: libSQL keeps no statistics about +finished statements. The empty state says exactly that instead of naming a PostgreSQL extension. + +### 3.12 The dialect facts were re-measured, not inherited + +The grammar registry maps `libsql` to the SAME `SQLITE_GRAMMAR` object, and every one of its four +facts was re-measured over Hrana rather than assumed: + +| Fact | Measured on sqld 0.24.33 | +|---|---| +| `#` starts a comment | **No** — `SELECT 1 # x` is "bad variable name" | +| `[…]` quotes an identifier | **Yes** — `SELECT [id] FROM probe_customers` parses | +| Block comments nest | **No** — `/* outer /* inner */ SELECT 1` runs | +| `q'…'` is a literal | **No** — syntax error | +| `''` escapes a quote | **Yes** — `SELECT 'it''s'` answers `it's` | +| `hex(X'0102deadbeef')` | `0102DEADBEEF`; `typeof(X'')` is `blob`, `length(X'')` is 0 | + +--- + +## 4. Connection + +### 4.1 Configuration fields + +| Field | Required | Meaning | +|---|---|---| +| `host` | yes (or a URL) | `libredb-probe.turso.io`, or the host running `sqld` | +| `port` | no | Defaults to `8080` plaintext, `443` under TLS | +| `password` | when the server requires one | The auth TOKEN, sent as `Authorization: Bearer` | +| `ssl` | no | Any mode other than `disable` selects HTTPS | +| `connectionString` | no | A `libsql://` URL, resolved into the fields above | + +There is no `user` (libSQL has no user names) and no `database` (the database is the host). + +### 4.2 Connection strings + +``` +libsql://-.turso.io?authToken= +``` + +That is the URL `turso db show --url` and the dashboard print. `libsql://` implies TLS and 443, and +there is no plaintext form of the scheme: a self-hosted server on plain HTTP is reached through the +host/port fields with TLS off, because `http://` already resolves to ClickHouse in +`src/lib/connection-string-parser.ts` and two engines cannot own one scheme. + +### 4.3 Getting a token + +```bash +turso db tokens create # full access +turso db tokens create --read-only # read-only, the engine-side answer to §3.6 +``` + +A self-hosted `sqld` started without authentication takes no token at all, and sending an empty one +is a 400 rather than an anonymous connection — so a connection with no token sends no header. + +--- + +## 5. Query interface + +`query(sql, params)` sends one statement and returns `{ rows, fields, rowCount, executionTime, +columnTypes? }`. + +- **Parameters are positional** (`?`), encoded per type: an integer as a decimal string, a + non-integral number as a float, a boolean as SQLite's own 1/0, `Uint8Array` as base64, a `Date` as + an ISO string. Passing none sends no `args` member at all. +- **`rowCount`** is the row count for a read and the engine's `affected_row_count` for a write. +- **`columnTypes`** carries SQLite's declared types verbatim (`INTEGER`, `TEXT`) and is OMITTED when + the engine declared none — which it does for every computed column and every PRAGMA, so an absent + map is the common case rather than a failure. +- **`executionTime`** is the engine's own measurement when it rounds to at least a millisecond, and + the wall-clock one otherwise. + +### EXPLAIN + +`EXPLAIN QUERY PLAN` answers the four-column `id/parent/notused/detail` shape, read by the shared +`sqlite-queryplan` strategy. Measured against the probe fixture: + +``` +SEARCH probe_customers USING INDEX idx_customers_country (country=?) +``` + +--- + +## 6. Schema introspection + +| Surface | Statement | +|---|---| +| Tables | `SELECT name FROM sqlite_master WHERE type = 'table' AND name NOT LIKE 'sqlite_%'` | +| Row count | `SELECT COUNT(*) AS row_count FROM ""` | +| Columns | `SELECT cid, name, type, "notnull", dflt_value, pk FROM pragma_table_info('
')` | +| Indexes | `SELECT seq, name, "unique", origin FROM pragma_index_list('
')` | +| Index columns | `SELECT seqno, cid, name FROM pragma_index_info('')` | +| Foreign keys | `SELECT id, seq, "table", "from", "to" FROM pragma_foreign_key_list('
')` | +| Bytes | `SELECT name, SUM(pgsize) AS bytes FROM dbstat GROUP BY name` | + +`sqlite_autoindex_*` entries are dropped: no user declared them and no user can drop them, so listing +them reports objects the schema does not contain. + +Degradation is per reading, not per tree: + +| What failed | What the user sees | +|---|---| +| The table list | The read fails — there is nothing to degrade to | +| One table's columns | That table listed with no columns; every other table intact | +| One table's row count | No count for it; the others keep theirs | +| `dbstat` | No sizes anywhere; every row count still real | + +--- + +## 7. Monitoring & health + +Measured through the provider against both deployments (fixture: 2 tables, 3 and 2000 rows, 1 index): + +| Panel | Reading | +|---|---| +| Version | `sqld 0.24.33 (f8fb14f3 2026-08-11) (SQLite 3.47.0)` / `SQLite 3.47.0` on Turso Cloud | +| Database size | `64 KB` (65536 bytes), from `page_count × page_size` | +| Tables / indexes | 2 / 1 | +| Uptime | `N/A` — libSQL publishes none | +| Max connections | `0` — no limit published | +| Active connections | absent — Hrana is stateless | +| Cache hit ratio | `N/A` — SQLite's counters are behind the C API, and no statement reaches them | +| Health rows | `Integrity: OK`, `Journal Mode: wal` | +| Slow queries | Empty, permanently: libSQL keeps no statement statistics | +| Active sessions | Empty: no session exists to report | +| Deadlocks | `0`, and it is a fact rather than a gap — SQLite serializes writers behind one write lock and refuses a second with `SQLITE_BUSY` | +| Table stats | `probe_customers` 3 rows, 4096 B table + 4096 B index; `probe_orders` 2000 rows, 53248 B | +| Index stats | `idx_customers_country`, columns `[country]`, 4096 B, `scans: 0` (no per-index counter exists) | +| Storage | One entry, `main`, 65536 B. No WAL size: no statement reports it | + +--- + +## 8. Maintenance + +| Operation | Offered | Statement | +|---|---|---| +| `reindex` | per table and globally | `REINDEX` / `REINDEX "
"` | +| `check` | globally | `PRAGMA integrity_check`, and the ANSWER is read — a corrupt database reports damage in its row while the statement itself succeeds | +| `vacuum`, `analyze`, `optimize`, `kill` | withheld | Refused by the server (§3.5); a direct API call is refused by the provider with the reason | + +--- + +## 9. Testing + +```bash +# Just this provider +bun test tests/unit/db/libsql tests/integration/db/libsql-provider.test.ts + +# The live server the tests were written against +docker compose -f database-compose.yml up -d libsql # sqld on localhost:18080 +``` + +`globalThis.fetch` is replaced per test and restored afterwards; `mock.module()` is refused, being +process-wide in bun. Every payload in the tests was captured from the two live deployments. + +### Verifying against a live server + +```bash +docker compose -f database-compose.yml up -d libsql +# then point a Studio connection at 127.0.0.1:18080 with TLS off and no token +``` + +For Turso Cloud, create a database and a token with the `turso` CLI and paste the +`libsql://…?authToken=…` URL into the connection dialog. + +--- + +## 10. Known limitations + +| Limitation | Cause | Owner | +|---|---|---| +| No transaction controls | The provider closes its Hrana stream with each statement, so it holds no session | Ours. Hrana's `baton` is exactly the feature that would carry one | +| No agent read-only profile | `PRAGMA query_only` is refused by the server | The engine's; a read-only Turso token is its answer | +| No Vacuum, Analyze or Optimize | Refused by the server's statement allowlist | The engine's | +| No slow queries, no sessions, no uptime, no cache ratio | libSQL publishes none of them | The engine's | +| No WAL size on the Storage tab | No statement reports it, and `PRAGMA wal_checkpoint` is refused | The engine's | +| Turso Database (the Rust engine) is not reachable | It publishes no server image and ships in-process | Revisit when a server image exists | + +--- + +## 11. References + +- Turso documentation — +- libSQL — +- Turso Database (the Rust engine) — +- Hrana protocol specification — +- [`docs/ADDING_A_PROVIDER.md`](../ADDING_A_PROVIDER.md) — the registration checklist this provider followed +- [`docs/providers/sqlite.md`](sqlite.md) — the same dialect against a file diff --git a/e2e/libsql-provider.spec.ts b/e2e/libsql-provider.spec.ts new file mode 100644 index 00000000..c72a57fe --- /dev/null +++ b/e2e/libsql-provider.spec.ts @@ -0,0 +1,74 @@ +import { test, expect } from "@playwright/test"; + +/** + * Gate 5 for issue #424 Phase 5. + * + * The unit and integration tests drive the provider through a fake transport, so they + * prove the code is right and prove nothing about whether a user can reach it. + * Everything a new type-id needs in order to be SELECTABLE lives outside the provider + * - the `DatabaseType` union, `DB_UI_CONFIG`, `selectableTypes`, an icon export - and + * a miss in any of them leaves a provider that works and cannot be picked. Several of + * those surfaces are not type-enforced, which is exactly why this runs in a browser. + * + * The assertion specific to THIS engine, and that no other spec here makes, is the + * credential's NAME: libSQL has no user names, the server checks a token it minted, + * and a field labelled Password invites a password no libSQL server has. + */ +test.describe("libSQL in the connection dialog", () => { + test.beforeEach(async ({ page }) => { + await page.goto("/login"); + await page.locator('input[type="email"]').fill("user@libredb.org"); + await page.locator('input[type="password"]').fill("test-user"); + await page.getByRole("button", { name: "Sign In" }).click(); + await page.waitForURL("/"); + await expect(page.locator("text=Query 1").first()).toBeVisible({ timeout: 10000 }); + + const sidebarButtons = page.locator("text=LibreDB Studio").locator("..").locator("..").locator("button"); + await sidebarButtons.last().click(); + await expect(page.locator('[role="dialog"]')).toBeVisible({ timeout: 5000 }); + }); + + test("is offered as its own driver", async ({ page }) => { + await expect(page.locator('[role="dialog"]').getByRole("button", { name: "libSQL", exact: true })).toBeVisible(); + }); + + test("prefills sqld's own port and offers the URL Turso prints", async ({ page }) => { + const dialog = page.locator('[role="dialog"]'); + await dialog.getByRole("button", { name: "libSQL", exact: true }).click(); + + await expect(dialog.locator('input[value="8080"]')).toBeVisible(); + // `libsql://-.turso.io?authToken=` is a real string a user has + // in front of them, unlike Trino's JDBC URL - so the toggle IS offered here. + await expect(dialog.getByText("Connection String", { exact: true }).first()).toBeVisible(); + }); + + test("asks for an auth token rather than a password, and says where one comes from", async ({ page }) => { + const dialog = page.locator('[role="dialog"]'); + await dialog.getByRole("button", { name: "libSQL", exact: true }).click(); + + await expect(dialog.getByText("Auth Token", { exact: true })).toBeVisible(); + await expect(dialog.getByText("Password", { exact: true })).toHaveCount(0); + await expect(dialog.getByText(/turso db tokens create/)).toBeVisible(); + }); + + test("offers no user field, because libSQL has no user names", async ({ page }) => { + const dialog = page.locator('[role="dialog"]'); + await dialog.getByRole("button", { name: "libSQL", exact: true }).click(); + + await expect(dialog.locator("#user")).toHaveCount(0); + // No database field either: on Turso Cloud the database IS the hostname, and a + // self-hosted server serves one per namespace hostname. + await expect(dialog.locator("#database")).toHaveCount(0); + }); + + test("claims no wire-compatible relative it has not probed", async ({ page }) => { + // The Phase 0 hint renders from the compatibility registry, which has no entry for + // libSQL: Turso Database - the Rust rewrite - publishes no server image, so it has + // never been connected to. A hint appearing here would mean a name was published + // without a gate-4 probe, which is the overclaim #424 exists to prevent. + const dialog = page.locator('[role="dialog"]'); + await dialog.getByRole("button", { name: "libSQL", exact: true }).click(); + + await expect(dialog.getByTestId("wire-compat-hint")).toHaveCount(0); + }); +}); diff --git a/operator/helm-charts/libredb-studio/Chart.yaml b/operator/helm-charts/libredb-studio/Chart.yaml index dab7df27..81881b68 100644 --- a/operator/helm-charts/libredb-studio/Chart.yaml +++ b/operator/helm-charts/libredb-studio/Chart.yaml @@ -2,7 +2,7 @@ apiVersion: v2 name: libredb-studio description: Web-based SQL IDE for cloud-native teams supporting fourteen engines - PostgreSQL, MySQL, SQLite, Oracle, SQL Server, MongoDB, Redis, Couchbase, ClickHouse, Apache Druid, Elasticsearch, OpenSearch, Apache Trino and Apache Cassandra type: application -version: 0.1.52 +version: 0.1.53 appVersion: "0.13.4" kubeVersion: ">=1.26.0-0" home: https://github.com/libredb/libredb-studio @@ -39,6 +39,10 @@ annotations: artifacthub.io/category: database artifacthub.io/license: MIT artifacthub.io/prerelease: "false" + # False: 0.1.53 changes no packaged template and no value - it names one more engine in + # the README, libSQL, which is a new provider in the app rather than a chart change. The + # README is a packaged file, so #167 costs it a chart version even though nothing an + # operator deploys moves. # False: 0.1.52 adds one value, config.authCookieSecure, and changes no behaviour on its # own - unset (the default) writes no AUTH_COOKIE_SECURE and the app keeps deciding, so # every existing install renders exactly as before. It makes an already-supported setting diff --git a/operator/helm-charts/libredb-studio/README.md b/operator/helm-charts/libredb-studio/README.md index be20978c..c88d45d5 100644 --- a/operator/helm-charts/libredb-studio/README.md +++ b/operator/helm-charts/libredb-studio/README.md @@ -2,7 +2,7 @@ [![Artifact Hub](https://img.shields.io/endpoint?url=https://artifacthub.io/badge/repository/libredb-studio)](https://artifacthub.io/packages/search?repo=libredb-studio) -Web-based SQL IDE for cloud-native teams supporting fourteen engines - PostgreSQL, MySQL, SQLite, Oracle, SQL Server, MongoDB, Redis, Couchbase, ClickHouse, Apache Druid, Elasticsearch, OpenSearch, Apache Trino and Apache Cassandra. +Web-based SQL IDE for cloud-native teams supporting fifteen engines - PostgreSQL, MySQL, SQLite, libSQL, Oracle, SQL Server, MongoDB, Redis, Couchbase, ClickHouse, Apache Druid, Elasticsearch, OpenSearch, Apache Trino and Apache Cassandra. ## Prerequisites @@ -40,7 +40,7 @@ helm install libredb libredb/libredb-studio \ ```bash helm install libredb oci://ghcr.io/libredb/charts/libredb-studio \ - --version 0.1.52 \ + --version 0.1.53 \ --set secrets.jwtSecret=$(openssl rand -base64 32) \ --set secrets.adminPassword=MyAdmin123 ``` diff --git a/probe-libsql.ts b/probe-libsql.ts new file mode 100644 index 00000000..3c571541 --- /dev/null +++ b/probe-libsql.ts @@ -0,0 +1,58 @@ +/** Gate-4 harness: every surface, called separately, against a live libSQL. */ +import { LibSQLProvider } from "./src/lib/db/providers/sql/libsql"; +import type { DatabaseConnection } from "./src/lib/db/types"; + +const [, , label, host, portRaw, token] = process.argv; +const connection = { + id: `probe-${label}`, + name: `libSQL ${label}`, + type: "libsql", + host, + ...(portRaw === "-" ? {} : { port: Number(portRaw) }), + ...(token && token !== "-" ? { password: token } : {}), + ...(portRaw === "-" ? { ssl: { mode: "require" } } : {}), + createdAt: new Date(), +} as unknown as DatabaseConnection; + +const provider = new LibSQLProvider(connection); +const results: Record = {}; + +async function surface(name: string, run: () => Promise): Promise { + try { + results[name] = { ok: true, value: await run() }; + } catch (error) { + results[name] = { ok: false, error: error instanceof Error ? error.message : String(error) }; + } +} + +await surface("connect", async () => { + await provider.connect(); + return "connected"; +}); +await surface("query", () => provider.query("SELECT id, name, country FROM probe_customers ORDER BY id")); +await surface("queryParams", () => provider.query("SELECT COUNT(*) AS c FROM probe_orders WHERE customer_id = ?", [1])); +await surface("explain", () => provider.query("EXPLAIN QUERY PLAN SELECT * FROM probe_customers WHERE country = 'tr'")); +await surface("write", () => provider.query("UPDATE probe_customers SET country = 'tr' WHERE id = 1")); +await surface("schema", () => provider.getSchema()); +await surface("overview", () => provider.getOverview()); +await surface("health", () => provider.getHealth()); +await surface("performance", () => provider.getPerformanceMetrics()); +await surface("slowQueries", () => provider.getSlowQueries()); +await surface("activeSessions", () => provider.getActiveSessions()); +await surface("tableStats", () => provider.getTableStats()); +await surface("indexStats", () => provider.getIndexStats()); +await surface("storageStats", () => provider.getStorageStats()); +await surface("maintenanceCheck", () => provider.runMaintenance("check")); +await surface("maintenanceReindex", () => provider.runMaintenance("reindex")); +await surface("maintenanceVacuum", () => provider.runMaintenance("vacuum")); +await surface("badStatement", () => provider.query("SELECT * FROM no_such_table")); +await surface("disconnect", async () => { + await provider.disconnect(); + return "disconnected"; +}); + +await Bun.write(`probe-results-${label}.json`, JSON.stringify(results, null, 2)); +for (const [name, outcome] of Object.entries(results)) { + const record = outcome as { ok: boolean; error?: string }; + console.log(record.ok ? "OK " : "ERR ", name.padEnd(20), record.ok ? "" : record.error); +} diff --git a/probe-results-cloud.json b/probe-results-cloud.json new file mode 100644 index 00000000..c5c7727a --- /dev/null +++ b/probe-results-cloud.json @@ -0,0 +1,294 @@ +{ + "connect": { + "ok": true, + "value": "connected" + }, + "query": { + "ok": true, + "value": { + "rows": [ + { + "id": 1, + "name": "Ada", + "country": "tr" + }, + { + "id": 2, + "name": "Linus", + "country": "de" + }, + { + "id": 3, + "name": "Grace", + "country": "us" + } + ], + "fields": [ + "id", + "name", + "country" + ], + "rowCount": 3, + "executionTime": 64, + "columnTypes": { + "id": "INTEGER", + "name": "TEXT", + "country": "TEXT" + } + } + }, + "queryParams": { + "ok": true, + "value": { + "rows": [ + { + "c": 666 + } + ], + "fields": [ + "c" + ], + "rowCount": 1, + "executionTime": 67 + } + }, + "explain": { + "ok": true, + "value": { + "rows": [ + { + "id": 3, + "parent": 0, + "notused": 62, + "detail": "SEARCH probe_customers USING INDEX idx_customers_country (country=?)" + } + ], + "fields": [ + "id", + "parent", + "notused", + "detail" + ], + "rowCount": 1, + "executionTime": 98 + } + }, + "write": { + "ok": true, + "value": { + "rows": [], + "fields": [], + "rowCount": 1, + "executionTime": 72 + } + }, + "schema": { + "ok": true, + "value": [ + { + "name": "probe_customers", + "rowCount": 3, + "size": "8 KB", + "columns": [ + { + "name": "id", + "type": "INTEGER", + "nullable": true, + "isPrimary": true + }, + { + "name": "name", + "type": "TEXT", + "nullable": false, + "isPrimary": false + }, + { + "name": "country", + "type": "TEXT", + "nullable": true, + "isPrimary": false, + "defaultValue": "'tr'" + } + ], + "indexes": [ + { + "name": "idx_customers_country", + "columns": [ + "country" + ], + "unique": false + } + ], + "foreignKeys": [] + }, + { + "name": "probe_orders", + "rowCount": 2000, + "size": "52 KB", + "columns": [ + { + "name": "id", + "type": "INTEGER", + "nullable": true, + "isPrimary": true + }, + { + "name": "customer_id", + "type": "INTEGER", + "nullable": true, + "isPrimary": false + }, + { + "name": "amount", + "type": "REAL", + "nullable": true, + "isPrimary": false + }, + { + "name": "note", + "type": "TEXT", + "nullable": true, + "isPrimary": false + } + ], + "indexes": [], + "foreignKeys": [ + { + "columnName": "customer_id", + "referencedTable": "probe_customers", + "referencedColumn": "id" + } + ] + } + ] + }, + "overview": { + "ok": true, + "value": { + "version": "SQLite 3.47.0", + "uptime": "N/A", + "maxConnections": 0, + "databaseSize": "64 KB", + "databaseSizeBytes": 65536, + "tableCount": 2, + "indexCount": 1 + } + }, + "health": { + "ok": true, + "value": { + "databaseSize": "64 KB", + "cacheHitRatio": "N/A", + "slowQueries": [ + { + "query": "Integrity: OK", + "calls": 0, + "avgTime": "N/A" + }, + { + "query": "Journal Mode: wal", + "calls": 0, + "avgTime": "N/A" + } + ], + "activeSessions": [] + } + }, + "performance": { + "ok": true, + "value": { + "deadlocks": 0 + } + }, + "slowQueries": { + "ok": true, + "value": [] + }, + "activeSessions": { + "ok": true, + "value": [] + }, + "tableStats": { + "ok": true, + "value": [ + { + "schemaName": "main", + "tableName": "probe_customers", + "rowCount": 3, + "tableSize": "4 KB", + "tableSizeBytes": 4096, + "indexSize": "4 KB", + "indexSizeBytes": 4096, + "totalSize": "8 KB", + "totalSizeBytes": 8192 + }, + { + "schemaName": "main", + "tableName": "probe_orders", + "rowCount": 2000, + "tableSize": "52 KB", + "tableSizeBytes": 53248, + "indexSize": "0 B", + "indexSizeBytes": 0, + "totalSize": "52 KB", + "totalSizeBytes": 53248 + } + ] + }, + "indexStats": { + "ok": true, + "value": [ + { + "schemaName": "main", + "tableName": "probe_customers", + "indexName": "idx_customers_country", + "columns": [ + "country" + ], + "isUnique": false, + "isPrimary": false, + "indexSize": "4 KB", + "indexSizeBytes": 4096, + "scans": 0 + } + ] + }, + "storageStats": { + "ok": true, + "value": [ + { + "name": "main", + "size": "64 KB", + "sizeBytes": 65536 + } + ] + }, + "maintenanceCheck": { + "ok": true, + "value": { + "success": true, + "executionTime": 67, + "message": "ok" + } + }, + "maintenanceReindex": { + "ok": true, + "value": { + "success": true, + "executionTime": 84, + "message": "REINDEX completed successfully" + } + }, + "maintenanceVacuum": { + "ok": false, + "error": "libSQL servers do not accept VACUUM: only REINDEX and PRAGMA integrity_check are allowed" + }, + "badStatement": { + "ok": false, + "error": "SQLite error: no such table: no_such_table" + }, + "disconnect": { + "ok": true, + "value": "disconnected" + } +} \ No newline at end of file diff --git a/probe-results-self.json b/probe-results-self.json new file mode 100644 index 00000000..c792b262 --- /dev/null +++ b/probe-results-self.json @@ -0,0 +1,294 @@ +{ + "connect": { + "ok": true, + "value": "connected" + }, + "query": { + "ok": true, + "value": { + "rows": [ + { + "id": 1, + "name": "Ada", + "country": "tr" + }, + { + "id": 2, + "name": "Linus", + "country": "de" + }, + { + "id": 3, + "name": "Grace", + "country": "us" + } + ], + "fields": [ + "id", + "name", + "country" + ], + "rowCount": 3, + "executionTime": 1, + "columnTypes": { + "id": "INTEGER", + "name": "TEXT", + "country": "TEXT" + } + } + }, + "queryParams": { + "ok": true, + "value": { + "rows": [ + { + "c": 666 + } + ], + "fields": [ + "c" + ], + "rowCount": 1, + "executionTime": 1 + } + }, + "explain": { + "ok": true, + "value": { + "rows": [ + { + "id": 3, + "parent": 0, + "notused": 62, + "detail": "SEARCH probe_customers USING INDEX idx_customers_country (country=?)" + } + ], + "fields": [ + "id", + "parent", + "notused", + "detail" + ], + "rowCount": 1, + "executionTime": 1 + } + }, + "write": { + "ok": true, + "value": { + "rows": [], + "fields": [], + "rowCount": 1, + "executionTime": 4 + } + }, + "schema": { + "ok": true, + "value": [ + { + "name": "probe_customers", + "rowCount": 3, + "size": "8 KB", + "columns": [ + { + "name": "id", + "type": "INTEGER", + "nullable": true, + "isPrimary": true + }, + { + "name": "name", + "type": "TEXT", + "nullable": false, + "isPrimary": false + }, + { + "name": "country", + "type": "TEXT", + "nullable": true, + "isPrimary": false, + "defaultValue": "'tr'" + } + ], + "indexes": [ + { + "name": "idx_customers_country", + "columns": [ + "country" + ], + "unique": false + } + ], + "foreignKeys": [] + }, + { + "name": "probe_orders", + "rowCount": 2000, + "size": "52 KB", + "columns": [ + { + "name": "id", + "type": "INTEGER", + "nullable": true, + "isPrimary": true + }, + { + "name": "customer_id", + "type": "INTEGER", + "nullable": true, + "isPrimary": false + }, + { + "name": "amount", + "type": "REAL", + "nullable": true, + "isPrimary": false + }, + { + "name": "note", + "type": "TEXT", + "nullable": true, + "isPrimary": false + } + ], + "indexes": [], + "foreignKeys": [ + { + "columnName": "customer_id", + "referencedTable": "probe_customers", + "referencedColumn": "id" + } + ] + } + ] + }, + "overview": { + "ok": true, + "value": { + "version": "sqld 0.24.33 (f8fb14f3 2026-08-11) (SQLite 3.47.0)", + "uptime": "N/A", + "maxConnections": 0, + "databaseSize": "64 KB", + "databaseSizeBytes": 65536, + "tableCount": 2, + "indexCount": 1 + } + }, + "health": { + "ok": true, + "value": { + "databaseSize": "64 KB", + "cacheHitRatio": "N/A", + "slowQueries": [ + { + "query": "Integrity: OK", + "calls": 0, + "avgTime": "N/A" + }, + { + "query": "Journal Mode: wal", + "calls": 0, + "avgTime": "N/A" + } + ], + "activeSessions": [] + } + }, + "performance": { + "ok": true, + "value": { + "deadlocks": 0 + } + }, + "slowQueries": { + "ok": true, + "value": [] + }, + "activeSessions": { + "ok": true, + "value": [] + }, + "tableStats": { + "ok": true, + "value": [ + { + "schemaName": "main", + "tableName": "probe_customers", + "rowCount": 3, + "tableSize": "4 KB", + "tableSizeBytes": 4096, + "indexSize": "4 KB", + "indexSizeBytes": 4096, + "totalSize": "8 KB", + "totalSizeBytes": 8192 + }, + { + "schemaName": "main", + "tableName": "probe_orders", + "rowCount": 2000, + "tableSize": "52 KB", + "tableSizeBytes": 53248, + "indexSize": "0 B", + "indexSizeBytes": 0, + "totalSize": "52 KB", + "totalSizeBytes": 53248 + } + ] + }, + "indexStats": { + "ok": true, + "value": [ + { + "schemaName": "main", + "tableName": "probe_customers", + "indexName": "idx_customers_country", + "columns": [ + "country" + ], + "isUnique": false, + "isPrimary": false, + "indexSize": "4 KB", + "indexSizeBytes": 4096, + "scans": 0 + } + ] + }, + "storageStats": { + "ok": true, + "value": [ + { + "name": "main", + "size": "64 KB", + "sizeBytes": 65536 + } + ] + }, + "maintenanceCheck": { + "ok": true, + "value": { + "success": true, + "executionTime": 1, + "message": "ok" + } + }, + "maintenanceReindex": { + "ok": true, + "value": { + "success": true, + "executionTime": 4, + "message": "REINDEX completed successfully" + } + }, + "maintenanceVacuum": { + "ok": false, + "error": "libSQL servers do not accept VACUUM: only REINDEX and PRAGMA integrity_check are allowed" + }, + "badStatement": { + "ok": false, + "error": "SQLite error: no such table: no_such_table" + }, + "disconnect": { + "ok": true, + "value": "disconnected" + } +} \ No newline at end of file diff --git a/src/components/ConnectionModal.tsx b/src/components/ConnectionModal.tsx index 04947765..7c2fa421 100644 --- a/src/components/ConnectionModal.tsx +++ b/src/components/ConnectionModal.tsx @@ -178,11 +178,19 @@ export function ConnectionModal({ // So the ordinary deployment - users in `admin`, data elsewhere - had no way through // the discrete fields at all, and failed as a credentials error. const isMongoDB = type === "mongodb"; + // libSQL has no user names at all: the credential a server checks is a TOKEN it + // minted (Turso prints one per database), so the shared `password` field holds a + // JWT here. A field labelled Password invites a password no libSQL server has, + // which is why this one is relabelled rather than left to be guessed at. + const isLibSQL = type === "libsql"; + const passwordFieldLabel = isLibSQL ? "Auth Token" : "Password"; const databaseFieldLabel = isCouchbase ? "Bucket" : isTrino ? "Catalog" : isCassandra ? "Keyspace" : "Database"; const databaseFieldPlaceholder = isTrino ? "tpch" : isCassandra ? "probe" : "db"; const connectionUriPlaceholder = isCouchbase ? "couchbase://localhost:8091/travel-sample or couchbases://cb..cloud.couchbase.com/..." - : "mongodb://localhost:27017/mydb or mongodb+srv://..."; + : isLibSQL + ? "libsql://-.turso.io?authToken=" + : "mongodb://localhost:27017/mydb or mongodb+srv://..."; const formContent = ( <> @@ -465,7 +473,7 @@ export function ConnectionModal({
+ Turso Cloud mints this per database (`turso db tokens create`). A self-hosted libSQL server + started without authentication takes none - leave it empty. +

+ )} {isTrino && (

Trino refuses a password over plain HTTP. Enable TLS below, or leave this empty to connect as diff --git a/src/components/icons/db-icons.tsx b/src/components/icons/db-icons.tsx index ca3f9a9d..e58f46ba 100644 --- a/src/components/icons/db-icons.tsx +++ b/src/components/icons/db-icons.tsx @@ -359,3 +359,32 @@ export const CassandraIcon: React.FC = ({ className, ...props }) => ( ); + +/** + * libSQL: SQLite's file, reached across a network. + * + * Deliberately not a copy of either project's logo. What distinguishes this engine + * from the SQLite icon above is the whole point of the id - the same store, with a + * server in front of it - so the mark is the document outline that icon uses with a + * connecting arc and an endpoint drawn to it. Single stroke at weight 1.5, like + * every other mark here, and it still reads at 14px. + */ +export const LibSQLIcon: React.FC = ({ className, ...props }) => ( + + + + + + + + +); diff --git a/src/hooks/use-connection-form.ts b/src/hooks/use-connection-form.ts index c6878ef4..004bb3e2 100644 --- a/src/hooks/use-connection-form.ts +++ b/src/hooks/use-connection-form.ts @@ -592,6 +592,7 @@ export function useConnectionForm({ isOpen, onConnect, editConnection, onTestCon "opensearch", "trino", "cassandra", + "libsql", ]; const dbTypes = selectableTypes.map((t) => { const cfg = getDBConfig(t); diff --git a/src/lib/connection-string-parser.ts b/src/lib/connection-string-parser.ts index ba291933..c9c6c622 100644 --- a/src/lib/connection-string-parser.ts +++ b/src/lib/connection-string-parser.ts @@ -85,6 +85,7 @@ export const ENGINE_URI_SCHEMES: Partial> = { mssql: "mssql", couchbase: "couchbase", clickhouse: "clickhouse", + libsql: "libsql", }; /** @@ -162,6 +163,16 @@ export function parseConnectionString(input: string): ParsedConnection | null { // clickhouse:// names no transport -> say nothing, defer to the form // http:// explicitly plaintext -> disable, or a stale "require" survives // https:// explicitly TLS -> require, or a Cloud endpoint gets plain HTTP + // libSQL. TLS always, and the token rides in the query string rather than in the + // authority: `libsql://-.turso.io?authToken=` is what Turso's + // own CLI prints, and `libsql://` has no plaintext form - a self-hosted server on + // plain HTTP is reached through the host/port fields with TLS off, because + // `http://` already resolves to ClickHouse here and two engines cannot own one + // scheme. + if (trimmed.startsWith("libsql://")) { + return parseLibSQLURL(trimmed); + } + if (trimmed.startsWith("clickhouse://")) { return parseGenericURL(trimmed, "clickhouse", "8123"); } @@ -508,6 +519,31 @@ function parseADONetString(input: string): ParsedConnection | null { } } +/** + * A `libsql://` URL as a connection. + * + * Three things differ from `parseGenericURL` and each is measured against what Turso + * emits: the credential is `?authToken=`, so the URL's own password slot is only a + * fallback; there is no database in the path, because the database IS the hostname; + * and the default port is 443 under required TLS, which is how Turso Cloud serves + * every database. + */ +function parseLibSQLURL(uri: string): ParsedConnection | null { + try { + const url = new URL(uri); + + return { + type: "libsql", + host: url.hostname || "localhost", + port: url.port || "443", + password: url.searchParams.get("authToken") ?? (url.password ? decodeURIComponent(url.password) : undefined), + sslMode: "require", + }; + } catch { + return null; + } +} + function parseGenericURL(uri: string, type: DatabaseType, defaultPort: string): ParsedConnection | null { try { const url = new URL(uri); @@ -538,6 +574,7 @@ export function detectConnectionStringType(input: string): DatabaseType | null { if (trimmed.startsWith("oracle://")) return "oracle"; if (trimmed.startsWith("mssql://") || trimmed.startsWith("sqlserver://")) return "mssql"; if (trimmed.startsWith("couchbase://") || trimmed.startsWith("couchbases://")) return "couchbase"; + if (trimmed.startsWith("libsql://")) return "libsql"; if (trimmed.startsWith("clickhouse://") || trimmed.startsWith("http://") || trimmed.startsWith("https://")) return "clickhouse"; if (/^server\s*=/i.test(trimmed)) return "mssql"; diff --git a/src/lib/db-showcase.ts b/src/lib/db-showcase.ts index 2aff573f..9adbab09 100644 --- a/src/lib/db-showcase.ts +++ b/src/lib/db-showcase.ts @@ -41,11 +41,15 @@ export const SHOWCASE_RANK: Record = { // it is the engine a data platform is usually met THROUGH rather than one more // store to choose between. trino: 13, + // Behind Trino and ahead of the embedded store: libSQL is the newest name on this + // page and the one an evaluator is least likely to have met, but it is a product + // name (Turso's server) rather than our own, so it goes ahead of `libredb`. + libsql: 14, // Last on purpose: the embedded store is the least recognisable name here. It is // still shown - it is a shipped provider with a doc (docs/providers/libredb.md), an // icon and a slot in the connection picker, so omitting it would make the login page // contradict the app (issue #425, step 2). - libredb: 14, + libredb: 15, }; /** diff --git a/src/lib/db-ui-config.ts b/src/lib/db-ui-config.ts index 4b92025d..76a2318a 100644 --- a/src/lib/db-ui-config.ts +++ b/src/lib/db-ui-config.ts @@ -15,6 +15,7 @@ import { OpenSearchIcon, TrinoIcon, CassandraIcon, + LibSQLIcon, } from "@/components/icons/db-icons"; import type { DatabaseType } from "@/lib/types"; @@ -71,6 +72,30 @@ export const DB_UI_CONFIG: Record = { showConnectionStringToggle: false, connectionFields: ["database"], }, + libsql: { + icon: LibSQLIcon, + // Turso's own mark is a bright mint green (#4FF8D2). emerald-300 is the nearest + // free shade - emerald-400 is MongoDB's, both teals are taken by Couchbase and + // Elasticsearch, and the distinct-colour assertion in + // tests/unit/lib/db-ui-config.test.ts rules a duplicate out. + color: "text-emerald-300", + // The protocol's name rather than the product's: one connection here reaches a + // self-hosted libSQL server OR Turso Cloud, and naming the managed product would + // read as though the self-hosted one belonged somewhere else. + label: "libSQL", + // sqld's own default HTTP port. A Turso Cloud connection names no port at all - + // it is TLS on 443, which the transport picks up from the ssl setting. + defaultPort: "8080", + // `libsql://-.turso.io?authToken=` is the URL Turso's CLI + // prints, so there IS a canonical form to paste - unlike Trino's JDBC URL. + showConnectionStringToggle: true, + // No `user`: libSQL has no user names at all, and the credential is a token the + // server mints. No `database` either - the database IS the host on Turso Cloud, + // and a self-hosted server serves one per namespace hostname. The form labels + // `password` "Auth Token" (see ConnectionModal.tsx), because a field labelled + // Password invites a password that no libSQL server has. + connectionFields: ["host", "port", "password", "connectionString"], + }, mongodb: { icon: MongoDBIcon, color: "text-emerald-400", diff --git a/src/lib/db/compatibility.ts b/src/lib/db/compatibility.ts index c0dcb194..a4620284 100644 --- a/src/lib/db/compatibility.ts +++ b/src/lib/db/compatibility.ts @@ -32,6 +32,10 @@ const SHIPPED: Readonly> = Object.freeze({ postgres: true, mysql: true, sqlite: true, + // libSQL (#424 Phase 5): its own provider, doc and integration test. A separate + // driver from `sqlite` rather than a relative of it - the two share a dialect and + // nothing else, since one holds a file handle and the other speaks HTTP. + libsql: true, oracle: true, mssql: true, clickhouse: true, @@ -85,6 +89,7 @@ const EXTERNAL: Readonly> = Object.freeze({ postgres: true, mysql: true, sqlite: true, + libsql: true, oracle: true, mssql: true, clickhouse: true, diff --git a/src/lib/db/factory.ts b/src/lib/db/factory.ts index 678a0862..e2014975 100644 --- a/src/lib/db/factory.ts +++ b/src/lib/db/factory.ts @@ -86,6 +86,11 @@ export async function createDatabaseProvider( return new SQLiteProvider(connection, options, execution); } + case "libsql": { + const { LibSQLProvider } = await import("./providers/sql/libsql"); + return new LibSQLProvider(connection, options); + } + case "oracle": { const { OracleProvider } = await import("./providers/sql/oracle"); return new OracleProvider(connection, options); @@ -173,7 +178,7 @@ export async function createDatabaseProvider( // This list is NOT type-checked against the union - a new case above with no // entry here is silent - so it is kept in the same order as the cases and // tests/unit/db/factory.test.ts pins individual names in it by regex. - `Unknown database type: ${connection.type}. Supported types: postgres, mysql, sqlite, oracle, mssql, clickhouse, druid, trino, cassandra, elasticsearch, opensearch, mongodb, couchbase, redis, libredb`, + `Unknown database type: ${connection.type}. Supported types: postgres, mysql, sqlite, libsql, oracle, mssql, clickhouse, druid, trino, cassandra, elasticsearch, opensearch, mongodb, couchbase, redis, libredb`, connection.type, ); } diff --git a/src/lib/db/providers/sql/libsql/hrana-transport.ts b/src/lib/db/providers/sql/libsql/hrana-transport.ts new file mode 100644 index 00000000..f92d338c --- /dev/null +++ b/src/lib/db/providers/sql/libsql/hrana-transport.ts @@ -0,0 +1,436 @@ +/** + * libSQL Hrana HTTP transport (issue #424 Phase 5) + * + * The only implementation of the `LibSQLTransport` seam, and the only file in the + * provider allowed to know how Hrana encodes a request or a result: its + * `/v2/pipeline` endpoint, its request/response envelope, its `baton` and its + * `{ type, value }` value encoding. `seam-guard.test.ts` fails the build the + * moment any of that vocabulary appears elsewhere in the directory, which is what + * keeps "a WebSocket or embedded implementation is one new file" true rather than + * aspirational. + * + * Zero runtime dependency: the statement is JSON in the body of a POST and the + * answer comes back through the runtime's own `fetch`. `@libsql/client` would add + * a dependency to speak a protocol that is three JSON shapes wide. + * + * Four measured shapes drive nearly every decision below (2026-08-27, against + * `ghcr.io/tursodatabase/libsql-server` running sqld 0.24.33 and against a Turso + * Cloud database, both SQLite 3.47.0), and each is the opposite of what a JSON + * API teaches: + * + * - A failed statement answers HTTP **200**. `response.ok` says the pipeline was + * accepted, never that the statement ran, so the failure is read out of + * `results[]` (the same trap Trino's provider documents). + * - An AUTH failure uses a different envelope entirely - `{"error": ""}` + * under 401 with no token and under **400** with a malformed one - so the error + * path must not assume the `{ message, code }` object shape. + * - Every integer arrives as a decimal STRING, including `last_insert_rowid`. + * That is the protocol protecting 64-bit values from a double, and reading them + * with `Number()` throws that protection away. + * - `GET /version` is a sqld route that Turso Cloud does not have. A deployment + * that publishes no version is not a broken one. + */ + +import type { DatabaseConnection } from "@/lib/db/types"; +import { + type LibSQLBatchOutcome, + type LibSQLExecuteOptions, + type LibSQLRow, + type LibSQLStatement, + type LibSQLStatementResult, + type LibSQLTransport, + LibSQLTransportError, +} from "./transport"; + +// ============================================================================ +// Constants +// ============================================================================ + +const DEFAULT_HOST = "localhost"; +/** sqld's own default HTTP port. */ +const DEFAULT_PORT = 8080; +/** + * 443, and not a libSQL-specific number: the TLS deployment this reaches is Turso + * Cloud, which serves every database on the ordinary HTTPS port of a hostname + * that identifies the database. + */ +const DEFAULT_TLS_PORT = 443; + +const PIPELINE_PATH = "/v2/pipeline"; +const VERSION_PATH = "/version"; + +const NOT_AN_ENVELOPE = "The server answered with a body that is not a libSQL answer"; +const NO_RESULT = "The server accepted the pipeline and returned no result for the statement"; + +// ============================================================================ +// Wire shapes +// ---------------------------------------------------------------------------- +// Documented in full because this file is the record of what the wire looks like. +// Members are `unknown` where the code guards them: the shape is what the server +// promised, not what a proxy in front of it is guaranteed to deliver. +// ============================================================================ + +/** A value as Hrana spells it, in either direction. */ +interface HranaValue { + type?: unknown; + value?: unknown; + base64?: unknown; +} + +interface HranaColumn { + name?: unknown; + decltype?: unknown; +} + +interface HranaStatementResult { + cols?: unknown; + rows?: unknown; + affected_row_count?: unknown; + last_insert_rowid?: unknown; + query_duration_ms?: unknown; + /** + * Declared and deliberately NOT surfaced, so this file stays the record of what + * the wire carries (the seam guard checks that every one of these is named here + * and nowhere else): + * + * - `replication_index` is the primary's frame number, and it is one of the two + * places the deployments differ - "1" on a self-hosted sqld, null on Turso + * Cloud. Nothing in the product consumes it, and widening the neutral result + * for it would force an embedded implementation to fabricate a value. + * - `rows_read` and `rows_written` are Turso's BILLING counters for the + * statement. They are honest numbers about work done, but not about the result: + * `SELECT COUNT(*)` reads 2000 rows and returns one, so surfacing either as a + * row count would report the wrong number in the one place a user checks. + */ + replication_index?: unknown; + rows_read?: unknown; + rows_written?: unknown; +} + +/** + * The pipeline envelope itself, declared for the same reason. + * + * `baton` is the handle for a server-side stream and `base_url` is where a + * continuation must be sent. Both are answered on every call and both are ignored: + * this transport closes its stream in the same request, so there is never a + * continuation to route. An interactive transaction is exactly the feature that + * would consume them, and `supportsTransactions: false` in the provider is the + * other half of that decision. + */ +interface HranaPipelineEnvelope { + baton?: unknown; + base_url?: unknown; + results?: unknown; +} + +// ============================================================================ +// Pure helpers +// ============================================================================ + +function asRecord(value: unknown): Record | null { + return typeof value === "object" && value !== null && !Array.isArray(value) + ? (value as Record) + : null; +} + +function parseJson(text: string): unknown { + try { + return JSON.parse(text) as unknown; + } catch { + return null; + } +} + +function formatHost(host: string): string { + return host.includes(":") && !host.startsWith("[") ? `[${host}]` : host; +} + +/** + * An integer the protocol sent as a decimal string, as a number where a double + * holds it exactly and as that same string where it does not. + * + * SQLite integers and rowids are 64-bit, and Hrana quotes them for exactly this + * reason. `Number("9007199254740993")` answers 9007199254740992 - a corruption + * nothing downstream can detect, which is why Trino's provider quotes wide + * integers before parsing (#460). Here the wire has already done that work, so + * the only mistake available is undoing it. + */ +function decodeInteger(raw: unknown): number | string | null { + if (typeof raw === "number") return Number.isSafeInteger(raw) ? raw : String(raw); + if (typeof raw !== "string" || raw.trim() === "") return null; + const parsed = Number(raw); + if (!Number.isFinite(parsed)) return null; + return Number.isSafeInteger(parsed) ? parsed : raw; +} + +/** Base64 as bytes. Blobs are the one SQLite type JSON cannot carry directly. */ +function decodeBlob(raw: unknown): Uint8Array | null { + if (typeof raw !== "string") return null; + return Uint8Array.from(Buffer.from(raw, "base64")); +} + +/** + * One wire value as a JavaScript one. + * + * An unrecognised `type` decodes to null rather than to its raw envelope: a + * future value type rendered as `{"type":"vector","value":…}` in a results grid + * would look like data the engine returned, and null at least reads as "nothing + * this build understands". Hrana has added types before (`blob` and `float` were + * not in the first version), so this branch is reachable by design. + */ +function decodeValue(raw: unknown): unknown { + const value = asRecord(raw); + if (!value) return null; + + switch (value.type) { + case "null": + return null; + case "integer": + return decodeInteger(value.value); + case "float": + return typeof value.value === "number" ? value.value : Number(value.value); + case "text": + return typeof value.value === "string" ? value.value : String(value.value ?? ""); + case "blob": + return decodeBlob(value.base64); + default: + return null; + } +} + +/** + * One JavaScript parameter as a wire value. + * + * `bigint` is encoded from its own decimal form rather than through `Number`, for + * the reason `decodeInteger` states in the other direction. A boolean becomes 1 + * or 0 because that is what SQLite stores - it has no boolean type - and a `Date` + * becomes an ISO 8601 string because that is the only form SQLite's own date + * functions read. + */ +function encodeValue(param: unknown): HranaValue { + if (param === null || param === undefined) return { type: "null" }; + if (typeof param === "bigint") return { type: "integer", value: param.toString() }; + if (typeof param === "boolean") return { type: "integer", value: param ? "1" : "0" }; + if (typeof param === "number") { + return Number.isInteger(param) ? { type: "integer", value: param.toString() } : { type: "float", value: param }; + } + if (param instanceof Uint8Array) return { type: "blob", base64: Buffer.from(param).toString("base64") }; + if (param instanceof Date) return { type: "text", value: param.toISOString() }; + return { type: "text", value: String(param) }; +} + +/** The column names and the types the engine declared for them. */ +function readColumns(cols: unknown): { fieldNames: string[]; columnTypes: Record } { + const fieldNames: string[] = []; + const columnTypes: Record = {}; + if (!Array.isArray(cols)) return { fieldNames, columnTypes }; + + for (const [index, raw] of cols.entries()) { + const col = (asRecord(raw) ?? {}) as HranaColumn; + // A column the engine did not name still occupies a position, so it gets a + // positional name rather than being dropped - dropping one would shift every + // later value into the wrong key. + const name = typeof col.name === "string" && col.name !== "" ? col.name : `column_${index + 1}`; + fieldNames.push(name); + if (typeof col.decltype === "string" && col.decltype !== "") columnTypes[name] = col.decltype; + } + + return { fieldNames, columnTypes }; +} + +function readRows(rows: unknown, fieldNames: string[]): LibSQLRow[] { + if (!Array.isArray(rows)) return []; + + return rows.map((raw) => { + const values = Array.isArray(raw) ? raw : []; + const row: LibSQLRow = {}; + for (const [index, name] of fieldNames.entries()) { + row[name] = decodeValue(values[index]); + } + return row; + }); +} + +function toNumber(raw: unknown, fallback: number): number { + const value = Number(raw); + return Number.isFinite(value) ? value : fallback; +} + +/** + * One step of an answered pipeline as an outcome. + * + * A step the server did not send becomes that statement's OWN failure rather + * than shifting every later result onto the wrong statement - the shape a short + * answer would otherwise silently produce. + */ +function readOutcome(raw: unknown): LibSQLBatchOutcome { + const step = asRecord(raw); + if (!step) return { ok: false, error: new LibSQLTransportError(NO_RESULT, 200) }; + + if (step.type === "error") { + const error = asRecord(step.error); + const message = typeof error?.message === "string" ? error.message : "the statement failed"; + const code = typeof error?.code === "string" ? error.code : null; + // Status 200 is the truth here and is carried deliberately: the provider maps + // this to a QueryError so the user reads SQLite's own wording, and a reader of + // the raw error should see that the transport itself succeeded. + return { ok: false, error: new LibSQLTransportError(message, 200, code) }; + } + + return { ok: true, result: toStatementResult(asRecord(step.response)?.result) }; +} + +function toStatementResult(raw: unknown): LibSQLStatementResult { + const result = (asRecord(raw) ?? {}) as HranaStatementResult; + const { fieldNames, columnTypes } = readColumns(result.cols); + + return { + rows: readRows(result.rows, fieldNames), + fieldNames, + columnTypes, + affectedRowCount: toNumber(result.affected_row_count, 0), + lastInsertRowId: result.last_insert_rowid === null ? null : decodeInteger(result.last_insert_rowid), + executionTimeMs: toNumber(result.query_duration_ms, 0), + }; +} + +/** + * The error a non-2xx response carried. + * + * Two envelopes, both measured: the auth failures answer `{"error": ""}` + * while other refusals may answer the `{ message, code }` object the statement + * path uses. Anything else falls back to the raw body, trimmed, because a proxy's + * HTML page is more use to a reader than "request failed". + */ +function httpError(status: number, text: string): LibSQLTransportError { + const body = asRecord(parseJson(text)); + const bare = typeof body?.error === "string" ? body.error : null; + const nested = asRecord(body?.error); + const message = + bare ?? + (typeof nested?.message === "string" ? nested.message : null) ?? + (text.trim() === "" ? `HTTP ${status}` : text.trim()); + const code = typeof nested?.code === "string" ? nested.code : null; + + return new LibSQLTransportError(`libSQL request failed: ${message}`, status, code); +} + +// ============================================================================ +// Transport +// ============================================================================ + +export class LibSQLHranaTransport implements LibSQLTransport { + public readonly kind = "hrana-http" as const; + + private readonly origin: string; + private readonly authorization: string | undefined; + + constructor(config: DatabaseConnection) { + const secure = config.ssl !== undefined && config.ssl.mode !== "disable"; + const port = config.port ?? (secure ? DEFAULT_TLS_PORT : DEFAULT_PORT); + this.origin = `${secure ? "https" : "http"}://${formatHost(config.host ?? DEFAULT_HOST)}:${port}`; + // The credential is a token, not a password: libSQL has no user names, and + // Turso mints a JWT per database. A connection with no token sends no header, + // which is what an unauthenticated local sqld expects - sending an empty + // bearer to it is a 400 rather than an anonymous connection. + this.authorization = config.password ? `Bearer ${config.password}` : undefined; + } + + public async execute(sql: string, options: LibSQLExecuteOptions = {}): Promise { + const [outcome] = await this.pipeline([{ sql, params: options.params }], options.timeoutMs); + // A single statement raises its own failure rather than handing one back: the + // caller asked one question, so there is no other panel for the answer to + // cost. `executeBatch` is the shape for the other case. + if (!outcome || !outcome.ok) throw outcome?.error ?? new LibSQLTransportError(NO_RESULT, 200); + return outcome.result; + } + + public async executeBatch( + statements: LibSQLStatement[], + options: LibSQLExecuteOptions = {}, + ): Promise { + if (statements.length === 0) return []; + return this.pipeline(statements, options.timeoutMs); + } + + /** + * One round trip carrying every statement, and one outcome per statement. + * + * `close` travels with the statements instead of following them. Hrana keeps a + * server-side stream alive between requests and hands back a `baton` to + * continue it; a provider that never continues one must close it in the same + * request, or every call leaves a stream for the server to time out. + */ + private async pipeline(statements: LibSQLStatement[], timeoutMs?: number): Promise { + const requests: Record[] = statements.map((statement) => { + const stmt: Record = { sql: statement.sql }; + // Omitted rather than sent empty: an `args` member on a statement with no + // placeholders is accepted, but sending one makes every log and every + // capture of a plain statement look parameterised. + if (statement.params && statement.params.length > 0) stmt.args = statement.params.map(encodeValue); + return { type: "execute", stmt }; + }); + requests.push({ type: "close" }); + + const text = await this.send(PIPELINE_PATH, JSON.stringify({ requests }), timeoutMs); + + const envelope = asRecord(parseJson(text)) as HranaPipelineEnvelope | null; + const results = envelope?.results; + // The envelope is the transport's own contract, so a body that is not one + // fails the whole call: there is no per-statement answer to hand back. + if (!Array.isArray(results)) throw new LibSQLTransportError(NOT_AN_ENVELOPE, 200); + + return statements.map((_statement, index) => readOutcome(results[index])); + } + + public async serverVersion(): Promise { + try { + const response = await fetch(`${this.origin}${VERSION_PATH}`, { + method: "GET", + headers: this.headers(), + }); + if (!response.ok) return null; + const text = (await response.text()).trim(); + return text === "" ? null : text; + } catch { + // A deployment without the route, and a deployment that could not be + // reached at all, are both "no version to show" for this call. The version + // panel's connection has already been established by the time it runs, so a + // failure here cannot mean the server is down. + return null; + } + } + + /** Nothing to release: every pipeline closes its own stream. */ + public async close(): Promise { + return Promise.resolve(); + } + + private headers(): Record { + const headers: Record = { "content-type": "application/json" }; + if (this.authorization) headers.authorization = this.authorization; + return headers; + } + + private async send(path: string, body: string, timeoutMs?: number): Promise { + let response: Response; + try { + response = await fetch(`${this.origin}${path}`, { + method: "POST", + headers: this.headers(), + body, + // A statement that hangs would otherwise hang the request forever: fetch + // has no default timeout. The signal covers the connect and the read, + // which a timer wrapped around the promise would not. + signal: timeoutMs === undefined ? undefined : AbortSignal.timeout(timeoutMs), + }); + } catch (cause) { + const reason = cause instanceof Error ? cause.message : String(cause); + throw new LibSQLTransportError(`libSQL request failed: ${reason}`, 0); + } + + const text = await response.text(); + if (!response.ok) throw httpError(response.status, text); + return text; + } +} diff --git a/src/lib/db/providers/sql/libsql/index.ts b/src/lib/db/providers/sql/libsql/index.ts new file mode 100644 index 00000000..b66d5c4c --- /dev/null +++ b/src/lib/db/providers/sql/libsql/index.ts @@ -0,0 +1,465 @@ +/** + * libSQL Database Provider (issue #424 Phase 5) + * + * One type-id for two deployments of the same engine: a self-hosted libSQL server + * (`sqld`) and Turso Cloud, which is that server managed. Both speak Hrana, both + * embed SQLite 3.47.0, and everything the provider asks is SQL - so this file is + * the SQLite dialect over a network rather than a second engine. + * + * It is NOT the SQLite provider with a different handle, and the differences are + * measured rather than assumed (2026-08-27, sqld 0.24.33 and Turso Cloud): + * + * - `VACUUM`, `ANALYZE`, `PRAGMA optimize` and `PRAGMA wal_checkpoint` are refused + * by the server's statement allowlist on BOTH deployments, so this provider + * offers `reindex` and `check` and nothing else. Offering a vacuum that always + * fails is the Cloud Spanner shape - a control that reports success while + * nothing happens, or fails every time - and #424 refuses rows for it. + * - `PRAGMA query_only = true` is refused as well, which is why this provider + * implements no `queryReadOnly`: the agent read-only profile delegates + * enforcement to the engine (`sqlite.ts`), and here there is nothing to delegate + * to. A read-only Turso token is an engine-side answer to the same question and + * is the right route if the profile is ever wanted - it is a credential the user + * mints, not a statement this provider can issue. + * - `dbstat` IS available on both deployments, so per-table and per-index bytes are + * measured here - which `bun:sqlite` cannot do at all. + * - There is no file to `stat` and no handle to hold: sizes come from the page + * counters, and there is no session to count anywhere. + * + * Zero runtime dependency: see `hrana-transport.ts`. + */ + +import { SQLBaseProvider } from "../sql-base"; +import { + type DatabaseConnection, + type ActiveSessionDetails, + type DatabaseOverview, + type HealthInfo, + type IndexStats, + type MaintenanceResult, + type MaintenanceType, + type PerformanceMetrics, + type ProviderCapabilities, + type ProviderLabels, + type ProviderOptions, + type QueryResult, + type SlowQueryStats, + type StorageStats, + type TableSchema, + type TableStats, +} from "@/lib/db/types"; +import { AuthenticationError, ConnectionError, DatabaseConfigError, QueryError } from "@/lib/db/errors"; +import { LibSQLHranaTransport } from "./hrana-transport"; +import { + readActiveSessions, + readHealth, + readIndexStats, + readOverview, + readSchema, + readSlowQueries, + readStorageStats, + readTableStats, +} from "./introspect"; +import { type LibSQLStatementResult, type LibSQLTransport, LibSQLTransportError } from "./transport"; + +// ============================================================================ +// Constants +// ============================================================================ + +/** + * The cheapest statement that proves the server, the credential and the database + * together. A libSQL server answers `/health` without looking at the token, so a + * health route is not a connection test. + */ +const CONNECT_PROBE_SQL = "SELECT 1"; + +const INTEGRITY_CHECK_SQL = "PRAGMA integrity_check"; + +/** + * The credential is a token rather than a password, and it arrives in the query + * string rather than in the authority: `libsql://.turso.io?authToken=` is + * the form Turso's own CLI prints. + */ +const AUTH_TOKEN_PARAM = "authToken"; + +/** Statuses a libSQL deployment answers a credential problem with. */ +const AUTH_STATUSES = new Set([400, 401, 403]); + +// ============================================================================ +// Connection resolution +// ============================================================================ + +/** + * The configuration with a hand-typed `libsql://` URL resolved into fields. + * + * `libsql://` implies TLS, which is why the scheme carries no second form here: + * Turso serves every database over HTTPS on 443, and a self-hosted plaintext + * server is reached through the host/port fields with TLS off. Inventing a + * `libsql+http://` scheme no libSQL tool emits would be worse than that gap. + */ +function resolveConnection(config: DatabaseConnection): DatabaseConnection { + if (!config.connectionString) return config; + + let url: URL; + try { + url = new URL(config.connectionString); + } catch { + // Left to `validate()` and to the transport to report: a string that is not a + // URL is a configuration error, and swallowing it here would send the request + // to whatever the form fields happened to hold. + return config; + } + + const token = url.searchParams.get(AUTH_TOKEN_PARAM); + + return { + ...config, + host: url.hostname || config.host, + port: url.port === "" ? config.port : Number(url.port), + password: token ?? (url.password === "" ? config.password : decodeURIComponent(url.password)), + ssl: config.ssl ?? { mode: "require" }, + }; +} + +// ============================================================================ +// Result mapping +// ============================================================================ + +function toQueryResult(result: LibSQLStatementResult, measuredMs: number): QueryResult { + const reportedMs = Math.round(result.executionTimeMs); + + return { + rows: result.rows, + fields: result.fieldNames, + // A write returns no rows, so its count is what the engine says it changed. + rowCount: result.rows.length > 0 ? result.rows.length : result.affectedRowCount, + // The engine's own measurement when it is at least a millisecond, and the + // wall-clock one otherwise: a statement that really took 0.02 ms would + // otherwise be reported as having taken no time at all. + executionTime: reportedMs > 0 ? reportedMs : measuredMs, + // Declared types travel with the result (#273) and are omitted when the engine + // declared none - which it does for every computed column and every PRAGMA, so + // an empty map is the common case rather than a failure. + ...(Object.keys(result.columnTypes).length > 0 ? { columnTypes: result.columnTypes } : {}), + }; +} + +// ============================================================================ +// libSQL Provider +// ============================================================================ + +export class LibSQLProvider extends SQLBaseProvider { + private transport: LibSQLTransport | null = null; + + private readonly connection: DatabaseConnection; + + constructor(config: DatabaseConnection, options: ProviderOptions = {}) { + super(config, options); + this.connection = resolveConnection(config); + this.validate(); + } + + // ========================================================================== + // Provider metadata + // ========================================================================== + + public override getCapabilities(): ProviderCapabilities { + return { + ...super.getCapabilities(), + // sqld's own default. Turso Cloud serves on 443, which the TLS branch of the + // transport picks up - a connection there names no port at all. + defaultPort: 8080, + supportsExplain: true, + // The same plan SQLite produces, because it IS SQLite: `EXPLAIN QUERY PLAN` + // answers the id/parent/notused/detail shape on both deployments. + explainFormat: "sqlite-queryplan", + supportsConnectionString: true, + supportsInlineRowEdit: true, + // libSQL HAS transactions - `BEGIN` is accepted and Hrana keeps an + // interactive stream alive with a `baton` to continue one - but this provider + // closes its stream in the same request as the statement, so it holds no + // session for a transaction to live in. POST /api/db/transaction refuses the + // call and the controls stay hidden, which is the SQLite provider's position + // for the same reason. + supportsTransactions: false, + // Measured on BOTH deployments: `VACUUM`, `ANALYZE`, `PRAGMA optimize` and + // `PRAGMA wal_checkpoint` are all refused by the server's statement + // allowlist ("unsupported statement" on sqld, "SQL not allowed statement" on + // Turso Cloud). `REINDEX` and `PRAGMA integrity_check` are accepted, and they + // are the only two offered here. + maintenanceOperations: ["reindex", "check"], + maintenanceOperationSpecs: { + reindex: { label: "Reindex Table", perEntity: true, global: true }, + check: { label: "Integrity Check", perEntity: false, global: true }, + }, + // Every statement is a fresh request, so a `CREATE TABLE` typed in the editor + // behaves exactly as it does against a file. + supportsCreateTable: true, + schemaRefreshPattern: "(CREATE|DROP|ALTER|TRUNCATE|REINDEX)\\b", + }; + } + + /** + * The two labels the engine's own gaps require, and no others: `tables` and + * `rows` are the right words for SQLite. + */ + public override getLabels(): ProviderLabels { + return { + ...super.getLabels(), + slowQueriesEmptyState: "libSQL keeps no statistics about finished statements, so there is nothing to enable.", + reindexGlobalLabel: "Run Reindex", + reindexGlobalTitle: "Rebuild Indexes", + reindexGlobalDesc: "Runs bare REINDEX, rebuilding every index in the database.", + }; + } + + // ========================================================================== + // Validation + // ========================================================================== + + public validate(): void { + super.validate(); + + if (!this.connection.host && !this.config.connectionString) { + throw new DatabaseConfigError( + 'A host is required for libSQL (or a libsql:// URL in "connectionString")', + "libsql", + ); + } + } + + // ========================================================================== + // Connection management + // ========================================================================== + + public async connect(): Promise { + if (this.transport) return; + + const transport = new LibSQLHranaTransport(this.connection); + + try { + await transport.execute(CONNECT_PROBE_SQL, { timeoutMs: this.queryTimeout }); + } catch (error) { + await transport.close(); + const failure = this.describeConnectFailure(error); + this.setError(failure); + throw failure; + } + + this.transport = transport; + this.setConnected(true); + } + + public async disconnect(): Promise { + if (this.transport) { + await this.transport.close(); + this.transport = null; + } + this.setConnected(false); + } + + // ========================================================================== + // Queries + // ========================================================================== + + public async query(sql: string, params?: unknown[]): Promise { + const transport = this.requireTransport(); + + return this.trackQuery(async () => { + const { result, executionTime } = await this.measureExecution(async () => { + try { + return await transport.execute(sql, { params, timeoutMs: this.queryTimeout }); + } catch (error) { + throw this.mapLibSQLError(error, sql); + } + }); + + return toQueryResult(result, executionTime); + }); + } + + // ========================================================================== + // Schema + // ========================================================================== + + public async getSchema(): Promise { + const transport = this.requireTransport(); + try { + return await readSchema(transport); + } catch (error) { + throw this.mapLibSQLError(error); + } + } + + // ========================================================================== + // Health and monitoring + // ========================================================================== + + public async getHealth(): Promise { + const transport = this.requireTransport(); + try { + return await readHealth(transport); + } catch (error) { + throw this.mapLibSQLError(error); + } + } + + public async getOverview(): Promise { + const transport = this.requireTransport(); + try { + return await readOverview(transport); + } catch (error) { + throw this.mapLibSQLError(error); + } + } + + /** + * Only what libSQL can actually be asked, which is no cache hit ratio at all. + * + * SQLite's hit and miss counters live behind the C API (`sqlite3_db_status()`), + * and no statement reaches them - so nothing over Hrana can either. The one + * field kept is `deadlocks`, and it is a statement about the engine rather than a + * reading that failed: SQLite serializes writers behind a single write lock and + * refuses a second one with SQLITE_BUSY, so there are no deadlocks to count and 0 + * is the true count. + */ + public async getPerformanceMetrics(): Promise { + this.ensureConnected(); + + return { deadlocks: 0 }; + } + + public async getSlowQueries(): Promise { + this.ensureConnected(); + + return readSlowQueries(); + } + + public async getActiveSessions(): Promise { + this.ensureConnected(); + + return readActiveSessions(); + } + + public async getTableStats(): Promise { + const transport = this.requireTransport(); + try { + return await readTableStats(transport); + } catch (error) { + throw this.mapLibSQLError(error); + } + } + + public async getIndexStats(): Promise { + const transport = this.requireTransport(); + try { + return await readIndexStats(transport); + } catch (error) { + throw this.mapLibSQLError(error); + } + } + + public async getStorageStats(): Promise { + const transport = this.requireTransport(); + try { + return await readStorageStats(transport); + } catch (error) { + throw this.mapLibSQLError(error); + } + } + + // ========================================================================== + // Maintenance + // ========================================================================== + + /** + * The two operations the server accepts, and an honest refusal for the rest. + * + * `check` reads the answer rather than the status: `PRAGMA integrity_check` + * succeeds as a statement and reports the damage in its row, so a provider that + * only checked for an exception would report a corrupt database as healthy. + */ + public async runMaintenance(type: MaintenanceType, target?: string): Promise { + const transport = this.requireTransport(); + + const { result, executionTime } = await this.measureExecution(async () => { + if (type === "check") { + const check = await this.run(transport, INTEGRITY_CHECK_SQL); + const verdict = check.rows[0]?.integrity_check; + return { success: verdict === "ok", message: typeof verdict === "string" ? verdict : "Unknown" }; + } + + if (type === "reindex") { + const sql = target ? `REINDEX ${this.escapeIdentifier(target)}` : "REINDEX"; + await this.run(transport, sql); + return { success: true, message: "REINDEX completed successfully" }; + } + + // Everything else is refused HERE rather than sent and allowed to fail, so + // the message names the reason instead of relaying "unsupported statement" + // from a server the user did not ask to talk to. `maintenanceOperations` + // already withholds these controls; this covers a direct API call. + throw new QueryError( + `libSQL servers do not accept ${type.toUpperCase()}: only REINDEX and PRAGMA integrity_check are allowed`, + "libsql", + ); + }); + + return { success: result.success, executionTime, message: result.message }; + } + + // ========================================================================== + // Internals + // ========================================================================== + + private async run(transport: LibSQLTransport, sql: string): Promise { + try { + return await transport.execute(sql, { timeoutMs: this.queryTimeout }); + } catch (error) { + throw this.mapLibSQLError(error, sql); + } + } + + private requireTransport(): LibSQLTransport { + this.ensureConnected(); + // Assigned before setConnected(true) and cleared after setConnected(false), so + // a connected provider always has one. + return this.transport!; + } + + private describeConnectFailure(error: unknown): Error { + const mapped = this.mapLibSQLError(error); + if (mapped instanceof AuthenticationError) return mapped; + + return new ConnectionError( + `Failed to connect to libSQL: ${mapped.message}`, + "libsql", + this.connection.host, + this.connection.port, + ); + } + + /** + * A transport failure as the error the rest of the app expects. + * + * The status carries the distinction, not the wording: the two deployments word + * the same refusal differently ("unsupported statement" against "SQL not allowed + * statement"), while a statement error is always a 200 and a credential problem + * is always a 4xx. Matching on text would have been wrong on one of the two + * deployments from the first day. + */ + private mapLibSQLError(error: unknown, sql?: string): Error { + if (!(error instanceof LibSQLTransportError)) { + return error instanceof Error ? error : new QueryError(String(error), "libsql", sql); + } + + if (AUTH_STATUSES.has(error.status)) { + return new AuthenticationError(error.message, "libsql"); + } + + if (error.status === 0) { + return new ConnectionError(error.message, "libsql", this.connection.host, this.connection.port); + } + + return new QueryError(error.message, "libsql", sql); + } +} diff --git a/src/lib/db/providers/sql/libsql/introspect.ts b/src/lib/db/providers/sql/libsql/introspect.ts new file mode 100644 index 00000000..b7557571 --- /dev/null +++ b/src/lib/db/providers/sql/libsql/introspect.ts @@ -0,0 +1,542 @@ +/** + * libSQL introspection (issue #424 Phase 5) + * + * Every reading here is SQL, because SQLite's introspection is SQL: `sqlite_master` + * for the object list, the `pragma_*` table-valued functions for columns, indexes + * and foreign keys, and `dbstat` for bytes. There is no management API to call and + * nothing in this file knows how the statements travel - that is the transport's + * job, and the seam guard enforces it. + * + * Two shapes drive the design, both measured on 2026-08-27 against sqld 0.24.33 and + * against a Turso Cloud database: + * + * - A batch answers EACH statement separately, so a per-table read that fails costs + * its own reading and nothing else. That is why the per-table sweep is one batch + * whose outcomes are read individually rather than a `Promise.all` that a single + * refusal collapses (BACKLOG D22, #477). + * - `dbstat` is available on BOTH deployments, which is more than the SQLite provider + * can say of its own drivers (`bun:sqlite` has no dbstat at all). So per-table and + * per-index bytes here are measured rather than absent - and when the table IS + * missing, every byte figure is omitted rather than zeroed, which is the same rule + * `buildTableStats` follows in `sqlite.ts`. + * + * The `pragma_*` functions are used instead of the `PRAGMA` statement form on + * purpose: they are ordinary table-valued functions, so they can be projected and + * filtered, and they carry the object name as a bound-looking literal rather than as + * part of the statement keyword. Both forms are accepted by both deployments. + */ + +import { CACHE_HIT_RATIO_UNAVAILABLE } from "@/lib/monitoring-cache-ratio"; +import type { + ActiveSessionDetails, + DatabaseOverview, + HealthInfo, + IndexStats, + SlowQueryStats, + StorageStats, + TableSchema, + TableStats, +} from "@/lib/db/types"; +import { formatBytes } from "@/lib/db/utils/pool-manager"; +import type { LibSQLBatchOutcome, LibSQLRow, LibSQLTransport } from "./transport"; + +// ============================================================================ +// Introspection SQL +// ---------------------------------------------------------------------------- +// Hoisted to module scope (not inlined in the functions) on purpose. bun's +// coverage instruments the interior lines of a multi-line template literal in a +// function body as 0-hit in any test process that imports this file but does not +// exercise the function, and the merged lcov then reports those SQL lines as +// uncovered even though the caller is tested. Evaluated once at module load, these +// consts are reported as covered everywhere (same pattern as sqlite.ts). +// ============================================================================ + +/** The user tables, with SQLite's own objects left out. */ +const TABLES_SQL = `SELECT name FROM sqlite_master + WHERE type = 'table' AND name NOT LIKE 'sqlite_%' + ORDER BY name`; + +const TABLE_COUNT_SQL = `SELECT COUNT(*) AS table_count FROM sqlite_master + WHERE type = 'table' AND name NOT LIKE 'sqlite_%'`; + +const INDEX_COUNT_SQL = `SELECT COUNT(*) AS index_count FROM sqlite_master + WHERE type = 'index' AND name NOT LIKE 'sqlite_%'`; + +/** + * The database's own footprint, from the page counters rather than from a file. + * + * `fs.statSync` is what the SQLite provider uses and is exactly what this provider + * cannot do: the file is on the server's disk, not ours. The page product is the + * same number - measured 282624 bytes against a database sqld reported as 276 KB. + */ +const DB_SIZE_SQL = `SELECT (SELECT page_count FROM pragma_page_count()) * + (SELECT page_size FROM pragma_page_size()) AS size_bytes`; + +const SQLITE_VERSION_SQL = "SELECT sqlite_version() AS version"; +const INTEGRITY_CHECK_SQL = "PRAGMA integrity_check"; +const JOURNAL_MODE_SQL = "PRAGMA journal_mode"; + +/** + * Per-object page bytes. + * + * `dbstat` is a compile-time option, and this provider does not get to choose the + * build it talks to - so the caller treats a refusal as "no bytes to show" rather + * than as a failure. Available on both deployments measured. + */ +const DBSTAT_SIZES_SQL = `SELECT name, SUM(pgsize) AS bytes FROM dbstat GROUP BY name`; + +/** Which table each index belongs to, so an index's pages land on its table. */ +const INDEX_OWNERS_SQL = `SELECT name, tbl_name FROM sqlite_master WHERE type = 'index'`; + +// ============================================================================ +// Pure helpers +// ============================================================================ + +/** + * A SQLite string literal. Doubling the quote is the whole escape SQLite defines. + * + * Object names reach these statements from `sqlite_master`, so they are the + * engine's own words rather than a user's - but a table really can be named + * `it's`, and a name that breaks the statement it is embedded in would cost the + * whole sweep. + */ +function literal(value: string): string { + return `'${value.replaceAll("'", "''")}'`; +} + +/** A quoted identifier. `"` is doubled, the one escape SQLite defines for these. */ +function identifier(value: string): string { + return `"${value.replaceAll('"', '""')}"`; +} + +/** + * A statistic as a number, or absent. + * + * The string branch is the transport's wide-integer form (`decodeInteger` keeps an + * integer past 2^53 as its exact decimal string rather than rounding it), and here + * it IS parsed to a double. That is deliberate and bounded: every caller of this + * function is a COUNT or a byte total for a panel, where the reading is a display + * figure - a row count above 2^53 is 9 quadrillion rows - while the values that + * must not be rounded are result CELLS, which never pass through here. + */ +function readNumber(value: unknown): number | undefined { + if (typeof value === "number") return Number.isFinite(value) ? value : undefined; + if (typeof value !== "string" || value.trim() === "") return undefined; + const parsed = Number(value); + return Number.isFinite(parsed) ? parsed : undefined; +} + +function readText(value: unknown): string | undefined { + return typeof value === "string" && value !== "" ? value : undefined; +} + +/** The rows of one outcome, or null when that statement did not answer. */ +function rowsOf(outcome: LibSQLBatchOutcome | undefined): LibSQLRow[] | null { + return outcome?.ok === true ? outcome.result.rows : null; +} + +/** The first row of one outcome, or null when that statement did not answer. */ +function firstRow(outcome: LibSQLBatchOutcome | undefined): LibSQLRow | null { + const rows = rowsOf(outcome); + return rows === null ? null : (rows[0] ?? null); +} + +/** + * True for an index SQLite created for itself. + * + * A UNIQUE column and a non-INTEGER primary key each get an `sqlite_autoindex_*` + * that no user declared and no user can drop, so listing them in the object tree + * reports objects the schema does not contain. + */ +function isInternalIndex(name: string): boolean { + return name.startsWith("sqlite_"); +} + +// ============================================================================ +// Object bytes +// ============================================================================ + +/** One table's measured page bytes, split between its own b-tree and its indexes. */ +interface ObjectSizeBytes { + tableSizeBytes: number; + indexSizeBytes: number; +} + +interface MeasuredSizes { + /** Bytes per table, indexes folded in under the table that owns them. */ + byTable: Map; + /** Bytes per index object, for the index stats tab. */ + byIndex: Map; +} + +/** + * Page bytes per object, or null when this build has no `dbstat`. + * + * Null is the whole point: it is what makes every byte figure downstream ABSENT + * rather than 0. A zero there reads as an empty table, which is a claim, and the + * SQLite provider learned that lesson the expensive way (`rowCount * 100`). + */ +async function readSizes(transport: LibSQLTransport): Promise { + const [pagesOutcome, ownersOutcome] = await transport.executeBatch([ + { sql: DBSTAT_SIZES_SQL }, + { sql: INDEX_OWNERS_SQL }, + ]); + + const pages = rowsOf(pagesOutcome); + if (pages === null) return null; + + const owners = rowsOf(ownersOutcome) ?? []; + const bytesByObject = new Map(); + for (const row of pages) { + const name = readText(row.name); + if (name !== undefined) bytesByObject.set(name, readNumber(row.bytes) ?? 0); + } + + const byTable = new Map(); + const byIndex = new Map(); + const entryFor = (tableName: string): ObjectSizeBytes => { + const existing = byTable.get(tableName); + if (existing) return existing; + const created = { tableSizeBytes: 0, indexSizeBytes: 0 }; + byTable.set(tableName, created); + return created; + }; + + // Indexes first, so an index's pages land on the table that owns them: the + // Storage tab's index total is a per-TABLE figure. + const indexNames = new Set(); + for (const row of owners) { + const name = readText(row.name); + const owner = readText(row.tbl_name); + if (name === undefined || owner === undefined) continue; + indexNames.add(name); + const bytes = bytesByObject.get(name) ?? 0; + entryFor(owner).indexSizeBytes += bytes; + byIndex.set(name, bytes); + } + for (const [name, bytes] of bytesByObject) { + if (!indexNames.has(name)) entryFor(name).tableSizeBytes += bytes; + } + + return { byTable, byIndex }; +} + +// ============================================================================ +// Schema +// ============================================================================ + +/** One index as `pragma_index_list` describes it, before its columns are read. */ +interface IndexDescriptor { + tableName: string; + name: string; + unique: boolean; + isPrimary: boolean; +} + +interface CollectedTable { + name: string; + rowCount: number | undefined; + columns: TableSchema["columns"]; + foreignKeys: NonNullable; + indexes: IndexDescriptor[]; +} + +/** The four questions asked of every table, in the order the outcomes are read. */ +function tableStatements(tableName: string): { sql: string }[] { + return [ + { sql: `SELECT COUNT(*) AS row_count FROM ${identifier(tableName)}` }, + // `"notnull"` is QUOTED because it is a SQLite keyword - the postfix `x NOTNULL` + // operator - and projecting it bare is a parse error, not a column: measured on + // sqld 0.24.33, `SELECT cid, name, type, notnull, ... FROM pragma_table_info(...)` + // answers "near NOTNULL: syntax error" while the same statement with the name + // quoted returns the rows. Nothing above the transport could see that failure - + // it costs the COLUMNS of every table and leaves the tree otherwise intact - so + // this is a live-probe finding rather than a test one. + { sql: `SELECT cid, name, type, "notnull", dflt_value, pk FROM pragma_table_info(${literal(tableName)})` }, + { sql: `SELECT seq, name, "unique", origin FROM pragma_index_list(${literal(tableName)})` }, + { sql: `SELECT id, seq, "table", "from", "to" FROM pragma_foreign_key_list(${literal(tableName)})` }, + ]; +} + +function collectTable(tableName: string, outcomes: LibSQLBatchOutcome[]): CollectedTable { + const [count, columns, indexes, foreignKeys] = outcomes; + + return { + name: tableName, + // Absent rather than 0 when the count was refused: a table reported as empty + // is a reading, and this one failed. + rowCount: readNumber(firstRow(count)?.row_count), + columns: (rowsOf(columns) ?? []).map((row) => ({ + name: readText(row.name) ?? "", + // SQLite allows a column with no declared type at all (it is then a BLOB + // affinity column). TEXT is what the SQLite provider substitutes, and the + // same substitution is kept here so the two read alike. + type: readText(row.type) ?? "TEXT", + nullable: readNumber(row.notnull) !== 1, + isPrimary: readNumber(row.pk) === 1, + ...(row.dflt_value === null || row.dflt_value === undefined ? {} : { defaultValue: String(row.dflt_value) }), + })), + foreignKeys: (rowsOf(foreignKeys) ?? []).map((row) => ({ + columnName: readText(row.from) ?? "", + referencedTable: readText(row.table) ?? "", + referencedColumn: readText(row.to) ?? "", + })), + indexes: (rowsOf(indexes) ?? []) + .map((row) => ({ + tableName, + name: readText(row.name) ?? "", + unique: readNumber(row.unique) === 1, + isPrimary: readText(row.origin) === "pk", + })) + .filter((index) => index.name !== "" && !isInternalIndex(index.name)), + }; +} + +/** + * Every table with its columns, indexes and foreign keys, in as few round trips as + * the questions allow. + * + * Three round trips regardless of table count: the object list, one batch carrying + * four statements per table, and one batch carrying an `index_info` per user index. + * The SQLite provider issues the same questions one at a time, which is free on a + * file and is four network round trips per table here. + */ +async function collectTables( + transport: LibSQLTransport, +): Promise<{ tables: CollectedTable[]; columnsByIndex: Map }> { + // Not tolerated: with no object list there is nothing to degrade to, and the + // caller needs to know the read failed rather than see an empty database. + const tableRows = (await transport.execute(TABLES_SQL)).rows; + const tableNames = tableRows.map((row) => readText(row.name)).filter((name): name is string => name !== undefined); + if (tableNames.length === 0) return { tables: [], columnsByIndex: new Map() }; + + const outcomes = await transport.executeBatch(tableNames.flatMap(tableStatements)); + const tables = tableNames.map((tableName, index) => + collectTable(tableName, outcomes.slice(index * 4, index * 4 + 4)), + ); + + const indexNames = tables.flatMap((table) => table.indexes.map((index) => index.name)); + const columnsByIndex = new Map(); + if (indexNames.length > 0) { + const infoOutcomes = await transport.executeBatch( + indexNames.map((name) => ({ sql: `SELECT seqno, cid, name FROM pragma_index_info(${literal(name)})` })), + ); + for (const [position, name] of indexNames.entries()) { + const columns = (rowsOf(infoOutcomes[position]) ?? []) + .map((row) => readText(row.name)) + .filter((column): column is string => column !== undefined); + columnsByIndex.set(name, columns); + } + } + + return { tables, columnsByIndex }; +} + +export async function readSchema(transport: LibSQLTransport): Promise { + const { tables, columnsByIndex } = await collectTables(transport); + if (tables.length === 0) return []; + + const sizes = await readSizes(transport); + + return tables.map((table) => { + const size = sizes?.byTable.get(table.name); + + return { + name: table.name, + ...(table.rowCount === undefined ? {} : { rowCount: table.rowCount }), + // Omitted, not "0 B", when this build has no dbstat: the object tree draws + // nothing for an absent size and draws "0 B" for a present zero, and one of + // those two is a claim about the table. + ...(size === undefined ? {} : { size: formatBytes(size.tableSizeBytes + size.indexSizeBytes) }), + columns: table.columns, + indexes: table.indexes.map((index) => ({ + name: index.name, + columns: columnsByIndex.get(index.name) ?? [], + unique: index.unique, + })), + foreignKeys: table.foreignKeys, + }; + }); +} + +// ============================================================================ +// Overview, health, metrics +// ============================================================================ + +/** + * What the server is and how much it holds. + * + * The version is the one place the two deployments disagree, and both answers are + * kept: a self-hosted sqld publishes its own version on a route Turso Cloud does + * not have, so the panel reads `sqld 0.24.33 (…) (SQLite 3.47.0)` there and + * `SQLite 3.47.0` on the cloud. Neither is "Unknown", because the engine answered. + */ +export async function readOverview(transport: LibSQLTransport): Promise { + const [serverVersion, outcomes] = await Promise.all([ + transport.serverVersion(), + transport.executeBatch([ + { sql: SQLITE_VERSION_SQL }, + { sql: DB_SIZE_SQL }, + { sql: TABLE_COUNT_SQL }, + { sql: INDEX_COUNT_SQL }, + ]), + ]); + + const sqliteVersion = readText(firstRow(outcomes[0])?.version); + const engine = sqliteVersion === undefined ? "SQLite" : `SQLite ${sqliteVersion}`; + const sizeBytes = readNumber(firstRow(outcomes[1])?.size_bytes); + + return { + version: serverVersion === null ? engine : `${serverVersion} (${engine})`, + // libSQL publishes no start time and no uptime on any route or in any catalog. + // "N/A" is the SQLite provider's own wording for the same absence. + uptime: "N/A", + // No `activeConnections` at all: Hrana is stateless, so a statement is a + // request and there is no session anywhere to count. A 1 here would be this + // provider counting itself. + maxConnections: 0, + databaseSize: sizeBytes === undefined ? "N/A" : formatBytes(sizeBytes), + databaseSizeBytes: sizeBytes ?? 0, + tableCount: readNumber(firstRow(outcomes[2])?.table_count) ?? 0, + indexCount: readNumber(firstRow(outcomes[3])?.index_count) ?? 0, + }; +} + +/** + * The two readings libSQL has, and no invented third. + * + * `PRAGMA integrity_check` and `PRAGMA journal_mode` are both accepted by both + * deployments (unlike `VACUUM`, `ANALYZE`, `PRAGMA optimize` and + * `PRAGMA wal_checkpoint`, which sqld's statement allowlist refuses outright), and + * they are what the health panel shows. The cache hit ratio is the shared + * "not measured" constant rather than a number: SQLite's hit and miss counters + * live behind the C API, and no statement reaches them - which is as true over + * Hrana as it is in the SQLite provider. + */ +export async function readHealth(transport: LibSQLTransport): Promise { + const outcomes = await transport.executeBatch([ + { sql: DB_SIZE_SQL }, + { sql: INTEGRITY_CHECK_SQL }, + { sql: JOURNAL_MODE_SQL }, + ]); + + const sizeBytes = readNumber(firstRow(outcomes[0])?.size_bytes); + const integrity = readText(firstRow(outcomes[1])?.integrity_check); + const journalMode = readText(firstRow(outcomes[2])?.journal_mode) ?? "unknown"; + + return { + databaseSize: sizeBytes === undefined ? "N/A" : formatBytes(sizeBytes), + cacheHitRatio: CACHE_HIT_RATIO_UNAVAILABLE, + slowQueries: [ + { query: `Integrity: ${integrity === "ok" ? "OK" : "FAILED"}`, calls: 0, avgTime: "N/A" }, + { query: `Journal Mode: ${journalMode}`, calls: 0, avgTime: "N/A" }, + ], + // Empty, not a row describing this process: the SQLite provider can name the + // one handle it holds open, and this provider holds none - the server has the + // sessions and publishes none of them. + activeSessions: [], + }; +} + +/** + * Nothing, and the reason is the engine rather than the transport. + * + * libSQL keeps no statement statistics: there is no `SLOWLOG`, no + * `pg_stat_statements` and no equivalent to enable. `query_duration_ms` comes back + * with each answer, but that is this client's own statement, measured once, which + * is not a statistic about finished ones. + */ +export async function readSlowQueries(): Promise { + return []; +} + +/** + * Nothing, for the same reason `readHealth` reports no sessions. + * + * Hrana is stateless over HTTP, and no route publishes the server's connected + * clients. A row for the request in flight would be this provider describing + * itself. + */ +export async function readActiveSessions(): Promise { + return []; +} + +// ============================================================================ +// Stats tabs +// ============================================================================ + +export async function readTableStats(transport: LibSQLTransport): Promise { + const { tables } = await collectTables(transport); + if (tables.length === 0) return []; + + const sizes = await readSizes(transport); + + return tables.map((table) => { + const size = sizes?.byTable.get(table.name); + const rowCount = table.rowCount ?? 0; + if (size === undefined) { + // `totalSize`/`totalSizeBytes` are required by `TableStats`, so they carry + // the "N/A" / 0 placeholder the tab keys off - it draws neither figure once + // `tableSizeBytes` is absent (#469). + return { schemaName: "main", tableName: table.name, rowCount, totalSize: "N/A", totalSizeBytes: 0 }; + } + + const totalSizeBytes = size.tableSizeBytes + size.indexSizeBytes; + return { + schemaName: "main", + tableName: table.name, + rowCount, + tableSize: formatBytes(size.tableSizeBytes), + tableSizeBytes: size.tableSizeBytes, + indexSize: formatBytes(size.indexSizeBytes), + indexSizeBytes: size.indexSizeBytes, + totalSize: formatBytes(totalSizeBytes), + totalSizeBytes, + }; + }); +} + +export async function readIndexStats(transport: LibSQLTransport): Promise { + const { tables, columnsByIndex } = await collectTables(transport); + if (tables.length === 0) return []; + + const sizes = await readSizes(transport); + + return tables.flatMap((table) => + table.indexes.map((index) => { + const bytes = sizes?.byIndex.get(index.name); + + return { + schemaName: "main", + tableName: table.name, + indexName: index.name, + columns: columnsByIndex.get(index.name) ?? [], + isUnique: index.unique, + isPrimary: index.isPrimary, + indexSize: bytes === undefined ? "N/A" : formatBytes(bytes), + ...(bytes === undefined ? {} : { indexSizeBytes: bytes }), + // SQLite keeps no per-index scan counter anywhere, so this is the count of + // a statistic that does not exist rather than a measurement of no scans. + // `scans` is required by `IndexStats`; the tab renders the 0. + scans: 0, + }; + }), + ); +} + +/** + * The one file libSQL has, or nothing. + * + * An entry reading 0 B would draw an empty database on the Storage tab, which is a + * claim; no entry draws the tab's own empty state, which is the honest one. + */ +export async function readStorageStats(transport: LibSQLTransport): Promise { + const [outcome] = await transport.executeBatch([{ sql: DB_SIZE_SQL }]); + const sizeBytes = readNumber(firstRow(outcome)?.size_bytes); + if (sizeBytes === undefined) return []; + + // No `walSize`: the WAL is a file on the server's disk and no statement reports + // its size. `PRAGMA wal_checkpoint` - which would at least prove it exists - is + // one of the statements sqld refuses. + return [{ name: "main", size: formatBytes(sizeBytes), sizeBytes }]; +} diff --git a/src/lib/db/providers/sql/libsql/transport.ts b/src/lib/db/providers/sql/libsql/transport.ts new file mode 100644 index 00000000..6e6b01a6 --- /dev/null +++ b/src/lib/db/providers/sql/libsql/transport.ts @@ -0,0 +1,162 @@ +/** + * libSQL transport seam (issue #424 Phase 5) + * + * Provider logic never talks to a libSQL server directly. It goes through this + * interface, so a second implementation - Hrana over WebSocket, the embedded + * `@tursodatabase/database` engine, or `@libsql/client` if a driver is ever + * warranted - is one new file rather than a rewrite of the provider, the + * introspection and the explain strategy. Sibling of the ClickHouse seam in + * `providers/sql/clickhouse/transport.ts` and the Couchbase one. + * + * The types below are deliberately NEUTRAL: they describe what a caller needs, + * not how one protocol encodes it. Everything Hrana invented - its + * `/v2/pipeline` endpoint, its request/response envelope, its `baton`, its + * `{ type, value }` value encoding - stays inside `hrana-transport.ts`, and + * `seam-guard.test.ts` fails the build when that vocabulary appears anywhere + * else in this directory. That is what keeps the "one new file" estimate true. + * + * Apart from the error type this file is purely structural: no I/O. + */ + +/** + * One result row, keyed by the column name the engine declared. + * + * libSQL answers a statement with declared columns and positional rows, so an + * object row is a mapping this layer performs rather than a shape the wire + * carries. Duplicate column names collapse - `SELECT 1 AS a, 2 AS a` keeps the + * last - which is SQLite's own behaviour through every driver here and is why + * `fieldNames` is carried separately. + */ +export type LibSQLRow = Record; + +/** Normalized outcome of one statement. */ +export interface LibSQLStatementResult { + rows: LibSQLRow[]; + + /** + * Column order exactly as the engine declared it. Never null: libSQL declares + * columns for every statement, answering an empty list for one that projects + * nothing (a `CREATE TABLE`), which is a declaration rather than an absence. + */ + fieldNames: string[]; + + /** + * SQLite's declared type per column, verbatim - `TEXT`, `INTEGER`, `NUMERIC`. + * + * Populated only for the columns that have one. SQLite declares a type for a + * column read straight out of a table and NOTHING for a computed one, which + * is not this transport's gap to fill: `SELECT name FROM t` declares `TEXT` + * while `SELECT sqlite_version()` and every `PRAGMA` column declare nothing + * (measured on sqld 0.24.33 and Turso Cloud, both SQLite 3.47.0). An empty + * record therefore means "the engine declared no types", never "the transport + * did not look". + */ + columnTypes: Record; + + /** Rows a write changed, as the engine counted them. */ + affectedRowCount: number; + + /** + * The rowid the last INSERT produced, or null when the statement produced none. + * + * A number where the value is exactly representable and a decimal STRING where + * it is not, for the reason `decodeInteger` states: a rowid past 2^53 that + * arrives rounded is a silent corruption, and SQLite rowids are 64-bit. + */ + lastInsertRowId: number | string | null; + + executionTimeMs: number; +} + +/** One statement and the parameters it binds. */ +export interface LibSQLStatement { + sql: string; + /** Positional parameters, in the order the statement's `?` placeholders appear. */ + params?: unknown[]; +} + +/** + * What one statement of a batch produced: its result, or the failure that is + * ITS failure alone. + * + * A discriminated outcome rather than a result array plus a throw, and that is + * the whole point of the shape. Measured on both deployments: a batch whose + * second statement fails still runs the third, and each statement carries its own + * outcome. Collapsing that onto a single rejection is how one refused read costs + * a whole monitoring dashboard (#477, BACKLOG D22) - so the transport hands the + * failures back individually and the provider decides, per panel, what an absent + * reading means. + */ +export type LibSQLBatchOutcome = + | { ok: true; result: LibSQLStatementResult } + | { ok: false; error: LibSQLTransportError }; + +/** Options a caller may attach to one statement. */ +export interface LibSQLExecuteOptions { + /** Positional parameters, in the order the statement's `?` placeholders appear. */ + params?: unknown[]; + timeoutMs?: number; +} + +/** + * What the provider is allowed to ask of a libSQL server. + * + * Narrow on purpose: everything else the provider needs it builds out of + * statements, because SQLite's introspection is SQL (`sqlite_master`, the + * `pragma_*` table-valued functions) rather than a management API. + */ +export interface LibSQLTransport { + readonly kind: "hrana-http"; + + execute(sql: string, options?: LibSQLExecuteOptions): Promise; + + /** + * Several statements in ONE round trip, each with its own outcome. + * + * Not an optimisation dressed up as a contract: SQLite introspection is + * per-table (`pragma_table_info`, `pragma_index_list`, `pragma_foreign_key_list` + * and a `COUNT(*)` for every table), so a schema read that issues one request + * per statement is four round trips per table - and a libSQL server is normally + * across a network rather than on the filesystem, which is the difference + * between this provider and the SQLite one it shares a dialect with. An + * implementation without a batch protocol may honestly satisfy this by looping. + * + * An empty list answers an empty list without touching the network. + */ + executeBatch(statements: LibSQLStatement[], options?: LibSQLExecuteOptions): Promise; + + /** + * The server's own version string, or null when the deployment publishes none. + * + * Null is a real answer here rather than a failure, and this is the one place + * the two deployments measurably disagree: a self-hosted sqld answers + * `sqld 0.24.33 (f8fb14f3 2026-08-11)`, while Turso Cloud has no such route at + * all (`{"error":"route not found: [\\"version\\"]"}`, measured 2026-08-27). + * The provider renders the absence as absent - never as an invented version and + * never as a failed connection. + */ + serverVersion(): Promise; + + close(): Promise; +} + +/** + * A failure that is the transport's to report: the request never reached a + * libSQL server, or what came back was not a libSQL answer. + * + * A statement the ENGINE rejected is not one of these - it arrives as an HTTP + * 200 carrying the engine's own message, and the provider maps it to a + * `QueryError` so a user sees SQLite's wording rather than a transport failure. + */ +export class LibSQLTransportError extends Error { + public readonly status: number; + /** The engine's own error code where it sent one (`SQLITE_UNKNOWN`), else null. */ + public readonly code: string | null; + + constructor(message: string, status: number, code: string | null = null) { + super(message); + this.name = "LibSQLTransportError"; + this.status = status; + this.code = code; + } +} diff --git a/src/lib/export/result-export.ts b/src/lib/export/result-export.ts index 7f75747a..c3ba1631 100644 --- a/src/lib/export/result-export.ts +++ b/src/lib/export/result-export.ts @@ -384,6 +384,50 @@ const STANDS_ALONE: Record = { "datetimeoffset", "year", ], + // The same list, for the same measured reason: libSQL IS SQLite 3.47.0, and + // `pragma_table_info` there answers the declared spelling verbatim too (measured on + // sqld 0.24.33). Written out rather than aliased so a future divergence can be + // recorded in one row without touching the other. + libsql: [ + "character varying", + "varchar", + "varchar2", + "nvarchar", + "nvarchar2", + "character", + "char", + "nchar", + "text", + "ntext", + "tinytext", + "mediumtext", + "longtext", + "clob", + "nclob", + "binary", + "varbinary", + "raw", + "blob", + "tinyblob", + "mediumblob", + "longblob", + "bytea", + "image", + "numeric", + "decimal", + "number", + "binary_double", + "binary_float", + "money", + "timestamp", + "timestamp without time zone", + "timestamp with time zone", + "datetime", + "datetime2", + "smalldatetime", + "datetimeoffset", + "year", + ], clickhouse: [ "character varying", "varchar", @@ -524,6 +568,10 @@ const BINARY_LITERAL: Record = { // Measured through `bun:sqlite`: `select hex(X'0102deadbeef')` -> `0102DEADBEEF`, // and `typeof(X'')` -> `blob` with `length(X'')` 0. sqlite: "standard-hex", + // Measured over Hrana on sqld 0.24.33: `SELECT hex(X'0102deadbeef')` answers + // `0102DEADBEEF`, `typeof(X'')` answers `blob` and `length(X'')` answers 0 - the + // same readings as the SQLite row above, taken again rather than assumed. + libsql: "standard-hex", // Trino measured on 476: `SELECT typeof(X'0102')` answers `varbinary`, // `to_hex(X'0102deadbeef')` answers `0102DEADBEEF`, `length(X'')` answers 0, and the // whole generated pair replays into the memory connector. Druid is the one row here diff --git a/src/lib/schema-diff/migration-generator.ts b/src/lib/schema-diff/migration-generator.ts index 804bb56d..b793d05b 100644 --- a/src/lib/schema-diff/migration-generator.ts +++ b/src/lib/schema-diff/migration-generator.ts @@ -71,6 +71,17 @@ function clickhouseDefaultKind(value: string): string { * this file spell their engine out the same way. */ const NO_COLUMN_MODIFICATION: Partial> = { + // Measured over Hrana on sqld 0.24.33, and the entry exists because the PostgreSQL + // branch this id would otherwise inherit emits text libSQL cannot parse: + // `ALTER TABLE t ALTER COLUMN c TYPE integer` is "unexpected end of input" and the + // MySQL spelling `MODIFY COLUMN` is "syntax error around `MODIFY`". SQLite has no + // column modification at all, and libSQL is SQLite - unlike DROP COLUMN and RENAME + // COLUMN, which it DOES accept (both measured), so this row is narrower than the + // `sqlite` branch below and deliberately so. + libsql: { + label: "libSQL", + reason: "SQLite cannot retype a column; recreate the table and copy the rows.", + }, couchbase: { label: "Couchbase", reason: "Collections hold schemaless JSON documents, so there is no column definition to change.", @@ -369,6 +380,18 @@ function generateAlterTable(table: TableDiff, dialect: DatabaseType): string { ); return; } + // libSQL refuses the statement below outright: `ALTER TABLE t ADD CONSTRAINT + // fk FOREIGN KEY (c) REFERENCES u(id)` is "near CONSTRAINT ... syntax error" + // (measured on sqld 0.24.33). SQLite has no ALTER that adds a constraint - the + // key has to be part of the CREATE TABLE - so the honest line names the + // recreation. The `sqlite` id has the same limit and still emits the statement + // below; that is its own defect rather than something to copy (BACKLOG D34). + if (dialect === "libsql") { + lines.push( + `-- libSQL: Cannot add a foreign key on ${escapeIdentifier(fk.columnName, dialect)}. SQLite declares one only in CREATE TABLE; recreate the table and copy the rows.`, + ); + return; + } lines.push( `ALTER TABLE ${id} ADD CONSTRAINT ${constraintName} FOREIGN KEY (${escapeIdentifier(fk.columnName, dialect)}) REFERENCES ${escapeIdentifier(fk.targetReferencedTable || "", dialect)}(${escapeIdentifier(fk.targetReferencedColumn || "", dialect)});`, ); @@ -383,6 +406,12 @@ function generateAlterTable(table: TableDiff, dialect: DatabaseType): string { lines.push(`ALTER TABLE ${id} DROP FOREIGN KEY ${constraintName};`); } else if (dialect === "sqlite") { lines.push(`-- SQLite: Cannot drop foreign key directly. Requires table recreation.`); + } else if (dialect === "libsql") { + // The generic `DROP CONSTRAINT` branch below is not parseable here, measured on + // sqld 0.24.33: `ALTER TABLE t DROP CONSTRAINT fk_x` is "near CONSTRAINT … + // syntax error". Same limit as SQLite, named separately so the comment names + // the engine the reader connected to. + lines.push(`-- libSQL: Cannot drop a foreign key directly. Requires table recreation.`); } else if (dialect === "cassandra") { // `DROP CONSTRAINT IF EXISTS fk_x` is "mismatched input 'IF' expecting EOF" // (measured), and dropping what was never declarable is not a statement. @@ -412,7 +441,10 @@ export function generateMigrationSQL(diff: SchemaDiff, dialect: DatabaseType): s // to be two independent conditions, which is a shape that can diverge into a `BEGIN;` // with no `COMMIT;`. // - // SQLite runs its own transaction; Cassandra has no transaction at all. Measured on + // SQLite runs its own transaction, and libSQL is SQLite - the same reasoning, with + // one addition of its own: this provider closes its Hrana stream in the same request + // as each statement, so a BEGIN it emitted could not be continued by the app that + // generated the file. Cassandra has no transaction at all. Measured on // 5.0.9: `BEGIN;` is "line 1:5 mismatched input ';' expecting K_BATCH" and `COMMIT;` // is "no viable alternative at input 'COMMIT'". The only grouping CQL has is // `BEGIN BATCH ... APPLY BATCH`, which is not a transaction and takes no DDL, so @@ -420,7 +452,7 @@ export function generateMigrationSQL(diff: SchemaDiff, dialect: DatabaseType): s // wrapper is still wrong for the other engines named in the module docstring; that // stays tracked there, and unlike them Cassandra emits DDL this generator was taught // to spell correctly, so the wrapper would be the only unrunnable line in it. - const wrapsInTransaction = dialect !== "sqlite" && dialect !== "cassandra"; + const wrapsInTransaction = dialect !== "sqlite" && dialect !== "libsql" && dialect !== "cassandra"; if (wrapsInTransaction) { sections.push("BEGIN;"); diff --git a/src/lib/seed/types.ts b/src/lib/seed/types.ts index a5296e21..c1e433c5 100644 --- a/src/lib/seed/types.ts +++ b/src/lib/seed/types.ts @@ -40,6 +40,7 @@ const SeedDatabaseType = z.enum([ "opensearch", "trino", "cassandra", + "libsql", ]); export const SeedDefaultsSchema = z.object({ diff --git a/src/lib/sql/fence-tags.ts b/src/lib/sql/fence-tags.ts index 1d101f9d..c4ec4349 100644 --- a/src/lib/sql/fence-tags.ts +++ b/src/lib/sql/fence-tags.ts @@ -34,6 +34,10 @@ const ENGINE_FENCE_TAGS: Readonly> = Object.freeze({ postgres: true, mysql: true, sqlite: true, + // A block tagged `libsql` holds SQLite's own dialect: same tokenizer, same + // statements, measured on sqld 0.24.33 (SQLite 3.47.0). `turso` is registered as + // an alias below because that is the product name a model is likelier to write. + libsql: true, mongodb: true, redis: true, oracle: true, @@ -79,6 +83,7 @@ const QUERY_FENCE_ALIASES: ReadonlySet = new Set([ "plpgsql", "mariadb", "sqlite3", + "turso", "plsql", "tsql", "sqlserver", @@ -100,6 +105,7 @@ const ALIAS_ENGINES: Readonly> = Object.freeze({ plpgsql: "postgres", mariadb: "mysql", sqlite3: "sqlite", + turso: "libsql", plsql: "oracle", tsql: "mssql", sqlserver: "mssql", diff --git a/src/lib/sql/grammar.ts b/src/lib/sql/grammar.ts index 794dfe12..4ea372cb 100644 --- a/src/lib/sql/grammar.ts +++ b/src/lib/sql/grammar.ts @@ -539,6 +539,14 @@ const SQL_GRAMMARS: Partial> = { oracle: ORACLE_GRAMMAR, mssql: MSSQL_GRAMMAR, sqlite: SQLITE_GRAMMAR, + // The SAME grammar object, and every one of its four facts was re-measured over + // Hrana on sqld 0.24.33 rather than inherited: `SELECT 1 # x` is refused ("bad + // variable name", so `#` is no comment), `SELECT [id] FROM probe_customers` parses + // (so brackets quote an identifier), `/* outer /* inner */ SELECT 1` RUNS (so block + // comments do not nest), and `q'[x]'` is a syntax error. Sharing the object rather + // than declaring a second identical one is deliberate: a divergence would then have + // to be written down as its own grammar, which is the change a reader should see. + libsql: SQLITE_GRAMMAR, elasticsearch: ELASTICSEARCH_GRAMMAR, opensearch: OPENSEARCH_GRAMMAR, trino: TRINO_GRAMMAR, diff --git a/src/lib/sql/values.ts b/src/lib/sql/values.ts index 28957d3d..a28d4ed7 100644 --- a/src/lib/sql/values.ts +++ b/src/lib/sql/values.ts @@ -20,6 +20,9 @@ const LITERAL_ESCAPE: Record = { // a backslash in a plain literal is data. postgres: "standard", sqlite: "standard", + // Measured on sqld 0.24.33: `SELECT 'it''s'` answers `it's`, and a backslash has no + // special meaning - the same standard doubling SQLite defines. + libsql: "standard", oracle: "standard", mssql: "standard", // Druid quotes a string with single quotes and puts its backslash escapes in the diff --git a/src/lib/types.ts b/src/lib/types.ts index bd5898a1..4dce9bd2 100644 --- a/src/lib/types.ts +++ b/src/lib/types.ts @@ -32,7 +32,23 @@ export type DatabaseType = // schema level. PrestoDB is deliberately NOT this id - the transport builds its // header names from a dialect descriptor's prefix, so that fork is a descriptor away // rather than a rewrite. - | "trino"; + | "trino" + // libSQL (issue #424 Phase 5). SQLite's dialect over a network: a self-hosted + // libSQL server (`sqld`) and Turso Cloud are the SAME id, because they speak the + // same protocol and embed the same SQLite - the cloud is that server managed, and + // a connection to either differs only in host and token. It is separate from + // `sqlite` for the reason the two cannot share a provider: the SQLite one holds a + // FILE handle through a synchronous driver, and this one holds no handle at all. + // The credential is a token rather than a password, so the form labels it that + // way, and the server refuses `VACUUM`, `ANALYZE` and `PRAGMA query_only` - which + // is why this id offers fewer maintenance operations than `sqlite` does. + // + // Turso Database, the Rust rewrite, is NOT this id and has no row anywhere yet: it + // publishes no server image (`tursodatabase/turso`, `tursodb` and `turso-server` + // were all unpullable on 2026-08-27) and ships as an in-process npm engine, so + // there is nothing to connect to and #424 publishes no name it has not connected + // to. + | "libsql"; export type ConnectionEnvironment = "production" | "staging" | "development" | "local" | "other"; diff --git a/tests/components/ConnectionModal.test.tsx b/tests/components/ConnectionModal.test.tsx index 8d259118..ec630088 100644 --- a/tests/components/ConnectionModal.test.tsx +++ b/tests/components/ConnectionModal.test.tsx @@ -794,6 +794,32 @@ describe("ConnectionModal", () => { expect(queryByText(/refuses a password over plain HTTP/)).toBeNull(); }); + // ── 34b-bis. libSQL asks for a TOKEN, and says where one comes from ─────── + // + // libSQL has no user names at all: the credential a server checks is a JWT it + // minted, so the shared `password` field holds a token here. A field labelled + // Password invites a password no libSQL server has, and a self-hosted server + // started without authentication takes none at all - both measured on sqld 0.24.33 + // and on Turso Cloud, 2026-08-27. + + test("libSQL type labels the password field Auth Token and says where one comes from", () => { + mockFormOverrides = { type: "libsql" }; + const props = createDefaultProps(); + const { queryByText } = render(React.createElement(ConnectionModal, props)); + + expect(queryByText("Auth Token")).not.toBeNull(); + expect(queryByText("Password")).toBeNull(); + expect(queryByText(/turso db tokens create/)).not.toBeNull(); + }); + + test("no other type is asked for an Auth Token", () => { + const props = createDefaultProps(); + const { queryByText } = render(React.createElement(ConnectionModal, props)); + + expect(queryByText("Auth Token")).toBeNull(); + expect(queryByText(/turso db tokens create/)).toBeNull(); + }); + // ── 34c. Cassandra asks for the one field its driver cannot start without ── // // `cassandra-driver` 4.9.0 refuses to connect with no local data centre at all diff --git a/tests/hooks/use-connection-form.test.ts b/tests/hooks/use-connection-form.test.ts index dc2550c4..625dba7d 100644 --- a/tests/hooks/use-connection-form.test.ts +++ b/tests/hooks/use-connection-form.test.ts @@ -1045,6 +1045,7 @@ describe("useConnectionForm", () => { opensearch: true, trino: true, cassandra: true, + libsql: true, }; test("dbTypes offers every database type a connection can carry", () => { diff --git a/tests/integration/db/libsql-provider.test.ts b/tests/integration/db/libsql-provider.test.ts new file mode 100644 index 00000000..2ea9b72c --- /dev/null +++ b/tests/integration/db/libsql-provider.test.ts @@ -0,0 +1,648 @@ +/** + * libSQL provider, end to end (issue #424 Phase 5) + * + * Every payload below was captured on 2026-08-27 from BOTH deployments this one + * type-id reaches: a self-hosted `ghcr.io/tursodatabase/libsql-server` (sqld + * 0.24.33, SQLite 3.47.0) and a Turso Cloud database in `aws-eu-west-1`. + * `globalThis.fetch` is REPLACED per test and restored afterwards - `mock.module()` + * is refused, being process-wide in bun and able to poison sibling files - so the + * real provider, the real introspection and the real Hrana transport all execute + * here and only the server is fake. + * + * Five measured behaviours drive what is asserted: + * + * 1. A FAILED STATEMENT IS AN HTTP 200 with the failure inside `results[]`, so + * `response.ok` is never the test. + * 2. THE TWO DEPLOYMENTS WORD THE SAME REFUSAL DIFFERENTLY - "unsupported + * statement: VACUUM" against "SQL not allowed statement: VACUUM" - under one + * code, so nothing may key on the wording. + * 3. `GET /version` IS A SQLD ROUTE TURSO CLOUD DOES NOT HAVE, and a deployment + * that publishes no version is not a broken one. + * 4. AN AUTH FAILURE USES A DIFFERENT ENVELOPE (`{"error": ""}`) and + * answers 401 with no token, 400 with a malformed one. + * 5. `dbstat` ANSWERS ON BOTH, so per-table bytes here are measured - which + * `bun:sqlite` cannot do at all. + */ +import { afterEach, beforeEach, describe, expect, test } from "bun:test"; +import { AuthenticationError, ConnectionError, DatabaseConfigError, QueryError } from "@/lib/db/errors"; +import { LibSQLProvider } from "@/lib/db/providers/sql/libsql"; +import type { DatabaseConnection } from "@/lib/db/types"; + +// ============================================================================ +// Harness +// ============================================================================ + +interface FetchCall { + url: string; + body: string | null; +} + +const originalFetch = globalThis.fetch; +let calls: FetchCall[] = []; + +/** A row set as Hrana encodes one: declared columns, positional typed values. */ +type Cell = Record; + +function result(cols: [string, string | null][], rows: Cell[][], extra: Record = {}): Cell { + return { + type: "ok", + response: { + type: "execute", + result: { + cols: cols.map(([name, decltype]) => ({ name, decltype })), + rows, + affected_row_count: 0, + last_insert_rowid: null, + replication_index: "1", + rows_read: rows.length, + rows_written: 0, + query_duration_ms: 0.107, + ...extra, + }, + }, + }; +} + +function text(value: string): Cell { + return { type: "text", value }; +} + +function int(value: number | string): Cell { + return { type: "integer", value: String(value) }; +} + +function failure(message: string, code: string): Cell { + return { type: "error", error: { message, code } }; +} + +/** + * The fixture both deployments answered for, verbatim in shape: two tables, one + * user index, one foreign key, `dbstat` populated. + */ +function answerFor(sql: string): Cell { + if (/FROM sqlite_master\s+WHERE type = 'table'\s+AND name NOT LIKE/.test(sql) && /COUNT/.test(sql)) { + return result([["table_count", null]], [[int(2)]]); + } + if (/FROM sqlite_master\s+WHERE type = 'table'/.test(sql)) { + return result([["name", "TEXT"]], [[text("probe_customers")], [text("probe_orders")]]); + } + if (/type = 'index' AND name NOT LIKE/.test(sql)) return result([["index_count", null]], [[int(1)]]); + if (/SELECT name, tbl_name FROM sqlite_master WHERE type = 'index'/.test(sql)) { + return result( + [ + ["name", "TEXT"], + ["tbl_name", "TEXT"], + ], + [[text("idx_country"), text("probe_customers")]], + ); + } + if (/FROM dbstat/.test(sql)) { + return result( + [ + ["name", "TEXT"], + ["bytes", null], + ], + [ + [text("probe_customers"), int(8192)], + [text("idx_country"), int(4096)], + [text("probe_orders"), int(270336)], + ], + ); + } + if (/COUNT\(\*\) AS row_count FROM "probe_customers"/.test(sql)) return result([["row_count", null]], [[int(3)]]); + if (/COUNT\(\*\) AS row_count FROM "probe_orders"/.test(sql)) return result([["row_count", null]], [[int(2000)]]); + if (/pragma_table_info\('probe_customers'\)/.test(sql)) { + return result( + [ + ["cid", null], + ["name", null], + ["type", null], + ["notnull", null], + ["dflt_value", null], + ["pk", null], + ], + [ + [int(0), text("id"), text("INTEGER"), int(1), { type: "null" }, int(1)], + [int(1), text("country"), text("TEXT"), int(0), { type: "null" }, int(0)], + ], + ); + } + if (/pragma_table_info\('probe_orders'\)/.test(sql)) { + return result( + [ + ["cid", null], + ["name", null], + ["type", null], + ["notnull", null], + ["dflt_value", null], + ["pk", null], + ], + [[int(0), text("id"), text("INTEGER"), int(0), { type: "null" }, int(1)]], + ); + } + if (/pragma_index_list\('probe_customers'\)/.test(sql)) { + return result( + [ + ["seq", null], + ["name", null], + ["unique", null], + ["origin", null], + ], + [ + [int(0), text("idx_country"), int(1), text("c")], + [int(1), text("sqlite_autoindex_probe_customers_1"), int(1), text("pk")], + ], + ); + } + if (/pragma_index_info\('idx_country'\)/.test(sql)) { + return result( + [ + ["seqno", null], + ["cid", null], + ["name", null], + ], + [[int(0), int(1), text("country")]], + ); + } + if (/pragma_foreign_key_list\('probe_orders'\)/.test(sql)) { + return result( + [ + ["id", null], + ["seq", null], + ["table", null], + ["from", null], + ["to", null], + ], + [[int(0), int(0), text("probe_customers"), text("customer_id"), text("id")]], + ); + } + if (/pragma_page_count/.test(sql)) return result([["size_bytes", null]], [[int(282624)]]); + if (/sqlite_version\(\)/.test(sql)) return result([["version", null]], [[text("3.47.0")]]); + if (/PRAGMA integrity_check/.test(sql)) return result([["integrity_check", null]], [[text("ok")]]); + if (/PRAGMA journal_mode/.test(sql)) return result([["journal_mode", null]], [[text("wal")]]); + if (/^SELECT 1$/.test(sql)) return result([["1", null]], [[int(1)]]); + + return result([], []); +} + +/** What the server does with one pipeline: an answer per statement, then a close. */ +type Server = (sql: string) => Cell; + +let server: Server = answerFor; +let versionRoute: () => Response = () => new Response("sqld 0.24.33 (f8fb14f3 2026-08-11)", { status: 200 }); + +function installFetch(): void { + globalThis.fetch = ((input: RequestInfo | URL, init?: RequestInit) => { + const url = String(input); + const body = init?.body === undefined ? null : String(init.body); + calls.push({ url, body }); + + if (url.endsWith("/version")) return Promise.resolve(versionRoute()); + + const requests = (JSON.parse(body ?? "{}") as { requests?: { type: string; stmt?: { sql?: string } }[] }).requests; + const results = (requests ?? []) + .filter((request) => request.type === "execute") + .map((request) => server(String(request.stmt?.sql))); + + return Promise.resolve( + new Response(JSON.stringify({ baton: null, base_url: null, results: [...results, { type: "ok" }] }), { + status: 200, + headers: { "content-type": "application/json" }, + }), + ); + }) as unknown as typeof fetch; +} + +function connection(overrides: Partial = {}): DatabaseConnection { + return { + id: "libsql-probe", + name: "libSQL probe", + type: "libsql", + host: "127.0.0.1", + port: 18081, + createdAt: new Date("2026-08-27T00:00:00.000Z"), + ...overrides, + }; +} + +async function connected(overrides: Partial = {}): Promise { + const provider = new LibSQLProvider(connection(overrides)); + await provider.connect(); + return provider; +} + +/** Every statement the provider sent, in order. */ +function sentStatements(): string[] { + return calls + .filter((call) => call.body !== null) + .flatMap((call) => { + const requests = (JSON.parse(call.body as string) as { requests?: { stmt?: { sql?: string } }[] }).requests ?? []; + return requests.filter((request) => request.stmt !== undefined).map((request) => String(request.stmt?.sql)); + }); +} + +beforeEach(() => { + calls = []; + server = answerFor; + versionRoute = () => new Response("sqld 0.24.33 (f8fb14f3 2026-08-11)", { status: 200 }); + installFetch(); +}); + +afterEach(() => { + globalThis.fetch = originalFetch; +}); + +// ============================================================================ +// Configuration and connection +// ============================================================================ + +describe("LibSQLProvider configuration", () => { + test("refuses a connection with neither a host nor a URL", () => { + expect(() => new LibSQLProvider(connection({ host: undefined }))).toThrow(DatabaseConfigError); + }); + + test("resolves a libsql:// URL into host, port and token", async () => { + const provider = await connected({ + host: undefined, + port: undefined, + connectionString: "libsql://libredb-probe-424-cevheri.aws-eu-west-1.turso.io?authToken=jwt-123", + }); + + expect(calls[0]?.url).toBe("https://libredb-probe-424-cevheri.aws-eu-west-1.turso.io:443/v2/pipeline"); + await provider.disconnect(); + }); + + test("connects with the cheapest statement there is, not a health route", async () => { + await connected(); + + expect(sentStatements()).toEqual(["SELECT 1"]); + }); + + test("connecting twice reuses the transport rather than probing again", async () => { + const provider = await connected(); + await provider.connect(); + + expect(sentStatements()).toEqual(["SELECT 1"]); + await provider.disconnect(); + }); + + test("reports a missing token as an authentication failure, not a connection one", async () => { + // 401, and the envelope is `{"error": ""}` rather than the statement + // shape - captured from Turso Cloud with no Authorization header. + globalThis.fetch = (() => + Promise.resolve( + new Response( + JSON.stringify({ error: "Unauthorized: `unauthorized access attempt on database: empty JWT token`" }), + { + status: 401, + }, + ), + )) as unknown as typeof fetch; + + await expect(new LibSQLProvider(connection()).connect()).rejects.toBeInstanceOf(AuthenticationError); + }); + + test("reports a malformed token as an authentication failure even though the status is 400", async () => { + globalThis.fetch = (() => + Promise.resolve( + new Response(JSON.stringify({ error: "JWT error: InvalidToken" }), { status: 400 }), + )) as unknown as typeof fetch; + + await expect(new LibSQLProvider(connection({ password: "notatoken" })).connect()).rejects.toBeInstanceOf( + AuthenticationError, + ); + }); + + test("reports an unreachable server as a connection failure naming host and port", async () => { + globalThis.fetch = (() => Promise.reject(new Error("connect ECONNREFUSED"))) as unknown as typeof fetch; + + const failed = new LibSQLProvider(connection()).connect(); + + await expect(failed).rejects.toBeInstanceOf(ConnectionError); + await expect(failed).rejects.toThrow(/connect ECONNREFUSED/); + }); + + test("refuses every read before connect, rather than answering an empty one", async () => { + const provider = new LibSQLProvider(connection()); + + await expect(provider.query("SELECT 1")).rejects.toThrow(); + await expect(provider.getSchema()).rejects.toThrow(); + await expect(provider.getOverview()).rejects.toThrow(); + }); +}); + +// ============================================================================ +// Capabilities +// ============================================================================ + +describe("LibSQLProvider capabilities", () => { + test("offers only the two maintenance operations the server accepts", () => { + const capabilities = new LibSQLProvider(connection()).getCapabilities(); + + expect(capabilities.maintenanceOperations).toEqual(["reindex", "check"]); + // Measured on BOTH deployments: each of these is refused by the server's own + // statement allowlist, so a control for it could only ever fail. + expect(capabilities.maintenanceOperations).not.toContain("vacuum"); + expect(capabilities.maintenanceOperations).not.toContain("analyze"); + }); + + test("reads EXPLAIN the way SQLite does, and takes a pasted URL", () => { + const capabilities = new LibSQLProvider(connection()).getCapabilities(); + + expect(capabilities.explainFormat).toBe("sqlite-queryplan"); + expect(capabilities.supportsConnectionString).toBe(true); + expect(capabilities.defaultPort).toBe(8080); + }); + + test("declares no transaction, because the stream closes with each statement", () => { + expect(new LibSQLProvider(connection()).getCapabilities().supportsTransactions).toBe(false); + }); + + test("tells the reader there is no statement history to enable", () => { + expect(new LibSQLProvider(connection()).getLabels().slowQueriesEmptyState).toBe( + "libSQL keeps no statistics about finished statements, so there is nothing to enable.", + ); + }); +}); + +// ============================================================================ +// Queries +// ============================================================================ + +describe("LibSQLProvider query", () => { + test("returns rows, declared fields and the types the engine declared", async () => { + const provider = await connected(); + + const answer = await provider.query("SELECT name, type FROM sqlite_master WHERE type = 'table'"); + + expect(answer.rows).toEqual([{ name: "probe_customers" }, { name: "probe_orders" }]); + expect(answer.fields).toEqual(["name"]); + expect(answer.columnTypes).toEqual({ name: "TEXT" }); + await provider.disconnect(); + }); + + test("omits columnTypes entirely when the engine declared none", async () => { + const provider = await connected(); + + const answer = await provider.query("SELECT sqlite_version() AS version"); + + expect(answer.rows).toEqual([{ version: "3.47.0" }]); + expect(answer.columnTypes).toBeUndefined(); + await provider.disconnect(); + }); + + test("counts a write by what the engine says it changed", async () => { + server = () => result([], [], { affected_row_count: 3, last_insert_rowid: "12" }); + const provider = await connected(); + + const answer = await provider.query("DELETE FROM probe_orders WHERE id < 4"); + + expect(answer.rowCount).toBe(3); + expect(answer.rows).toEqual([]); + await provider.disconnect(); + }); + + test("binds positional parameters as the protocol requires", async () => { + const provider = await connected(); + + await provider.query("SELECT * FROM probe_customers WHERE country = ?", ["tr"]); + + const sent = JSON.parse(calls[1]?.body ?? "{}") as { requests: { stmt?: { args?: unknown[] } }[] }; + expect(sent.requests[0]?.stmt?.args).toEqual([{ type: "text", value: "tr" }]); + await provider.disconnect(); + }); + + test("surfaces SQLite's own wording for a statement the engine rejected", async () => { + const provider = await connected(); + server = () => failure("SQLite error: no such table: nope", "SQLITE_UNKNOWN"); + + const failed = provider.query("SELECT * FROM nope"); + + await expect(failed).rejects.toBeInstanceOf(QueryError); + await expect(failed).rejects.toThrow("SQLite error: no such table: nope"); + await provider.disconnect(); + }); + + test("surfaces a refusal the same way whichever deployment worded it", async () => { + // sqld says "unsupported statement"; Turso Cloud says "SQL not allowed + // statement". Both are SQL_PARSE_ERROR and both must reach the user verbatim. + for (const message of [ + "SQL string could not be parsed: unsupported statement: VACUUM", + "SQL not allowed statement: VACUUM", + ]) { + server = answerFor; + const provider = await connected(); + server = () => failure(message, "SQL_PARSE_ERROR"); + + await expect(provider.query("VACUUM")).rejects.toThrow(message); + await provider.disconnect(); + } + }); +}); + +// ============================================================================ +// Schema +// ============================================================================ + +describe("LibSQLProvider getSchema", () => { + test("reads both tables with their columns, indexes, keys, counts and measured sizes", async () => { + const provider = await connected(); + + const schema = await provider.getSchema(); + + expect(schema.map((table) => table.name)).toEqual(["probe_customers", "probe_orders"]); + expect(schema[0]?.rowCount).toBe(3); + expect(schema[0]?.size).toBe("12 KB"); + expect(schema[0]?.columns).toEqual([ + { name: "id", type: "INTEGER", nullable: false, isPrimary: true }, + { name: "country", type: "TEXT", nullable: true, isPrimary: false }, + ]); + expect(schema[0]?.indexes).toEqual([{ name: "idx_country", columns: ["country"], unique: true }]); + expect(schema[1]?.foreignKeys).toEqual([ + { columnName: "customer_id", referencedTable: "probe_customers", referencedColumn: "id" }, + ]); + await provider.disconnect(); + }); + + test("reads the whole tree in three round trips, not four per table", async () => { + const provider = await connected(); + calls = []; + + await provider.getSchema(); + + // The object list, one batch of four statements per table, one batch for the + // single user index, and one for the two size reads. + expect(calls).toHaveLength(4); + await provider.disconnect(); + }); + + test("keeps every other table when one table's column read fails", async () => { + server = (sql) => + /pragma_table_info\('probe_customers'\)/.test(sql) + ? failure("SQLite error: no such table: probe_customers", "SQLITE_UNKNOWN") + : answerFor(sql); + const provider = await connected(); + + const schema = await provider.getSchema(); + + expect(schema).toHaveLength(2); + expect(schema[0]?.columns).toEqual([]); + expect(schema[1]?.columns).toHaveLength(1); + await provider.disconnect(); + }); + + test("omits every size when dbstat is missing, and keeps the row counts", async () => { + server = (sql) => + /FROM dbstat/.test(sql) ? failure("SQLite error: no such table: dbstat", "SQLITE_UNKNOWN") : answerFor(sql); + const provider = await connected(); + + const schema = await provider.getSchema(); + + expect(schema[0]?.size).toBeUndefined(); + expect(schema[0]?.rowCount).toBe(3); + await provider.disconnect(); + }); +}); + +// ============================================================================ +// Monitoring +// ============================================================================ + +describe("LibSQLProvider monitoring", () => { + test("names the server version and the SQLite it embeds", async () => { + const provider = await connected(); + + const overview = await provider.getOverview(); + + expect(overview.version).toBe("sqld 0.24.33 (f8fb14f3 2026-08-11) (SQLite 3.47.0)"); + expect(overview.databaseSize).toBe("276 KB"); + expect(overview.tableCount).toBe(2); + expect(overview.indexCount).toBe(1); + expect(overview.maxConnections).toBe(0); + await provider.disconnect(); + }); + + test("shows the SQLite version alone on Turso Cloud, where /version does not exist", async () => { + versionRoute = () => new Response(JSON.stringify({ error: 'route not found: ["version"]' }), { status: 404 }); + const provider = await connected(); + + expect((await provider.getOverview()).version).toBe("SQLite 3.47.0"); + await provider.disconnect(); + }); + + test("reports the integrity check and journal mode, and no invented cache ratio", async () => { + const provider = await connected(); + + const health = await provider.getHealth(); + + expect(health.databaseSize).toBe("276 KB"); + expect(health.cacheHitRatio).toBe("N/A"); + expect(health.slowQueries.map((entry) => entry.query)).toEqual(["Integrity: OK", "Journal Mode: wal"]); + expect(health.activeSessions).toEqual([]); + await provider.disconnect(); + }); + + test("counts zero deadlocks as a fact about the engine and measures nothing else", async () => { + const provider = await connected(); + + expect(await provider.getPerformanceMetrics()).toEqual({ deadlocks: 0 }); + expect(await provider.getSlowQueries()).toEqual([]); + expect(await provider.getActiveSessions()).toEqual([]); + await provider.disconnect(); + }); + + test("splits measured pages between tables and their indexes", async () => { + const provider = await connected(); + + const stats = await provider.getTableStats(); + + expect(stats[0]).toMatchObject({ tableName: "probe_customers", tableSizeBytes: 8192, indexSizeBytes: 4096 }); + expect(stats[1]).toMatchObject({ tableName: "probe_orders", rowCount: 2000, totalSizeBytes: 270336 }); + await provider.disconnect(); + }); + + test("reports the user index with its columns and measured bytes", async () => { + const provider = await connected(); + + expect(await provider.getIndexStats()).toEqual([ + { + schemaName: "main", + tableName: "probe_customers", + indexName: "idx_country", + columns: ["country"], + isUnique: true, + isPrimary: false, + indexSize: "4 KB", + indexSizeBytes: 4096, + scans: 0, + }, + ]); + await provider.disconnect(); + }); + + test("reports the one database as storage, from its own page counters", async () => { + const provider = await connected(); + + expect(await provider.getStorageStats()).toEqual([{ name: "main", size: "276 KB", sizeBytes: 282624 }]); + await provider.disconnect(); + }); +}); + +// ============================================================================ +// Maintenance +// ============================================================================ + +describe("LibSQLProvider runMaintenance", () => { + test("runs a bare REINDEX, and a targeted one against the named table", async () => { + const provider = await connected(); + calls = []; + + expect(await provider.runMaintenance("reindex")).toMatchObject({ success: true }); + expect(await provider.runMaintenance("reindex", "probe_customers")).toMatchObject({ success: true }); + + expect(sentStatements()).toEqual(["REINDEX", 'REINDEX "probe_customers"']); + await provider.disconnect(); + }); + + test("reads the integrity check's ANSWER rather than its status", async () => { + const provider = await connected(); + + expect(await provider.runMaintenance("check")).toMatchObject({ success: true, message: "ok" }); + await provider.disconnect(); + }); + + test("reports a corrupt database as a failed check, even though the statement succeeded", async () => { + server = (sql) => + /integrity_check/.test(sql) + ? result([["integrity_check", null]], [[text("*** in database main ***")]]) + : answerFor(sql); + const provider = await connected(); + + expect(await provider.runMaintenance("check")).toMatchObject({ + success: false, + message: "*** in database main ***", + }); + await provider.disconnect(); + }); + + test("refuses VACUUM here rather than relaying the server's refusal", async () => { + const provider = await connected(); + calls = []; + + const refused = provider.runMaintenance("vacuum"); + + await expect(refused).rejects.toThrow(/do not accept VACUUM/); + // Nothing was sent: the refusal is ours, so the user is not told about a + // statement they never asked for. + expect(sentStatements()).toEqual([]); + await provider.disconnect(); + }); + + test("refuses ANALYZE and OPTIMIZE for the same measured reason", async () => { + const provider = await connected(); + + await expect(provider.runMaintenance("analyze")).rejects.toThrow(/do not accept ANALYZE/); + await expect(provider.runMaintenance("optimize")).rejects.toThrow(/do not accept OPTIMIZE/); + await provider.disconnect(); + }); +}); diff --git a/tests/unit/db/libsql/hrana-transport.test.ts b/tests/unit/db/libsql/hrana-transport.test.ts new file mode 100644 index 00000000..66b40da3 --- /dev/null +++ b/tests/unit/db/libsql/hrana-transport.test.ts @@ -0,0 +1,787 @@ +/** + * libSQL Hrana HTTP transport (issue #424 Phase 5) + * + * globalThis.fetch is replaced per test and restored in afterEach. mock.module() + * is deliberately not used: it is process-wide in bun, so mocking a module here + * would poison every sibling test file sharing the process. + * + * Every envelope replayed below was captured verbatim on 2026-08-27 from BOTH + * deployments - a self-hosted `ghcr.io/tursodatabase/libsql-server` (sqld 0.24.33, + * SQLite 3.47.0) and a Turso Cloud database - including the shapes a hand-written + * client gets wrong: + * + * - a failed statement answers HTTP **200** with the error inside `results[]`, + * - an auth failure answers 401 (no token) or 400 (bad token) with a DIFFERENT + * envelope whose `error` is a bare string rather than `{ message, code }`, + * - every integer arrives as a decimal STRING, so a naive read rounds a 64-bit + * rowid, + * - `GET /version` exists on sqld and does not exist on Turso Cloud. + */ +import { afterEach, beforeEach, describe, expect, test } from "bun:test"; +import { LibSQLHranaTransport } from "@/lib/db/providers/sql/libsql/hrana-transport"; +import { LibSQLTransportError } from "@/lib/db/providers/sql/libsql/transport"; +import type { DatabaseConnection, DatabaseType } from "@/lib/db/types"; + +// ============================================================================ +// Harness +// ============================================================================ + +// The DatabaseType union gains "libsql" in the registration commit; the double +// assertion keeps this file compiling on either side of that change. +const LIBSQL: DatabaseType = "libsql" as unknown as DatabaseType; + +interface FetchCall { + url: string; + init: RequestInit | undefined; +} + +const originalFetch = globalThis.fetch; +let calls: FetchCall[] = []; +let handler: (url: string, init?: RequestInit) => Response; + +function respond(body: unknown, init: { status?: number } = {}): Response { + return new Response(typeof body === "string" ? body : JSON.stringify(body), { + status: init.status ?? 200, + headers: { "content-type": "application/json" }, + }); +} + +/** The `ok` wrapper every successful pipeline step carries. */ +function okStep(result: Record): Record { + return { type: "ok", response: { type: "execute", result } }; +} + +/** A captured result for `SELECT name, type FROM sqlite_master`. */ +function masterResult(): Record { + return { + cols: [ + { name: "name", decltype: "TEXT" }, + { name: "type", decltype: "TEXT" }, + ], + rows: [ + [ + { type: "text", value: "probe_customers" }, + { type: "text", value: "table" }, + ], + ], + affected_row_count: 0, + last_insert_rowid: null, + replication_index: "1", + rows_read: 1, + rows_written: 0, + query_duration_ms: 0.107, + }; +} + +function pipeline(...steps: Record[]): Record { + return { baton: null, base_url: null, results: [...steps, { type: "ok", response: { type: "close" } }] }; +} + +function connection(overrides: Partial = {}): DatabaseConnection { + return { + id: "libsql-1", + name: "libSQL probe", + type: LIBSQL, + host: "127.0.0.1", + port: 18081, + createdAt: new Date("2026-08-27T00:00:00.000Z"), + ...overrides, + }; +} + +function transport(overrides: Partial = {}): LibSQLHranaTransport { + return new LibSQLHranaTransport(connection(overrides)); +} + +/** The parsed request body of the nth fetch call. */ +function sentBody(index = 0): Record { + const raw = calls[index]?.init?.body; + return JSON.parse(String(raw)) as Record; +} + +/** The `sql` of the first statement in the nth request. */ +function sentSql(index = 0): string { + const requests = sentBody(index).requests as { type: string; stmt?: { sql?: string } }[]; + return String(requests[0]?.stmt?.sql); +} + +/** The `args` of the first statement in the nth request. */ +function sentArgs(index = 0): Record[] { + const requests = sentBody(index).requests as { stmt?: { args?: Record[] } }[]; + return requests[0]?.stmt?.args ?? []; +} + +beforeEach(() => { + calls = []; + handler = () => respond(pipeline(okStep(masterResult()))); + globalThis.fetch = ((input: RequestInfo | URL, init?: RequestInit) => { + calls.push({ url: String(input), init }); + return Promise.resolve(handler(String(input), init)); + }) as unknown as typeof fetch; +}); + +afterEach(() => { + globalThis.fetch = originalFetch; +}); + +// ============================================================================ +// Endpoint +// ============================================================================ + +describe("LibSQLHranaTransport endpoint", () => { + test("posts the pipeline to the plain-HTTP origin built from host and port", async () => { + await transport().execute("SELECT 1"); + + expect(calls).toHaveLength(1); + expect(calls[0]?.url).toBe("http://127.0.0.1:18081/v2/pipeline"); + expect(calls[0]?.init?.method).toBe("POST"); + }); + + test("uses https and the TLS default port when the connection carries ssl", async () => { + await transport({ + host: "libredb-probe-424-cevheri.aws-eu-west-1.turso.io", + port: undefined, + ssl: { mode: "require" }, + }).execute("SELECT 1"); + + expect(calls[0]?.url).toBe("https://libredb-probe-424-cevheri.aws-eu-west-1.turso.io:443/v2/pipeline"); + }); + + test("brackets an IPv6 literal so the URL stays parseable", async () => { + await transport({ host: "::1" }).execute("SELECT 1"); + + expect(calls[0]?.url).toBe("http://[::1]:18081/v2/pipeline"); + }); + + test("sends the auth token as a bearer credential", async () => { + await transport({ password: "tok-123" }).execute("SELECT 1"); + + const headers = new Headers(calls[0]?.init?.headers); + expect(headers.get("authorization")).toBe("Bearer tok-123"); + }); + + test("sends no authorization header at all when the connection carries no token", async () => { + await transport().execute("SELECT 1"); + + const headers = new Headers(calls[0]?.init?.headers); + expect(headers.has("authorization")).toBe(false); + }); +}); + +// ============================================================================ +// Statements and results +// ============================================================================ + +describe("LibSQLHranaTransport execute", () => { + test("closes the connection in the same pipeline as the statement", async () => { + await transport().execute("SELECT 1"); + + const requests = sentBody().requests as { type: string }[]; + expect(requests.map((r) => r.type)).toEqual(["execute", "close"]); + }); + + test("maps declared columns onto object rows and keeps the declared order", async () => { + const result = await transport().execute("SELECT name, type FROM sqlite_master"); + + expect(result.fieldNames).toEqual(["name", "type"]); + expect(result.rows).toEqual([{ name: "probe_customers", type: "table" }]); + expect(result.columnTypes).toEqual({ name: "TEXT", type: "TEXT" }); + }); + + test("reports an engine that declared no column types as an empty record, not a guess", async () => { + handler = () => + respond( + pipeline( + okStep({ + cols: [{ name: "v", decltype: null }], + rows: [[{ type: "text", value: "3.47.0" }]], + affected_row_count: 0, + last_insert_rowid: null, + rows_read: 0, + rows_written: 0, + query_duration_ms: 0.022, + }), + ), + ); + + const result = await transport().execute("SELECT sqlite_version() AS v"); + + expect(result.columnTypes).toEqual({}); + expect(result.rows).toEqual([{ v: "3.47.0" }]); + }); + + test("carries the engine's own duration and affected-row count", async () => { + handler = () => + respond( + pipeline( + okStep({ + cols: [], + rows: [], + affected_row_count: 2, + last_insert_rowid: "7", + rows_read: 1, + rows_written: 2, + query_duration_ms: 5.014, + }), + ), + ); + + const result = await transport().execute("INSERT INTO probe_customers(name) VALUES ('x')"); + + expect(result.affectedRowCount).toBe(2); + expect(result.lastInsertRowId).toBe(7); + expect(result.executionTimeMs).toBe(5.014); + expect(result.fieldNames).toEqual([]); + }); +}); + +// ============================================================================ +// Value codec +// ============================================================================ + +describe("LibSQLHranaTransport value decoding", () => { + function scalar(value: Record): Promise { + handler = () => + respond( + pipeline( + okStep({ + cols: [{ name: "v", decltype: null }], + rows: [[value]], + affected_row_count: 0, + last_insert_rowid: null, + rows_read: 0, + rows_written: 0, + query_duration_ms: 0.01, + }), + ), + ); + return transport() + .execute("SELECT ? AS v") + .then((r) => r.rows[0]?.v); + } + + test("decodes an integer that fits a double as a number", async () => { + expect(await scalar({ type: "integer", value: "2000" })).toBe(2000); + }); + + test("keeps an integer past 2^53 as its exact decimal string rather than rounding it", async () => { + // 9007199254740993 is 2^53 + 1: the first integer a double cannot hold, and + // Number() would silently answer 9007199254740992. A rounded id is a + // corruption nothing downstream can detect (#460, Trino's own bigint lesson). + expect(await scalar({ type: "integer", value: "9007199254740993" })).toBe("9007199254740993"); + }); + + test("decodes a float as a number", async () => { + expect(await scalar({ type: "float", value: 1.5 })).toBe(1.5); + }); + + test("decodes null", async () => { + expect(await scalar({ type: "null" })).toBeNull(); + }); + + test("decodes a blob from base64 into bytes", async () => { + const decoded = await scalar({ type: "blob", base64: "AQID" }); + expect(decoded).toBeInstanceOf(Uint8Array); + expect(Array.from(decoded as Uint8Array)).toEqual([1, 2, 3]); + }); + + test("passes an unrecognised value type through as null rather than inventing a reading", async () => { + expect(await scalar({ type: "future-type", value: "x" })).toBeNull(); + }); +}); + +describe("LibSQLHranaTransport parameter encoding", () => { + test("encodes an integer parameter as a decimal string, the way the protocol requires", async () => { + await transport().execute("SELECT ? AS a", { params: [7] }); + + expect(sentArgs()).toEqual([{ type: "integer", value: "7" }]); + }); + + test("encodes a non-integral number as a float", async () => { + await transport().execute("SELECT ? AS a", { params: [1.5] }); + + expect(sentArgs()).toEqual([{ type: "float", value: 1.5 }]); + }); + + test("encodes a bigint parameter without going through a double", async () => { + await transport().execute("SELECT ? AS a", { params: [BigInt("9007199254740993")] }); + + expect(sentArgs()).toEqual([{ type: "integer", value: "9007199254740993" }]); + }); + + test("encodes text, null and undefined", async () => { + await transport().execute("SELECT ?, ?, ?", { params: ["tr", null, undefined] }); + + expect(sentArgs()).toEqual([{ type: "text", value: "tr" }, { type: "null" }, { type: "null" }]); + }); + + test("encodes a boolean as SQLite's own 1 and 0, which is what SQLite stores", async () => { + await transport().execute("SELECT ?, ?", { params: [true, false] }); + + expect(sentArgs()).toEqual([ + { type: "integer", value: "1" }, + { type: "integer", value: "0" }, + ]); + }); + + test("encodes bytes as base64", async () => { + await transport().execute("SELECT ?", { params: [new Uint8Array([1, 2, 3])] }); + + expect(sentArgs()).toEqual([{ type: "blob", base64: "AQID" }]); + }); + + test("encodes a Date as an ISO string, the only reading SQLite's date functions accept", async () => { + await transport().execute("SELECT ?", { params: [new Date("2026-08-27T00:00:00.000Z")] }); + + expect(sentArgs()).toEqual([{ type: "text", value: "2026-08-27T00:00:00.000Z" }]); + }); + + test("sends no args member when the statement has no parameters", async () => { + await transport().execute("SELECT 1"); + + const requests = sentBody().requests as { stmt?: Record }[]; + expect(requests[0]?.stmt && "args" in requests[0].stmt).toBe(false); + }); +}); + +// ============================================================================ +// Failure paths +// ============================================================================ + +describe("LibSQLHranaTransport failures", () => { + test("a statement the engine rejected arrives as HTTP 200 and still raises the engine's message", async () => { + // Captured verbatim from both deployments: the status is 200 and the failure + // is inside the envelope, so `response.ok` is not the test. + handler = () => + respond({ + baton: null, + base_url: null, + results: [ + { type: "error", error: { message: "SQLite error: no such table: no_such_table", code: "SQLITE_UNKNOWN" } }, + { type: "ok", response: { type: "close" } }, + ], + }); + + const failure = transport().execute("SELECT * FROM no_such_table"); + + await expect(failure).rejects.toThrow("SQLite error: no such table: no_such_table"); + await expect(failure).rejects.toMatchObject({ status: 200, code: "SQLITE_UNKNOWN" }); + }); + + test("a statement sqld refuses to parse carries the refusal wording of its own deployment", async () => { + handler = () => + respond({ + baton: null, + base_url: null, + results: [ + { + type: "error", + error: { + message: "SQL string could not be parsed: unsupported statement: VACUUM", + code: "SQL_PARSE_ERROR", + }, + }, + ], + }); + + await expect(transport().execute("VACUUM")).rejects.toMatchObject({ code: "SQL_PARSE_ERROR" }); + }); + + test("Turso Cloud's different wording for the same refusal maps to the same code", async () => { + handler = () => + respond({ + baton: null, + base_url: null, + results: [{ type: "error", error: { message: "SQL not allowed statement: VACUUM", code: "SQL_PARSE_ERROR" } }], + }); + + const failure = transport().execute("VACUUM"); + + await expect(failure).rejects.toMatchObject({ code: "SQL_PARSE_ERROR" }); + await expect(failure).rejects.toThrow("SQL not allowed statement: VACUUM"); + }); + + test("a missing token answers 401 with a bare string error and is reported as that", async () => { + handler = () => + respond({ error: "Unauthorized: `unauthorized access attempt on database: empty JWT token`" }, { status: 401 }); + + const failure = transport().execute("SELECT 1"); + + await expect(failure).rejects.toBeInstanceOf(LibSQLTransportError); + await expect(failure).rejects.toThrow(/empty JWT token/); + await expect(failure).rejects.toMatchObject({ status: 401 }); + }); + + test("a malformed token answers 400 rather than 401, and the status is passed through", async () => { + handler = () => respond({ error: "JWT error: InvalidToken" }, { status: 400 }); + + await expect(transport({ password: "notatoken" }).execute("SELECT 1")).rejects.toMatchObject({ + status: 400, + }); + }); + + test("a body that is not a libSQL envelope is reported as such rather than parsed further", async () => { + handler = () => respond("proxy"); + + await expect(transport().execute("SELECT 1")).rejects.toThrow(/not a libSQL answer/); + }); + + test("an envelope with no results at all is reported rather than read as an empty success", async () => { + handler = () => respond({ baton: null, base_url: null, results: [] }); + + await expect(transport().execute("SELECT 1")).rejects.toThrow(/no result/); + }); + + test("a request that never reached a server is reported with status 0", async () => { + globalThis.fetch = (() => Promise.reject(new Error("connect ECONNREFUSED"))) as unknown as typeof fetch; + + const failure = transport().execute("SELECT 1"); + + await expect(failure).rejects.toMatchObject({ status: 0 }); + await expect(failure).rejects.toThrow(/connect ECONNREFUSED/); + }); +}); + +// ============================================================================ +// Server version +// ============================================================================ + +describe("LibSQLHranaTransport serverVersion", () => { + test("reads the version sqld publishes as plain text", async () => { + handler = (url) => + url.endsWith("/version") + ? new Response("sqld 0.24.33 (f8fb14f3 2026-08-11)", { status: 200 }) + : respond(pipeline(okStep(masterResult()))); + + expect(await transport().serverVersion()).toBe("sqld 0.24.33 (f8fb14f3 2026-08-11)"); + }); + + test("answers null on Turso Cloud, where the route does not exist", async () => { + // Measured 2026-08-27: `{"error":"route not found: [\"version\"]"}` with a + // non-2xx status. A deployment that publishes no version is not a failed + // connection, so this is null rather than a throw (#477's absence rule). + handler = () => respond({ error: 'route not found: ["version"]' }, { status: 404 }); + + expect(await transport().serverVersion()).toBeNull(); + }); + + test("answers null when the version route cannot be reached at all", async () => { + globalThis.fetch = (() => Promise.reject(new Error("socket hang up"))) as unknown as typeof fetch; + + expect(await transport().serverVersion()).toBeNull(); + }); + + test("answers null rather than an empty string when the route answers nothing", async () => { + handler = () => new Response(" ", { status: 200 }); + + expect(await transport().serverVersion()).toBeNull(); + }); +}); + +describe("LibSQLHranaTransport close", () => { + test("is a no-op that issues no request, because every pipeline closes itself", async () => { + const t = transport(); + await t.execute("SELECT 1"); + await t.close(); + + expect(calls).toHaveLength(1); + }); + + test("declares the transport kind", () => { + expect(transport().kind).toBe("hrana-http"); + }); +}); + +// ============================================================================ +// Shapes a proxy or a future protocol version can produce +// ---------------------------------------------------------------------------- +// Everything below is a guard rather than a captured reading: the protocol is +// free to change and a reverse proxy is free to mangle, and each of these is a +// branch that would otherwise be reached first in production. +// ============================================================================ + +describe("LibSQLHranaTransport tolerances", () => { + function withResult(result: Record): void { + handler = () => respond(pipeline(okStep(result))); + } + + test("aborts a statement that outruns its timeout", async () => { + globalThis.fetch = ((_input: RequestInfo | URL, init?: RequestInit) => { + // The signal is what carries the timeout; asserting on its presence is the + // only way to tell a real deadline from an unbounded request. + expect(init?.signal).toBeInstanceOf(AbortSignal); + return Promise.resolve(respond(pipeline(okStep(masterResult())))); + }) as unknown as typeof fetch; + + await transport().execute("SELECT 1", { timeoutMs: 250 }); + }); + + test("names a column the engine left unnamed by its position rather than dropping it", async () => { + withResult({ + cols: [{ decltype: null }, { name: "", decltype: "TEXT" }], + rows: [ + [ + { type: "integer", value: "1" }, + { type: "text", value: "x" }, + ], + ], + affected_row_count: 0, + last_insert_rowid: null, + query_duration_ms: 0.01, + }); + + const result = await transport().execute("SELECT 1, 'x'"); + + expect(result.fieldNames).toEqual(["column_1", "column_2"]); + expect(result.rows).toEqual([{ column_1: 1, column_2: "x" }]); + // The empty name is not a declared type either: the second column's TEXT + // lands under the positional name, never under "". + expect(result.columnTypes).toEqual({ column_2: "TEXT" }); + }); + + test("reads a column declaration that is not a list as no columns at all", async () => { + withResult({ cols: null, rows: null, affected_row_count: null, query_duration_ms: null }); + + const result = await transport().execute("SELECT 1"); + + expect(result).toMatchObject({ fieldNames: [], rows: [], affectedRowCount: 0, executionTimeMs: 0 }); + }); + + test("reads a row that is not a list as a row of nulls, keeping the column count", async () => { + withResult({ + cols: [{ name: "a" }, { name: "b" }], + rows: [null], + affected_row_count: 0, + query_duration_ms: 0, + }); + + expect((await transport().execute("SELECT a, b FROM t")).rows).toEqual([{ a: null, b: null }]); + }); + + test("decodes a float the protocol quoted, and a text value it did not", async () => { + withResult({ + cols: [{ name: "f" }, { name: "t" }, { name: "u" }], + rows: [[{ type: "float", value: "2.5" }, { type: "text", value: 7 }, { type: "text" }]], + affected_row_count: 0, + query_duration_ms: 0, + }); + + expect((await transport().execute("SELECT 1")).rows[0]).toEqual({ f: 2.5, t: "7", u: "" }); + }); + + test("decodes a value that is not an object at all as null", async () => { + withResult({ + cols: [{ name: "v" }], + rows: [["bare"]], + affected_row_count: 0, + query_duration_ms: 0, + }); + + expect((await transport().execute("SELECT 1")).rows[0]).toEqual({ v: null }); + }); + + test("decodes an integer sent as a number, and one sent as an unsafe number, without rounding silently", async () => { + withResult({ + cols: [{ name: "safe" }, { name: "wide" }, { name: "blank" }, { name: "junk" }], + rows: [ + [ + { type: "integer", value: 2000 }, + // Computed rather than written as a literal: a source literal that wide is + // a lint error, and JSON.parse would have rounded it here anyway - which is + // exactly the shape this row is about, an UNQUOTED integer past 2^53 that + // arrived already rounded and must not be rounded twice or silently kept. + { type: "integer", value: Number("9007199254740993") }, + { type: "integer", value: " " }, + { type: "integer", value: "not-a-number" }, + ], + ], + affected_row_count: 0, + query_duration_ms: 0, + }); + + // The third and fourth are absences rather than zeros: a value this layer + // cannot read must not become a number a reader would trust. + expect((await transport().execute("SELECT 1")).rows[0]).toEqual({ + safe: 2000, + wide: "9007199254740992", + blank: null, + junk: null, + }); + }); + + test("decodes a blob whose base64 is missing as null rather than as empty bytes", async () => { + withResult({ + cols: [{ name: "b" }], + rows: [[{ type: "blob" }]], + affected_row_count: 0, + query_duration_ms: 0, + }); + + expect((await transport().execute("SELECT 1")).rows[0]).toEqual({ b: null }); + }); + + test("carries a rowid too wide for a double as its exact decimal string", async () => { + withResult({ + cols: [], + rows: [], + affected_row_count: 1, + last_insert_rowid: "9007199254740993", + query_duration_ms: 0.4, + }); + + expect((await transport().execute("INSERT INTO t VALUES (1)")).lastInsertRowId).toBe("9007199254740993"); + }); + + test("reports the nested error shape a non-2xx response may carry", async () => { + handler = () => + respond({ error: { message: "namespace not found", code: "NAMESPACE_NOT_FOUND" } }, { status: 404 }); + + const failure = transport().execute("SELECT 1"); + + await expect(failure).rejects.toThrow("namespace not found"); + await expect(failure).rejects.toMatchObject({ status: 404, code: "NAMESPACE_NOT_FOUND" }); + }); + + test("falls back to the status when a non-2xx response carries no body", async () => { + handler = () => new Response("", { status: 502 }); + + await expect(transport().execute("SELECT 1")).rejects.toThrow("HTTP 502"); + }); + + test("falls back to the raw body when a non-2xx response is not JSON at all", async () => { + handler = () => new Response("502 Bad Gateway", { status: 502 }); + + await expect(transport().execute("SELECT 1")).rejects.toThrow("502 Bad Gateway"); + }); + + test("reports a statement error the engine sent without a message", async () => { + handler = () => respond({ baton: null, results: [{ type: "error", error: {} }] }); + + const failure = transport().execute("SELECT 1"); + + await expect(failure).rejects.toThrow("the statement failed"); + await expect(failure).rejects.toMatchObject({ code: null }); + }); + + test("reports a step that is not an object as no result", async () => { + handler = () => respond({ baton: null, results: [null] }); + + await expect(transport().execute("SELECT 1")).rejects.toThrow(/no result/); + }); + + test("reads a success whose result member is missing as an empty result", async () => { + handler = () => respond({ baton: null, results: [{ type: "ok", response: { type: "execute" } }] }); + + expect(await transport().execute("SELECT 1")).toMatchObject({ rows: [], fieldNames: [] }); + }); + + test("sends the token on the version route as well, since a private database refuses it otherwise", async () => { + handler = (url) => (url.endsWith("/version") ? new Response("sqld 0.24.33", { status: 200 }) : respond(pipeline())); + + await transport({ password: "tok-123" }).serverVersion(); + + const headers = new Headers(calls[0]?.init?.headers); + expect(headers.get("authorization")).toBe("Bearer tok-123"); + }); +}); + +// ============================================================================ +// Batches +// ---------------------------------------------------------------------------- +// Captured 2026-08-27 on sqld 0.24.33: a pipeline whose SECOND statement fails +// still runs the third, and every step carries its own outcome. That measurement +// is the reason executeBatch hands failures back individually. +// ============================================================================ + +describe("LibSQLHranaTransport executeBatch", () => { + function scalarStep(value: string): Record { + return okStep({ + cols: [{ name: "a", decltype: null }], + rows: [[{ type: "integer", value }]], + affected_row_count: 0, + last_insert_rowid: null, + query_duration_ms: 0.01, + }); + } + + test("sends every statement in one request and closes once", async () => { + handler = () => respond(pipeline(scalarStep("1"), scalarStep("2"))); + + const outcomes = await transport().executeBatch([{ sql: "SELECT 1 AS a" }, { sql: "SELECT 2 AS a" }]); + + expect(calls).toHaveLength(1); + const requests = sentBody().requests as { type: string }[]; + expect(requests.map((r) => r.type)).toEqual(["execute", "execute", "close"]); + expect(outcomes.map((o) => (o.ok ? o.result.rows[0]?.a : null))).toEqual([1, 2]); + }); + + test("a failing statement costs its own outcome and nothing else", async () => { + handler = () => + respond({ + baton: null, + base_url: null, + results: [ + scalarStep("1"), + { type: "error", error: { message: "SQLite error: no such table: nope", code: "SQLITE_UNKNOWN" } }, + scalarStep("3"), + { type: "ok", response: { type: "close" } }, + ], + }); + + const outcomes = await transport().executeBatch([ + { sql: "SELECT 1 AS a" }, + { sql: "SELECT * FROM nope" }, + { sql: "SELECT 3 AS a" }, + ]); + + expect(outcomes.map((o) => o.ok)).toEqual([true, false, true]); + expect(outcomes[1]).toMatchObject({ ok: false }); + if (!outcomes[1]?.ok) { + expect(outcomes[1]?.error).toBeInstanceOf(LibSQLTransportError); + expect(outcomes[1]?.error.message).toContain("no such table: nope"); + expect(outcomes[1]?.error.code).toBe("SQLITE_UNKNOWN"); + } + }); + + test("binds each statement's own parameters", async () => { + handler = () => respond(pipeline(scalarStep("1"), scalarStep("2"))); + + await transport().executeBatch([ + { sql: "SELECT ? AS a", params: [1] }, + { sql: "SELECT ? AS a", params: ["tr"] }, + ]); + + const requests = sentBody().requests as { stmt?: { args?: unknown[] } }[]; + expect(requests[0]?.stmt?.args).toEqual([{ type: "integer", value: "1" }]); + expect(requests[1]?.stmt?.args).toEqual([{ type: "text", value: "tr" }]); + }); + + test("answers an empty list without touching the network", async () => { + expect(await transport().executeBatch([])).toEqual([]); + expect(calls).toHaveLength(0); + }); + + test("reports a step the server never sent as that statement's own failure", async () => { + // A pipeline answered with fewer results than it carried statements: the + // missing ones become failures of their own rather than shifting every later + // result onto the wrong statement. + handler = () => respond({ baton: null, base_url: null, results: [scalarStep("1")] }); + + const outcomes = await transport().executeBatch([{ sql: "SELECT 1 AS a" }, { sql: "SELECT 2 AS a" }]); + + expect(outcomes.map((o) => o.ok)).toEqual([true, false]); + if (!outcomes[1]?.ok) expect(outcomes[1]?.error.message).toMatch(/no result/); + }); + + test("a transport failure fails the whole batch, because no statement ran", async () => { + globalThis.fetch = (() => Promise.reject(new Error("connect ECONNREFUSED"))) as unknown as typeof fetch; + + await expect(transport().executeBatch([{ sql: "SELECT 1" }])).rejects.toMatchObject({ status: 0 }); + }); + + test("a body that is not an envelope fails the whole batch rather than each statement", async () => { + handler = () => respond("proxy"); + + await expect(transport().executeBatch([{ sql: "SELECT 1" }])).rejects.toThrow(/not a libSQL answer/); + }); +}); diff --git a/tests/unit/db/libsql/introspect.test.ts b/tests/unit/db/libsql/introspect.test.ts new file mode 100644 index 00000000..f29726c6 --- /dev/null +++ b/tests/unit/db/libsql/introspect.test.ts @@ -0,0 +1,433 @@ +/** + * libSQL introspection (issue #424 Phase 5) + * + * The transport is a fake that answers by matching the statement text, so every + * test here states what the provider ASKED as well as what it did with the + * answer. Two behaviours are pinned that no engine can be trusted to keep: + * + * - a per-table read that fails costs its own reading and nothing else, because + * Hrana answers each statement of a batch separately (measured 2026-08-27); + * - a size that could not be read is ABSENT, never 0 (#477, and the same rule + * `buildTableStats` follows in the SQLite provider). + */ +import { describe, expect, test } from "bun:test"; +import { + readHealth, + readIndexStats, + readOverview, + readSchema, + readStorageStats, + readTableStats, +} from "@/lib/db/providers/sql/libsql/introspect"; +import { + type LibSQLBatchOutcome, + type LibSQLExecuteOptions, + type LibSQLStatement, + type LibSQLStatementResult, + type LibSQLTransport, + LibSQLTransportError, +} from "@/lib/db/providers/sql/libsql/transport"; + +// ============================================================================ +// Fake transport +// ============================================================================ + +type Answer = LibSQLStatementResult | LibSQLTransportError; + +function rows(fieldNames: string[], values: unknown[][]): LibSQLStatementResult { + return { + rows: values.map((row) => Object.fromEntries(fieldNames.map((name, index) => [name, row[index]]))), + fieldNames, + columnTypes: {}, + affectedRowCount: 0, + lastInsertRowId: null, + executionTimeMs: 0.1, + }; +} + +const EMPTY = rows([], []); + +class FakeTransport implements LibSQLTransport { + public readonly kind = "hrana-http" as const; + public readonly asked: string[] = []; + public batchSizes: number[] = []; + private readonly version: string | null; + private readonly answers: [RegExp, Answer][]; + + constructor(answers: [RegExp, Answer][], version: string | null = "sqld 0.24.33 (f8fb14f3 2026-08-11)") { + this.answers = answers; + this.version = version; + } + + public async execute(sql: string, _options?: LibSQLExecuteOptions): Promise { + const outcome = this.answer(sql); + if (!outcome.ok) throw outcome.error; + return outcome.result; + } + + public async executeBatch( + statements: LibSQLStatement[], + _options?: LibSQLExecuteOptions, + ): Promise { + this.batchSizes.push(statements.length); + return statements.map((statement) => this.answer(statement.sql)); + } + + public async serverVersion(): Promise { + return this.version; + } + + public async close(): Promise {} + + private answer(sql: string): LibSQLBatchOutcome { + this.asked.push(sql); + for (const [pattern, answer] of this.answers) { + if (pattern.test(sql)) { + return answer instanceof LibSQLTransportError ? { ok: false, error: answer } : { ok: true, result: answer }; + } + } + return { ok: true, result: EMPTY }; + } +} + +const REFUSED = new LibSQLTransportError("SQLite error: no such table: dbstat", 200, "SQLITE_UNKNOWN"); + +/** The reads a two-table database answers, with one index on the first table. */ +function twoTableTransport(overrides: [RegExp, Answer][] = []): FakeTransport { + return new FakeTransport([ + ...overrides, + [/FROM sqlite_master\s+WHERE type = 'table'/, rows(["name"], [["probe_customers"], ["probe_orders"]])], + [/COUNT\(\*\) AS row_count FROM "probe_customers"/, rows(["row_count"], [[3]])], + [/COUNT\(\*\) AS row_count FROM "probe_orders"/, rows(["row_count"], [[2000]])], + [ + /pragma_table_info\('probe_customers'\)/, + rows( + ["cid", "name", "type", "notnull", "dflt_value", "pk"], + [ + [0, "id", "INTEGER", 1, null, 1], + [1, "country", "TEXT", 0, "'tr'", 0], + ], + ), + ], + [ + /pragma_table_info\('probe_orders'\)/, + rows(["cid", "name", "type", "notnull", "dflt_value", "pk"], [[0, "id", "INTEGER", 0, null, 0]]), + ], + [ + /pragma_index_list\('probe_customers'\)/, + rows( + ["seq", "name", "unique", "origin", "partial"], + [ + [0, "idx_country", 1, "c", 0], + [1, "sqlite_autoindex_probe_customers_1", 1, "pk", 0], + ], + ), + ], + [ + /pragma_foreign_key_list\('probe_orders'\)/, + rows(["id", "seq", "table", "from", "to"], [[0, 0, "probe_customers", "customer_id", "id"]]), + ], + [/pragma_index_info\('idx_country'\)/, rows(["seqno", "cid", "name"], [[0, 1, "country"]])], + [ + /FROM dbstat/, + rows( + ["name", "bytes"], + [ + ["probe_customers", 8192], + ["idx_country", 4096], + ["probe_orders", 270336], + ], + ), + ], + [/type = 'index'/, rows(["name", "tbl_name"], [["idx_country", "probe_customers"]])], + [/sqlite_version\(\)/, rows(["version"], [["3.47.0"]])], + [/page_count/, rows(["size_bytes"], [[282624]])], + [/integrity_check/, rows(["integrity_check"], [["ok"]])], + [/journal_mode/, rows(["journal_mode"], [["wal"]])], + ]); +} + +// ============================================================================ +// Schema +// ============================================================================ + +describe("readSchema", () => { + test("reads columns, indexes, foreign keys and row counts for every table", async () => { + const schema = await readSchema(twoTableTransport()); + + expect(schema.map((t) => t.name)).toEqual(["probe_customers", "probe_orders"]); + expect(schema[0]?.rowCount).toBe(3); + expect(schema[0]?.columns).toEqual([ + { name: "id", type: "INTEGER", nullable: false, isPrimary: true }, + { name: "country", type: "TEXT", nullable: true, isPrimary: false, defaultValue: "'tr'" }, + ]); + expect(schema[0]?.indexes).toEqual([{ name: "idx_country", columns: ["country"], unique: true }]); + expect(schema[1]?.foreignKeys).toEqual([ + { columnName: "customer_id", referencedTable: "probe_customers", referencedColumn: "id" }, + ]); + }); + + test("drops SQLite's own internal indexes, which are not objects a user made", async () => { + const schema = await readSchema(twoTableTransport()); + + expect(schema[0]?.indexes.map((i) => i.name)).not.toContain("sqlite_autoindex_probe_customers_1"); + }); + + test("gives each table its OWN measured size rather than the whole database's", async () => { + const schema = await readSchema(twoTableTransport()); + + // 8192 of table pages + 4096 of index pages for probe_customers, and the + // SQLite provider's own reading of "the database file size, once per table" + // is what this deliberately does not do. + expect(schema[0]?.size).toBe("12 KB"); + expect(schema[1]?.size).toBe("264 KB"); + }); + + test("omits the size entirely when dbstat is not compiled in", async () => { + const schema = await readSchema(twoTableTransport([[/FROM dbstat/, REFUSED]])); + + expect(schema[0]?.size).toBeUndefined(); + // The rows are still real: an absent size costs the size and nothing else. + expect(schema[0]?.rowCount).toBe(3); + }); + + test("keeps a table whose column read failed, with its row count and no invented columns", async () => { + const schema = await readSchema(twoTableTransport([[/pragma_table_info\('probe_customers'\)/, REFUSED]])); + + expect(schema.map((t) => t.name)).toEqual(["probe_customers", "probe_orders"]); + expect(schema[0]?.columns).toEqual([]); + expect(schema[0]?.rowCount).toBe(3); + expect(schema[1]?.columns).toHaveLength(1); + }); + + test("omits a row count the engine refused rather than reporting zero rows", async () => { + const schema = await readSchema(twoTableTransport([[/COUNT\(\*\) AS row_count FROM "probe_customers"/, REFUSED]])); + + expect(schema[0]?.rowCount).toBeUndefined(); + expect(schema[1]?.rowCount).toBe(2000); + }); + + test("reads a count the transport kept as a wide decimal string, and refuses one that is not a number", async () => { + // The transport hands back a decimal STRING for an integer past 2^53 rather than + // a rounded number. For a display statistic that string is parsed; anything + // unreadable stays absent rather than becoming 0. + const schema = await readSchema( + twoTableTransport([ + [/COUNT\(\*\) AS row_count FROM "probe_customers"/, rows(["row_count"], [["9007199254740993"]])], + [/COUNT\(\*\) AS row_count FROM "probe_orders"/, rows(["row_count"], [["not-a-number"]])], + ]), + ); + + expect(schema[0]?.rowCount).toBe(9007199254740992); + expect(schema[1]?.rowCount).toBeUndefined(); + }); + + test("answers an empty schema without asking a single per-table question", async () => { + const transport = new FakeTransport([[/FROM sqlite_master\s+WHERE type = 'table'/, rows(["name"], [])]]); + + expect(await readSchema(transport)).toEqual([]); + expect(transport.batchSizes).toEqual([]); + }); + + test("raises when the table list itself cannot be read, because there is nothing to show", async () => { + const transport = new FakeTransport([[/FROM sqlite_master/, REFUSED]]); + + await expect(readSchema(transport)).rejects.toThrow(/no such table: dbstat/); + }); + + test("quotes the notnull column, which is a SQLite keyword and a parse error unquoted", async () => { + // Live-probe regression (sqld 0.24.33): `SELECT cid, name, type, notnull, ... FROM + // pragma_table_info(...)` is "near NOTNULL: syntax error", and the failure costs + // the COLUMNS of every table while leaving the rest of the tree intact - so the + // object browser listed both tables and showed each as having none. A fake + // transport cannot parse SQL, so what is pinned here is the statement text. + const transport = twoTableTransport(); + await readSchema(transport); + + const columnReads = transport.asked.filter((sql) => sql.includes("pragma_table_info")); + expect(columnReads).toHaveLength(2); + for (const sql of columnReads) { + expect(sql).toContain('"notnull"'); + expect(sql).not.toMatch(/,\s*notnull\s*,/); + } + }); + + test("asks the per-table questions in ONE round trip rather than four per table", async () => { + const transport = twoTableTransport(); + await readSchema(transport); + + // Two tables: four reads each in one batch, then one batch for the single + // user index's columns. A libSQL server is across a network, so a read that + // costs four round trips per table is the difference between a schema tree + // that opens and one that times out. + expect(transport.batchSizes[0]).toBe(8); + }); +}); + +// ============================================================================ +// Overview, health, metrics +// ============================================================================ + +describe("readOverview", () => { + test("names both the server and the SQLite version it embeds", async () => { + const overview = await readOverview(twoTableTransport()); + + expect(overview.version).toBe("sqld 0.24.33 (f8fb14f3 2026-08-11) (SQLite 3.47.0)"); + }); + + test("falls back to the SQLite version alone on a deployment that publishes no server version", async () => { + // Turso Cloud, measured 2026-08-27: there is no /version route at all, so the + // panel shows what the engine did answer instead of "Unknown". + const transport = twoTableTransport(); + const cloud = new FakeTransport( + [ + [/sqlite_version\(\)/, rows(["version"], [["3.47.0"]])], + [/page_count/, rows(["size_bytes"], [[282624]])], + [/type = 'table'/, rows(["table_count"], [[2]])], + [/type = 'index'/, rows(["index_count"], [[1]])], + ], + null, + ); + void transport; + + expect((await readOverview(cloud)).version).toBe("SQLite 3.47.0"); + }); + + test("reports the measured database size, the table count and the index count", async () => { + const overview = await readOverview( + new FakeTransport([ + [/sqlite_version\(\)/, rows(["version"], [["3.47.0"]])], + [/page_count/, rows(["size_bytes"], [[282624]])], + [/type = 'table'/, rows(["table_count"], [[2]])], + [/type = 'index'/, rows(["index_count"], [[1]])], + ]), + ); + + expect(overview.databaseSizeBytes).toBe(282624); + expect(overview.databaseSize).toBe("276 KB"); + expect(overview.tableCount).toBe(2); + expect(overview.indexCount).toBe(1); + }); + + test("publishes no connection ceiling and no uptime, because libSQL publishes neither", async () => { + const overview = await readOverview(twoTableTransport()); + + // 0 is this codebase's encoding for "no limit published" (trino, druid, mssql). + expect(overview.maxConnections).toBe(0); + expect(overview.uptime).toBe("N/A"); + // Hrana is stateless: a statement is a request, so there is no session to count. + expect(overview.activeConnections).toBeUndefined(); + }); + + test("leaves the size absent when the page counters could not be read", async () => { + const overview = await readOverview(twoTableTransport([[/page_count/, REFUSED]])); + + expect(overview.databaseSize).toBe("N/A"); + expect(overview.databaseSizeBytes).toBe(0); + }); +}); + +describe("readHealth", () => { + test("reports the integrity check and the journal mode as the two readings libSQL has", async () => { + const health = await readHealth(twoTableTransport()); + + expect(health.databaseSize).toBe("276 KB"); + expect(health.slowQueries.map((q) => q.query)).toEqual(["Integrity: OK", "Journal Mode: wal"]); + expect(health.activeSessions).toEqual([]); + }); + + test("says the cache hit ratio is not measured rather than inventing one", async () => { + expect((await readHealth(twoTableTransport())).cacheHitRatio).toBe("N/A"); + }); + + test("reports a failed integrity check as failed", async () => { + const health = await readHealth( + twoTableTransport([[/integrity_check/, rows(["integrity_check"], [["*** in database main ***"]])]]), + ); + + expect(health.slowQueries[0]?.query).toBe("Integrity: FAILED"); + }); + + test("says unknown for a journal mode the engine would not answer", async () => { + const health = await readHealth(twoTableTransport([[/journal_mode/, REFUSED]])); + + expect(health.slowQueries[1]?.query).toBe("Journal Mode: unknown"); + }); +}); + +// ============================================================================ +// Stats +// ============================================================================ + +describe("readTableStats", () => { + test("splits measured pages between a table and its indexes", async () => { + const stats = await readTableStats(twoTableTransport()); + + expect(stats[0]).toEqual({ + schemaName: "main", + tableName: "probe_customers", + rowCount: 3, + tableSize: "8 KB", + tableSizeBytes: 8192, + indexSize: "4 KB", + indexSizeBytes: 4096, + totalSize: "12 KB", + totalSizeBytes: 12288, + }); + }); + + test("omits every byte figure when dbstat is unavailable, keeping the row counts", async () => { + const stats = await readTableStats(twoTableTransport([[/FROM dbstat/, REFUSED]])); + + expect(stats[0]).toEqual({ + schemaName: "main", + tableName: "probe_customers", + rowCount: 3, + totalSize: "N/A", + totalSizeBytes: 0, + }); + }); +}); + +describe("readIndexStats", () => { + test("reports each user index with its columns and its measured size", async () => { + const stats = await readIndexStats(twoTableTransport()); + + expect(stats).toEqual([ + { + schemaName: "main", + tableName: "probe_customers", + indexName: "idx_country", + columns: ["country"], + isUnique: true, + isPrimary: false, + indexSize: "4 KB", + indexSizeBytes: 4096, + // SQLite keeps no per-index scan counter, and 0 here is the count of a + // statistic that does not exist rather than a measurement of no scans. + scans: 0, + }, + ]); + }); + + test("reports an index whose size could not be measured as N/A rather than as empty", async () => { + const stats = await readIndexStats(twoTableTransport([[/FROM dbstat/, REFUSED]])); + + expect(stats[0]?.indexSize).toBe("N/A"); + expect(stats[0]?.indexSizeBytes).toBeUndefined(); + }); +}); + +describe("readStorageStats", () => { + test("reports the one file libSQL has, measured from its own page counters", async () => { + const stats = await readStorageStats(twoTableTransport()); + + expect(stats).toEqual([{ name: "main", size: "276 KB", sizeBytes: 282624 }]); + }); + + test("answers nothing at all when the page counters could not be read", async () => { + // An entry reading 0 B would draw an empty database on the Storage tab; no + // entry draws the tab's own empty state, which is the honest one. + expect(await readStorageStats(twoTableTransport([[/page_count/, REFUSED]]))).toEqual([]); + }); +}); diff --git a/tests/unit/db/libsql/seam-guard.test.ts b/tests/unit/db/libsql/seam-guard.test.ts new file mode 100644 index 00000000..ffe0de10 --- /dev/null +++ b/tests/unit/db/libsql/seam-guard.test.ts @@ -0,0 +1,263 @@ +/** + * libSQL transport seam guard (issue #424 Phase 5) + * + * The libSQL provider is worth building without a client library only while + * swapping the transport stays cheap, and it stays cheap only while the Hrana + * envelope lives in exactly one file. This test is the mechanism that keeps that + * true: it parses every source in the provider directory and fails the build the + * moment the wire vocabulary is used outside `hrana-transport.ts`. It reads the + * directory from disk rather than from a list, so it keeps holding as the provider + * grows. + * + * There IS a plausible second implementation, which is why the seam is not + * ceremony: Hrana also runs over WebSocket, `@tursodatabase/database` embeds the + * engine in-process, and `@libsql/client` speaks both. Any of them would answer the + * same questions - none of them would answer them with a `baton`. + * + * The guard is a parser, not a grep, and it sorts the vocabulary into two classes: + * + * - Tokens Hrana invented and SQLite does not have (`baton`, `base_url`, + * `affected_row_count`, `query_duration_ms`, `replication_index`, `decltype`, + * `rows_read`, `rows_written`, and the endpoint path) are matched as TEXT, so + * naming one anywhere outside the transport is a leak. + * - `last_insert_rowid` is matched only as a READ off a payload, because it is also + * a real SQLite function: `SELECT last_insert_rowid()` is legitimate SQL that any + * implementation may issue, and a guard that fires on it would be crying wolf. + * + * `hrana` itself is deliberately NOT in the vocabulary: `index.ts` has to name the + * concrete `LibSQLHranaTransport` to construct one, and the neutral seam publishes + * `kind: "hrana-http"` on purpose so a caller can tell the implementations apart. + * A guard that cries wolf is a guard the next contributor deletes, so both + * directions are proven below: the detector must light up on the file that is + * SUPPOSED to speak Hrana, and stay silent on a compliant one. + */ +import { describe, expect, test } from "bun:test"; +import { readdirSync, readFileSync } from "node:fs"; +import { join } from "node:path"; +import ts from "typescript"; + +const ROOT = join(import.meta.dir, "..", "..", "..", ".."); +const PROVIDER_DIR = join(ROOT, "src", "lib", "db", "providers", "sql", "libsql"); + +/** The single file allowed to know the wire format. */ +const TRANSPORT_FILE = "hrana-transport.ts"; + +/** + * Identifiers Hrana invented. None of them is a SQLite keyword, a SQLite function + * or a column of anything SQLite publishes, so a mention outside the transport - in + * a string, a property name or an identifier - is a leak. + */ +const WIRE_TOKENS = [ + "/v2/pipeline", + "baton", + "base_url", + "affected_row_count", + "query_duration_ms", + "replication_index", + "decltype", + "rows_read", + "rows_written", +]; + +/** + * Payload fields that are ALSO real SQLite vocabulary, so text matching would fire + * on legitimate SQL. Reading one off a payload is still a leak: the neutral result + * calls it `lastInsertRowId`. + */ +const ENVELOPE_FIELDS = new Set(["last_insert_rowid"]); + +/** Everything the transport must speak, and nothing else may. */ +const WIRE_VOCABULARY = [...WIRE_TOKENS, ...ENVELOPE_FIELDS]; + +/** + * Why the rule exists, printed on failure. Whoever trips this needs to see the + * boundary they are crossing, otherwise the cheapest fix looks like deleting the + * test. + */ +const SEAM_RULE = [ + `The Hrana envelope leaked out of ${TRANSPORT_FILE}.`, + "", + "Hrana posts a list of requests to /v2/pipeline, answers one result per request, hands back a baton to", + "continue a server-side stream, and encodes every value as { type, value } with integers as decimal", + "strings. Issue #424 keeps all of that inside the transport: provider logic reads the neutral", + "LibSQLStatementResult (rows, fieldNames, columnTypes, affectedRowCount, lastInsertRowId,", + "executionTimeMs) and the per-statement LibSQLBatchOutcome through the LibSQLTransport seam. That is", + "what makes a WebSocket or embedded implementation one new file rather than a rewrite of the provider", + "and its introspection.", + "", + `Fix an access below by mapping the field inside ${TRANSPORT_FILE} and widening LibSQLStatementResult`, + "when the value is genuinely needed. If you tripped this on SQL - last_insert_rowid() is a real SQLite", + "function - read it under an alias (`last_insert_rowid() AS insertedId`) so one layer keeps one", + "vocabulary. Do not weaken or delete this test: it is the only thing keeping the seam real.", + "", + "Wire vocabulary outside the transport:", +].join("\n"); + +interface WireLeak { + file: string; + line: number; + token: string; + snippet: string; +} + +/** + * The wire tokens this node spells out. + * + * Only text carries them: a path, a property name in a constructed envelope, or an + * identifier that names one. Comments are trivia rather than nodes, so prose naming + * the envelope is deliberately free - the point is that no code depends on it. + */ +function spelledTokens(node: ts.Node): string[] { + const carriesText = ts.isStringLiteral(node) || ts.isTemplateLiteralToken(node) || ts.isIdentifier(node); + if (!carriesText) return []; + + const text = node.text.toLowerCase(); + return WIRE_TOKENS.filter((token) => text.includes(token.toLowerCase())); +} + +/** + * The payload field this node reads, or null when it reads none. + * + * Only three syntactic forms take a field off a payload, and all three are a leak + * regardless of how the value is spelled afterwards: + * + * result.last_insert_rowid / result?.last_insert_rowid PropertyAccessExpression + * result["last_insert_rowid"] ElementAccessExpression + * const { last_insert_rowid } = result BindingElement + * + * A declaration or a constructed literal is deliberately not a leak: a shape is + * inert until something reads it, and the read is what the three forms above catch. + */ +function accessedField(node: ts.Node): string | null { + if (ts.isPropertyAccessExpression(node)) return node.name.text; + if (ts.isElementAccessExpression(node)) { + const key = node.argumentExpression; + return ts.isStringLiteralLike(key) ? key.text : null; + } + // Array patterns bind by position, so only an object pattern names a field. + if (ts.isBindingElement(node) && ts.isObjectBindingPattern(node.parent)) { + const key = node.propertyName ?? node.name; + return ts.isIdentifier(key) || ts.isStringLiteralLike(key) ? key.text : null; + } + return null; +} + +function findWireLeaks(file: string, source: string): WireLeak[] { + const sourceFile = ts.createSourceFile(file, source, ts.ScriptTarget.Latest, true, ts.ScriptKind.TS); + const lines = source.split("\n"); + // One leak per line and token: an element access reports the same token as the + // string literal it contains, and reporting it twice reads like two problems. + const found = new Map(); + + const report = (node: ts.Node, token: string): void => { + const line = sourceFile.getLineAndCharacterOfPosition(node.getStart(sourceFile)).line; + found.set(`${line}:${token}`, { file, line: line + 1, token, snippet: lines[line].trim() }); + }; + + const visit = (node: ts.Node): void => { + for (const token of spelledTokens(node)) report(node, token); + + const field = accessedField(node); + if (field !== null && ENVELOPE_FIELDS.has(field)) report(node, field); + + ts.forEachChild(node, visit); + }; + + visit(sourceFile); + return [...found.values()].sort((a, b) => a.line - b.line); +} + +/** Empty when the seam holds; the rule plus every offending line when it does not. */ +function violationReport(leaks: WireLeak[]): string { + if (leaks.length === 0) return ""; + + const offences = leaks.map((leak) => ` ${leak.file}:${leak.line} uses "${leak.token}" -> ${leak.snippet}`); + return [SEAM_RULE, ...offences].join("\n"); +} + +function providerSources(): string[] { + return readdirSync(PROVIDER_DIR, { recursive: true }) + .map(String) + .filter((name) => name.endsWith(".ts")) + .sort(); +} + +function readProviderSource(file: string): string { + return readFileSync(join(PROVIDER_DIR, file), "utf8"); +} + +describe("libSQL transport seam", () => { + const sources = providerSources(); + + test("the guard scans the whole provider directory", () => { + expect(sources).toContain(TRANSPORT_FILE); + expect(sources.length).toBeGreaterThan(1); + }); + + // A detector that finds nothing anywhere is indistinguishable from a broken one, + // so the file that is SUPPOSED to speak Hrana must light it up - every token, + // because the transport is also the record of what the wire contains. + test.each(WIRE_VOCABULARY)("the transport itself uses %s, proving the detector reads real code", (token) => { + const tokens = findWireLeaks(TRANSPORT_FILE, readProviderSource(TRANSPORT_FILE)).map((leak) => leak.token); + + expect(tokens).toContain(token); + }); + + test(`the Hrana envelope is used only in ${TRANSPORT_FILE}`, () => { + const leaks = sources + .filter((file) => file !== TRANSPORT_FILE) + .flatMap((file) => findWireLeaks(file, readProviderSource(file))); + + expect(violationReport(leaks)).toBe(""); + }); +}); + +describe("the seam guard detector", () => { + /** + * Everything a compliant provider file legitimately does: name the envelope in + * prose, read the neutral result, and issue SQL that uses the one SQLite function + * whose name is also a payload field. None of it is a leak, and none of it may + * fire. + */ + const COMPLIANT_SAMPLE = ` +/** + * Prose may name the envelope: the baton, base_url and query_duration_ms all stay + * behind the seam. Even result.last_insert_rowid written in a comment is prose. + */ +import type { LibSQLTransport } from "./transport"; + +const INSERTED = "SELECT last_insert_rowid() AS insertedId"; + +export async function insertedId(transport: LibSQLTransport) { + const result = await transport.execute(INSERTED); + const { rows, fieldNames, columnTypes } = result; + return { rows, fieldNames, columnTypes, id: result.lastInsertRowId, ms: result.executionTimeMs }; +} +`; + + const VIOLATING_SAMPLE = ` +export function readEnvelope(payload: Record) { + const { baton } = payload; + const result = payload["result"] as Record; + return { baton, id: result.last_insert_rowid, ms: result.query_duration_ms, url: payload.base_url }; +} +`; + + test("passes a file that stays behind the seam", () => { + expect(findWireLeaks("introspect.ts", COMPLIANT_SAMPLE)).toEqual([]); + }); + + test("reports every crossing in a file that does not, with the rule attached", () => { + const leaks = findWireLeaks("introspect.ts", VIOLATING_SAMPLE); + + expect(leaks.map((leak) => leak.token).sort()).toEqual([ + "base_url", + "baton", + "baton", + "last_insert_rowid", + "query_duration_ms", + ]); + expect(violationReport(leaks)).toContain("The Hrana envelope leaked out of"); + expect(violationReport(leaks)).toContain('uses "baton"'); + }); +}); diff --git a/tests/unit/lib/connection-string-parser.test.ts b/tests/unit/lib/connection-string-parser.test.ts index 71ae7e1e..a62cbe96 100644 --- a/tests/unit/lib/connection-string-parser.test.ts +++ b/tests/unit/lib/connection-string-parser.test.ts @@ -350,6 +350,52 @@ describe("parseConnectionString", () => { }); }); + // ── libSQL ────────────────────────────────────────────────────────────── + + describe("libsql:// URLs", () => { + test("parses the URL Turso's own CLI prints, token and all", () => { + const result = parseConnectionString( + "libsql://libredb-probe-424-cevheri.aws-eu-west-1.turso.io?authToken=jwt-123", + ); + + expect(result).not.toBeNull(); + expect(result!.type).toBe("libsql"); + expect(result!.host).toBe("libredb-probe-424-cevheri.aws-eu-west-1.turso.io"); + // 443 under required TLS: `libsql://` has no plaintext form, and Turso serves + // every database over HTTPS on a hostname that identifies the database. + expect(result!.port).toBe("443"); + expect(result!.sslMode).toBe("require"); + // The credential is a TOKEN, and it rides in the query string rather than in + // the authority - so it lands in `password`, which is the field the provider + // sends as a bearer credential. + expect(result!.password).toBe("jwt-123"); + }); + + test("keeps an explicit port, for a self-hosted server behind TLS", () => { + expect(parseConnectionString("libsql://sqld.internal:8443?authToken=t")!.port).toBe("8443"); + }); + + test("falls back to a token written in the authority", () => { + expect(parseConnectionString("libsql://ignored:jwt-456@db.turso.io")!.password).toBe("jwt-456"); + }); + + test("names no database, because on libSQL the database IS the host", () => { + expect(parseConnectionString("libsql://db.turso.io?authToken=t")!.database).toBeUndefined(); + }); + + test("carries no token when the URL holds none, rather than an empty one", () => { + expect(parseConnectionString("libsql://db.turso.io")!.password).toBeUndefined(); + }); + + test("answers null for a libsql:// string that is not a URL", () => { + expect(parseConnectionString("libsql://:::bad")).toBeNull(); + }); + + test("detects the scheme without parsing it", () => { + expect(detectConnectionStringType("libsql://db.turso.io?authToken=t")).toBe("libsql"); + }); + }); + // ── ClickHouse ────────────────────────────────────────────────────────── describe("clickhouse://, http:// and https:// URLs", () => { diff --git a/tests/unit/lib/db-ui-config.test.ts b/tests/unit/lib/db-ui-config.test.ts index 87994617..6bfbad48 100644 --- a/tests/unit/lib/db-ui-config.test.ts +++ b/tests/unit/lib/db-ui-config.test.ts @@ -19,6 +19,7 @@ const ALL_TYPES: DatabaseType[] = [ "opensearch", "trino", "cassandra", + "libsql", ]; describe("db-ui-config", () => { @@ -51,7 +52,10 @@ describe("db-ui-config", () => { // full URI for; everything else is field-based. Druid is field-based on purpose: // it has no URI convention for its HTTP SQL API (its JDBC driver uses // `jdbc:avatica:remote:url=...`), so there is no string a user could paste. - const withToggle = new Set(["mongodb", "couchbase", "clickhouse"]); + // libSQL joins them: `libsql://-.turso.io?authToken=` is the + // URL Turso's own CLI prints, so there is a real string to paste here - unlike + // Trino, whose canonical form is a JDBC URL. + const withToggle = new Set(["mongodb", "couchbase", "clickhouse", "libsql"]); for (const type of ALL_TYPES) { expect(getDBConfig(type).showConnectionStringToggle).toBe(withToggle.has(type)); } @@ -235,6 +239,7 @@ describe("db-showcase", () => { "clickhouse", "druid", "trino", + "libsql", "libredb", ]); }); diff --git a/tests/unit/lib/sql/fence-tags.test.ts b/tests/unit/lib/sql/fence-tags.test.ts index c2dc5ea9..044b5c99 100644 --- a/tests/unit/lib/sql/fence-tags.test.ts +++ b/tests/unit/lib/sql/fence-tags.test.ts @@ -43,6 +43,9 @@ describe("fenceTagEngine", () => { expect(fenceTagEngine("sqlserver")).toBe("mssql"); expect(fenceTagEngine("mongo")).toBe("mongodb"); expect(fenceTagEngine("n1ql")).toBe("couchbase"); + // The product name a model is likelier to write than the protocol's: a block + // tagged `turso` holds a statement for a libSQL connection. + expect(fenceTagEngine("turso")).toBe("libsql"); }); test("a generic tag and an absent one name no engine, so they contradict none", () => { diff --git a/tests/unit/schema-diff/migration-generator.test.ts b/tests/unit/schema-diff/migration-generator.test.ts index 1382330e..79276f6f 100644 --- a/tests/unit/schema-diff/migration-generator.test.ts +++ b/tests/unit/schema-diff/migration-generator.test.ts @@ -814,6 +814,25 @@ describe("generateMigrationSQL: Cassandra spells ADD and DROP without the COLUMN expect(sql).toContain("-- Apache Cassandra: Cannot add a foreign key"); expect(sql).toContain("-- Apache Cassandra: Cannot drop a foreign key"); }); + + test("libSQL declines both directions too, because SQLite declares a key only in CREATE TABLE", () => { + // Measured over Hrana on sqld 0.24.33: `ALTER TABLE t ADD CONSTRAINT fk FOREIGN KEY + // (c) REFERENCES u(id)` is "near CONSTRAINT ... syntax error", and so is the DROP. + // The generic branch would emit both, so both are declined here - and the column + // statements around them are NOT: libSQL accepts ADD COLUMN and DROP COLUMN, which + // is what makes this narrower than the sqlite branch beside it. + const sql = generateMigrationSQL(makeModifiedTableDiff(), "libsql"); + + expect(sql).not.toContain("ADD CONSTRAINT"); + expect(sql).not.toContain("DROP CONSTRAINT"); + expect(sql).toContain("-- libSQL: Cannot add a foreign key"); + expect(sql).toContain("-- libSQL: Cannot drop a foreign key"); + expect(sql).toContain("ADD COLUMN"); + expect(sql).toContain("DROP COLUMN"); + // SQLite runs its own transaction and this provider could not continue one it + // emitted, so the file carries no wrapper. + expect(sql).not.toContain("BEGIN;"); + }); }); describe("generateMigrationSQL: Cassandra declines CREATE TABLE rather than guess the partitioning", () => { @@ -861,6 +880,13 @@ const MODIFIED_COLUMN_COVERAGE: Record { ["oracle", "code"], ["mssql", "code"], ["sqlite", "code"], + ["libsql", "code"], // Trino, probed 2026-08-20 on 476: `#` opens nothing in either position. // `SELECT 1 AS a # trailing` is "line 1:15: mismatched input '#'" and // `SELECT # x` is "line 1:8: mismatched input '#'", so the rest of the line is @@ -109,7 +110,7 @@ describe("resolveSqlGrammar", () => { expect(resolveSqlGrammar("oracle").alternateQuoting).toBe(true); }); - test.each(["mysql", "clickhouse", "postgres", "mssql", "sqlite", "trino", "cassandra"])( + test.each(["mysql", "clickhouse", "postgres", "mssql", "sqlite", "libsql", "trino", "cassandra"])( "%s does not read `q'…'` as a literal", (type) => { expect(resolveSqlGrammar(type).alternateQuoting).toBe(false); @@ -144,6 +145,7 @@ describe("resolveSqlGrammar", () => { test.each<[DatabaseType, BracketGrammar]>([ ["mssql", "quoted-identifier"], ["sqlite", "quoted-identifier"], + ["libsql", "quoted-identifier"], ["clickhouse", "subscript"], ["postgres", "subscript"], // Trino, probed on 476, and BOTH halves of the rule were measured rather than one @@ -213,6 +215,7 @@ describe("resolveSqlGrammar", () => { ["clickhouse", "nesting"], ["mysql", "flat"], ["sqlite", "flat"], + ["libsql", "flat"], ["oracle", "flat"], // Trino, probed on 476: `SELECT /* a /* b */ 1 AS a` returns the column, so the // FIRST `*/` closed the run. A nesting reader would have seen an unterminated @@ -282,6 +285,7 @@ describe("resolveSqlGrammar", () => { ["mssql", false], ["trino", false], ["sqlite", false], + ["libsql", false], ])("%s reads `//` as a line comment: %s", (type, doubleSlashComment) => { expect(resolveSqlGrammar(type).doubleSlashComment).toBe(doubleSlashComment); }); @@ -347,6 +351,10 @@ const GRAMMAR_COVERAGE: Record = { postgres: "established", mysql: "established", sqlite: "established", + // Re-measured over Hrana rather than inherited from the row above: all four facts + // answered the same way on sqld 0.24.33 (#, brackets, nesting, q'…'), which is why + // the registry shares SQLite's grammar object. + libsql: "established", oracle: "established", mssql: "established", clickhouse: "established", @@ -391,6 +399,7 @@ const SQL_TEXT_COVERAGE: Record = { postgres: true, mysql: true, sqlite: true, + libsql: true, oracle: true, mssql: true, clickhouse: true, From 14b9f07c14dcd799dff4de2db50e2db3094f1f56 Mon Sep 17 00:00:00 2001 From: cevheri Date: Thu, 27 Aug 2026 04:10:18 +0300 Subject: [PATCH 2/5] test(e2e,docs): the browser settled two claims the spec had guessed at (#424) Gate 5 and gate 7 for the libSQL provider, and both cost a correction rather than a tick. The spec asserted that libSQL's connection dialog renders no Username and no Database input, because `connectionFields` names neither. The browser refuted it: those two fields decide what a SAVE writes, and `ConnectionModal` draws Host, Username, Password and Database for every engine that is not file-based - so Druid and the two search engines have shown a Database box they do not take for as long as they have shipped. The assertion now pins the behaviour as it is, and the comment in `use-connection-form.ts` that claims otherwise is filed as U22. Also filed U23: the Storage tab labels its remainder "Other (TOAST, FSM)" on every engine. Both are PostgreSQL structures, and on libSQL that read 4.00 KB of a real 64 KB database under another engine's vocabulary. The login helper takes `.first()` on both inputs: a second, hidden pair exists for a moment after hydration, so a strict locator failed locally while passing in CI. The spec is now verifiable in both places - 6 passed against a live server. Gate 7 passed on a libSQL connection: plan mode grounded on the real schema (2 tables, ctx_a49a) and drafted `SELECT country, COUNT(*) ... FROM probe_customers ... LIMIT 1`, which then ran unchanged against sqld and answered `de 1`. The rail says what it should where it cannot execute: "Agent mode has no read-only statement path on libSQL". --- docs/BACKLOG.md | 44 +++++++++++++++++++++++++++++++++++++ e2e/libsql-provider.spec.ts | 34 ++++++++++++++++++++++------ 2 files changed, 71 insertions(+), 7 deletions(-) diff --git a/docs/BACKLOG.md b/docs/BACKLOG.md index 94547438..e2889050 100644 --- a/docs/BACKLOG.md +++ b/docs/BACKLOG.md @@ -427,6 +427,50 @@ Narrow, and it does not throw: the file generates, and the failure happens when recreation rather than a statement, and a test pins both directions for it - the way `tests/unit/schema-diff/migration-generator.test.ts` now pins them for `libsql` and `cassandra`. +### U22. The connection form renders fields the engine does not take, and a comment says otherwise + +Found 2026-08-27 in the browser while registering `libsql` (issue #424, Phase 5). + +`DB_UI_CONFIG[type].connectionFields` decides what a save WRITES +(`buildConnection` in `src/hooks/use-connection-form.ts`), and its comment there says it is +"the same list the modal renders inputs from". It is not. `ConnectionModal.tsx` draws Host, +Username, Password and Database for every engine that is not file-based, so: + +- **libSQL** shows a Username box for an engine that has no user names at all, +- **Druid**, **Elasticsearch** and **OpenSearch** show a Database box none of them takes. + +Nothing is written from those boxes, so a connection is not corrupted by typing in one - the +cost is a user filling a field that is silently discarded, and a comment that misdescribes +the code beside it. Measured for libSQL: `#user` and `#database` both render, and neither +value reaches the saved connection. + +**Done when:** the modal renders an addressing input only when `connectionFields` names it, +with the four engines above checked in a browser rather than in a mock - or, if the fields +are deliberately universal, the comment in `use-connection-form.ts` says so instead. Either +way `e2e/libsql-provider.spec.ts` has the assertion that pins today's behaviour and must +move with it. + +### U23. The Storage tab names PostgreSQL internals on every engine + +Seen 2026-08-27 on a libSQL connection (issue #424, Phase 5), and it is not libSQL's. + +`src/components/monitoring/tabs/StorageTab.tsx:179` labels the remainder of the breakdown +**"Other (TOAST, FSM)"** unconditionally. TOAST and the free space map are PostgreSQL +storage structures. SQLite and libSQL have neither - what the remainder actually holds +there is the schema, the freelist and page overhead - and neither do MySQL, Oracle, SQL +Server, ClickHouse or any of the HTTP engines. On libSQL it read `4.00 KB` under that +label against a real 64 KB database, so the number is right and the words are another +engine's. + +The same shape as a shared provider's refusal naming the wrong product: the reading is +sound, the vocabulary is borrowed. Narrow, cosmetic, and it misleads exactly the reader +who is trying to account for the bytes. + +**Done when:** the label is engine-neutral ("Other" / "Overhead"), or comes from the +provider's own labels the way the maintenance and slow-query wordings already do +(`ProviderLabels`), with a component test pinning it for one PostgreSQL and one +non-PostgreSQL engine. + --- ## Value interpolation diff --git a/e2e/libsql-provider.spec.ts b/e2e/libsql-provider.spec.ts index c72a57fe..7e8108be 100644 --- a/e2e/libsql-provider.spec.ts +++ b/e2e/libsql-provider.spec.ts @@ -17,8 +17,12 @@ import { test, expect } from "@playwright/test"; test.describe("libSQL in the connection dialog", () => { test.beforeEach(async ({ page }) => { await page.goto("/login"); - await page.locator('input[type="email"]').fill("user@libredb.org"); - await page.locator('input[type="password"]').fill("test-user"); + // `.first()` on both, unlike the older provider specs: a second, hidden pair of + // inputs exists for a moment after hydration, and a strict locator fails on it + // locally while passing in CI. Pinning the first match makes the spec verifiable in + // both places rather than in CI alone. + await page.locator('input[type="email"]').first().fill("user@libredb.org"); + await page.locator('input[type="password"]').first().fill("test-user"); await page.getByRole("button", { name: "Sign In" }).click(); await page.waitForURL("/"); await expect(page.locator("text=Query 1").first()).toBeVisible({ timeout: 10000 }); @@ -51,14 +55,30 @@ test.describe("libSQL in the connection dialog", () => { await expect(dialog.getByText(/turso db tokens create/)).toBeVisible(); }); - test("offers no user field, because libSQL has no user names", async ({ page }) => { + test("offers the URL form Turso prints, so a pasted connection is a real one", async ({ page }) => { const dialog = page.locator('[role="dialog"]'); await dialog.getByRole("button", { name: "libSQL", exact: true }).click(); + await dialog.getByText("Connection String", { exact: true }).first().click(); - await expect(dialog.locator("#user")).toHaveCount(0); - // No database field either: on Turso Cloud the database IS the hostname, and a - // self-hosted server serves one per namespace hostname. - await expect(dialog.locator("#database")).toHaveCount(0); + // The placeholder is the shape `turso db show --url` prints, token and all - the + // one string a user has in front of them. + await expect(dialog.locator('input[placeholder*="turso.io"]')).toBeVisible(); + }); + + test("renders the Username and Database inputs even though libSQL takes neither", async ({ page }) => { + // Pinned as it IS rather than as it should be, because the browser is what settled + // it: `connectionFields` in `db-ui-config.ts` decides what a save WRITES, not what + // the form renders - `ConnectionModal` draws Host, Username, Password and Database + // for every engine that is not file-based. So libSQL shows a Username box it has no + // user names for, exactly as Druid and the two search engines show a Database box + // they do not take. Nothing is written from either (BACKLOG U22), and a test that + // asserted these were absent would have been asserting a comment rather than the + // product. + const dialog = page.locator('[role="dialog"]'); + await dialog.getByRole("button", { name: "libSQL", exact: true }).click(); + + await expect(dialog.locator("#user")).toHaveCount(1); + await expect(dialog.locator("#database")).toHaveCount(1); }); test("claims no wire-compatible relative it has not probed", async ({ page }) => { From 233e987a08becf9d0dd7b950c5738e02028aa690 Mon Sep 17 00:00:00 2001 From: cevheri Date: Thu, 27 Aug 2026 12:35:50 +0300 Subject: [PATCH 3/5] fix(libsql): untrack the probe harness, pin the image it was measured on, and name libSQL where a count is published Review of #511. Every claim below was re-measured rather than taken from the report. - The gate-4 harness and its two result files were COMMITTED. `.gitignore` named `/probe-424.ts` literally, and this run's harness was named per engine, so none of the three matched. Untracked and the patterns globbed (`/probe-*.ts`, `/probe-results*.json`). They carried no token and no hostname - the harness takes both from argv - but `tsconfig`'s `**/*.ts` did pull the harness into typecheck. - `database-compose.yml` claimed the image was "pinned to the exact build" while asking for `:latest`. `:latest` is a ROLLING REBUILD of the same version: measured 2026-08-27 it answers `sqld 0.24.33 (f8fb14f3 2026-08-11)` where the `v0.24.33` tag answers `sqld 0.24.33 (40a151bd 2025-12-19)` - one version number, two digests. Pinned to `v0.24.33` and re-probed against it surface by surface: the same 17 of 19, the same four refusals with byte-identical wording, the same quoted-`"notnull"` behaviour. The doc now records both builds and that the pin reproduces the measurements. - Registering a fifteenth external engine moved denominators the PR left behind: `Chart.yaml` and the operator CSV both still described "fourteen engines" without libSQL (ArtifactHub and OperatorHub read those), the showcase and hero-proof comments still explained fifteen pills under a claim of fourteen, and the wire-compatibility hint still counted fourteen driver buttons. `compatibility.ts` quoted a README sentence ("fourteen drivers reach forty named engines") that no longer exists. Each is now the measured figure: 16 type-ids, 15 external, 26 relatives, 41 named products. - The chart's own comment says a new engine's keyword belongs in the release that ships it, and libSQL had none. Added `libsql` and `turso` - the searched term is the product name. No further chart bump: 0.1.53 is unreleased. - A failed connection-string paste listed nine supported schemes and omitted the tenth, `libsql://`, which the parser accepts. Two review findings are real and belong to the codebase rather than to this PR, so they are filed instead of fixed: D35, five `fetch`-based transports read `ssl.mode` and discard the rest of the TLS panel (Couchbase already solves it through `node:https`); D36, `getConnectionInfo` masks `:pass@` but not `?authToken=`, dead today with test-only callers. Gates in the worktree: format, lint, typecheck, knip, test (13175 pass / 0 fail), build, plus chart:check, readme:check, security:check, channels:showcase:check and `helm lint charts/libredb-studio --strict`. --- .gitignore | 7 +- charts/libredb-studio/Chart.yaml | 12 +- database-compose.yml | 8 +- docs/BACKLOG.md | 46 +++ docs/providers/libsql.md | 1 + ...studio-operator.clusterserviceversion.yaml | 6 +- ...studio-operator.clusterserviceversion.yaml | 6 +- .../helm-charts/libredb-studio/Chart.yaml | 12 +- probe-libsql.ts | 58 ---- probe-results-cloud.json | 294 ------------------ probe-results-self.json | 294 ------------------ src/components/WireCompatibilityHint.tsx | 2 +- src/components/login/hero-proof.tsx | 2 +- src/hooks/use-connection-form.ts | 2 +- src/lib/agent/investigation.ts | 2 +- src/lib/db-showcase.ts | 2 +- src/lib/db/compatibility.ts | 8 +- 17 files changed, 90 insertions(+), 672 deletions(-) delete mode 100644 probe-libsql.ts delete mode 100644 probe-results-cloud.json delete mode 100644 probe-results-self.json diff --git a/.gitignore b/.gitignore index 88f978d1..5b4eff0e 100644 --- a/.gitignore +++ b/.gitignore @@ -180,6 +180,9 @@ deploy/digitalocean/droplet/scripts/99-img-check.sh -# local gate-4 probe harness (issue #424), never committed -/probe-424.ts +# local gate-4 probe harness (issue #424), never committed. Globbed rather than named: +# PR #511 committed probe-libsql.ts and two probe-results-*.json files because the harness +# was named per engine and the literal `/probe-424.ts` matched none of them. +/probe-*.ts /probe-results/ +/probe-results*.json diff --git a/charts/libredb-studio/Chart.yaml b/charts/libredb-studio/Chart.yaml index 81881b68..cc16c1cc 100644 --- a/charts/libredb-studio/Chart.yaml +++ b/charts/libredb-studio/Chart.yaml @@ -1,6 +1,6 @@ apiVersion: v2 name: libredb-studio -description: Web-based SQL IDE for cloud-native teams supporting fourteen engines - PostgreSQL, MySQL, SQLite, Oracle, SQL Server, MongoDB, Redis, Couchbase, ClickHouse, Apache Druid, Elasticsearch, OpenSearch, Apache Trino and Apache Cassandra +description: Web-based SQL IDE for cloud-native teams supporting fifteen engines - PostgreSQL, MySQL, SQLite, Oracle, SQL Server, MongoDB, Redis, Couchbase, ClickHouse, Apache Druid, Elasticsearch, OpenSearch, Apache Trino, Apache Cassandra and libSQL type: application version: 0.1.53 appVersion: "0.13.4" @@ -31,6 +31,10 @@ keywords: # shipped in 0.12.0 (#448) - #167 makes a keyword-only fix cost a chart version # of its own, so a new engine's keyword belongs in the release that ships it. - cassandra + # Two keywords for one engine, and the second is the one that gets searched: the + # provider is registered as `libsql`, but the product an evaluator types is Turso. + - libsql + - turso - web-ide maintainers: - name: cevheri @@ -40,9 +44,9 @@ annotations: artifacthub.io/license: MIT artifacthub.io/prerelease: "false" # False: 0.1.53 changes no packaged template and no value - it names one more engine in - # the README, libSQL, which is a new provider in the app rather than a chart change. The - # README is a packaged file, so #167 costs it a chart version even though nothing an - # operator deploys moves. + # the README, the description and the keywords, libSQL, which is a new provider in the + # app rather than a chart change. The README is a packaged file, so #167 costs it a + # chart version even though nothing an operator deploys moves. # False: 0.1.52 adds one value, config.authCookieSecure, and changes no behaviour on its # own - unset (the default) writes no AUTH_COOKIE_SECURE and the app keeps deciding, so # every existing install renders exactly as before. It makes an already-supported setting diff --git a/database-compose.yml b/database-compose.yml index 75a016f3..a79c8cf1 100644 --- a/database-compose.yml +++ b/database-compose.yml @@ -458,7 +458,13 @@ services: # trino and clickhouse services follow: the refusal wording recorded in # docs/providers/libsql.md ("unsupported statement: VACUUM") is a claim about sqld # 0.24.33, and Turso Cloud words the identical refusal differently. - image: ghcr.io/tursodatabase/libsql-server:latest + # + # The tag is needed even though `:latest` reports the same 0.24.33: measured 2026-08-27, + # `:latest` is a ROLLING REBUILD of that version - it answers `/version` with + # `sqld 0.24.33 (f8fb14f3 2026-08-11)` where `v0.24.33` answers + # `sqld 0.24.33 (40a151bd 2025-12-19)`, two different digests under one version number. + # So `:latest` is not reproducible even while the number it prints looks pinned. + image: ghcr.io/tursodatabase/libsql-server:v0.24.33 container_name: libredb-libsql restart: unless-stopped environment: diff --git a/docs/BACKLOG.md b/docs/BACKLOG.md index e2889050..b59ef962 100644 --- a/docs/BACKLOG.md +++ b/docs/BACKLOG.md @@ -471,6 +471,52 @@ provider's own labels the way the maintenance and slow-query wordings already do (`ProviderLabels`), with a component test pinning it for one PostgreSQL and one non-PostgreSQL engine. +### D35. Five HTTP providers read the SSL mode and drop the rest of the TLS panel + +Found 2026-08-27 in the #511 review (issue #424, Phase 5). Not libSQL's - libSQL is the +fifth of five instances of one gap, and the fix already exists in the codebase. + +`ssl.caCert`, `ssl.clientCert`, `ssl.clientKey` and `ssl.rejectUnauthorized` reach the +driver on every provider that uses one. On the providers that speak HTTP through global +`fetch` they reach nothing: ClickHouse, Druid, Elasticsearch/OpenSearch, Trino and libSQL +each read `ssl.mode` only, to decide `http:` against `https:`, and Node's `fetch` cannot carry +a custom CA or relax verification without an undici `Agent` as `dispatcher` - and undici +must not become a dependency. So a self-hosted server with a private CA is reachable only +by trusting it at the OS level, and the form's own TLS fields silently do nothing. + +**Couchbase already solved this and is the pattern**: `providers/document/couchbase/http-transport.ts` +sends plaintext through `fetch` and TLS through `node:https`, a built-in that takes +`ca`/`cert`/`key`/`rejectUnauthorized` directly (D26). Its `CouchbaseTlsMaterial` mapping, +including `rejectUnauthorized: ssl.rejectUnauthorized ?? ssl.mode !== "require"`, is the +behaviour the other five need. + +Not a defect in what any of them measures - it is a field the form offers and the transport +discards, which is the kind of silence a security setting must not have. + +**Done when:** the TLS material mapping is shared rather than copied, the five `fetch` +transports route TLS through it, and one test per transport pins that a supplied CA and a +`verify-*` mode reach the request options - plus one that a `require` mode does not verify. + +### D36. `getConnectionInfo` masks a password in a connection string but not a token in a query string + +Found 2026-08-27 in the #511 review. Pre-existing, dead today, and cheap to close before it +is not. + +`base-provider.ts`'s `protected getConnectionInfo()` returns +`connectionString.replace(/:([^:@]+)@/, ":***@")`, which masks `:secret@` in an authority and +nothing else. A libSQL connection string carries its credential in the query string instead - +`libsql://db-org.turso.io?authToken=` - so the whole token would survive the mask. The +same is true of any `?password=`/`?sslkey=` form. + +It is currently unreachable: the only callers are in `tests/unit/db/base-provider.test.ts`, so +no production path prints it. That is the reason this is a backlog entry rather than an issue, +and also the reason it is worth doing - a future health-panel caller would inherit a leak that +looks redacted. + +**Done when:** the mask also strips the value of every credential-shaped query parameter +(`authToken`, `password`, `token`, `sslkey`), with a test per shape - or the method is deleted, +since nothing in `src/` calls it. + --- ## Value interpolation diff --git a/docs/providers/libsql.md b/docs/providers/libsql.md index 3affb19c..1b8ba963 100644 --- a/docs/providers/libsql.md +++ b/docs/providers/libsql.md @@ -25,6 +25,7 @@ | **Tests** | [`tests/integration/db/libsql-provider.test.ts`](../../tests/integration/db/libsql-provider.test.ts) + [`tests/unit/db/libsql/`](../../tests/unit/db/libsql/) | | **Tracking issue** | [#424 — the database coverage map](https://github.com/libredb/libredb-studio/issues/424) | | **Probed against** | `ghcr.io/tursodatabase/libsql-server` reporting `sqld 0.24.33 (f8fb14f3 2026-08-11)`, and a Turso Cloud database in `aws-eu-west-1`, both on 2026-08-27 | +| **Reproducible with** | `ghcr.io/tursodatabase/libsql-server:v0.24.33`, the tag `database-compose.yml` pins. It is a DIFFERENT build of the same version - `sqld 0.24.33 (40a151bd 2025-12-19)`, because `:latest` is a rolling rebuild - and it was re-probed surface by surface on 2026-08-27: the same 17 of 19, the same four refusals with byte-identical wording, and the same `"notnull"` behaviour. Every measurement below therefore holds on the pinned tag as well as on the build it was taken from. | --- diff --git a/operator/bundle/manifests/libredb-studio-operator.clusterserviceversion.yaml b/operator/bundle/manifests/libredb-studio-operator.clusterserviceversion.yaml index c6f3b3b7..732b3503 100644 --- a/operator/bundle/manifests/libredb-studio-operator.clusterserviceversion.yaml +++ b/operator/bundle/manifests/libredb-studio-operator.clusterserviceversion.yaml @@ -23,10 +23,10 @@ metadata: categories: Database, Developer Tools containerImage: ghcr.io/libredb/libredb-studio-operator:0.13.4 createdAt: "2026-08-24T19:49:18Z" - description: Open-source web-based SQL IDE for fourteen engines - PostgreSQL, + description: Open-source web-based SQL IDE for fifteen engines - PostgreSQL, MySQL, Oracle, SQL Server, SQLite, MongoDB, Redis, Couchbase, ClickHouse, Apache - Druid, Elasticsearch, OpenSearch, Apache Trino and Apache Cassandra - with AI-powered - query assistance. + Druid, Elasticsearch, OpenSearch, Apache Trino, Apache Cassandra and libSQL - + with AI-powered query assistance. operators.operatorframework.io/builder: operator-sdk-v1.42.3 operators.operatorframework.io/project_layout: helm.sdk.operatorframework.io/v1 repository: https://github.com/libredb/libredb-studio diff --git a/operator/config/manifests/bases/libredb-studio-operator.clusterserviceversion.yaml b/operator/config/manifests/bases/libredb-studio-operator.clusterserviceversion.yaml index 8d52e3eb..0c4a907f 100644 --- a/operator/config/manifests/bases/libredb-studio-operator.clusterserviceversion.yaml +++ b/operator/config/manifests/bases/libredb-studio-operator.clusterserviceversion.yaml @@ -6,10 +6,10 @@ metadata: capabilities: Basic Install categories: Database, Developer Tools containerImage: ghcr.io/libredb/libredb-studio-operator:0.0.0 - description: Open-source web-based SQL IDE for fourteen engines - PostgreSQL, + description: Open-source web-based SQL IDE for fifteen engines - PostgreSQL, MySQL, Oracle, SQL Server, SQLite, MongoDB, Redis, Couchbase, ClickHouse, Apache - Druid, Elasticsearch, OpenSearch, Apache Trino and Apache Cassandra - with AI-powered - query assistance. + Druid, Elasticsearch, OpenSearch, Apache Trino, Apache Cassandra and libSQL - + with AI-powered query assistance. repository: https://github.com/libredb/libredb-studio support: LibreDB labels: diff --git a/operator/helm-charts/libredb-studio/Chart.yaml b/operator/helm-charts/libredb-studio/Chart.yaml index 81881b68..cc16c1cc 100644 --- a/operator/helm-charts/libredb-studio/Chart.yaml +++ b/operator/helm-charts/libredb-studio/Chart.yaml @@ -1,6 +1,6 @@ apiVersion: v2 name: libredb-studio -description: Web-based SQL IDE for cloud-native teams supporting fourteen engines - PostgreSQL, MySQL, SQLite, Oracle, SQL Server, MongoDB, Redis, Couchbase, ClickHouse, Apache Druid, Elasticsearch, OpenSearch, Apache Trino and Apache Cassandra +description: Web-based SQL IDE for cloud-native teams supporting fifteen engines - PostgreSQL, MySQL, SQLite, Oracle, SQL Server, MongoDB, Redis, Couchbase, ClickHouse, Apache Druid, Elasticsearch, OpenSearch, Apache Trino, Apache Cassandra and libSQL type: application version: 0.1.53 appVersion: "0.13.4" @@ -31,6 +31,10 @@ keywords: # shipped in 0.12.0 (#448) - #167 makes a keyword-only fix cost a chart version # of its own, so a new engine's keyword belongs in the release that ships it. - cassandra + # Two keywords for one engine, and the second is the one that gets searched: the + # provider is registered as `libsql`, but the product an evaluator types is Turso. + - libsql + - turso - web-ide maintainers: - name: cevheri @@ -40,9 +44,9 @@ annotations: artifacthub.io/license: MIT artifacthub.io/prerelease: "false" # False: 0.1.53 changes no packaged template and no value - it names one more engine in - # the README, libSQL, which is a new provider in the app rather than a chart change. The - # README is a packaged file, so #167 costs it a chart version even though nothing an - # operator deploys moves. + # the README, the description and the keywords, libSQL, which is a new provider in the + # app rather than a chart change. The README is a packaged file, so #167 costs it a + # chart version even though nothing an operator deploys moves. # False: 0.1.52 adds one value, config.authCookieSecure, and changes no behaviour on its # own - unset (the default) writes no AUTH_COOKIE_SECURE and the app keeps deciding, so # every existing install renders exactly as before. It makes an already-supported setting diff --git a/probe-libsql.ts b/probe-libsql.ts deleted file mode 100644 index 3c571541..00000000 --- a/probe-libsql.ts +++ /dev/null @@ -1,58 +0,0 @@ -/** Gate-4 harness: every surface, called separately, against a live libSQL. */ -import { LibSQLProvider } from "./src/lib/db/providers/sql/libsql"; -import type { DatabaseConnection } from "./src/lib/db/types"; - -const [, , label, host, portRaw, token] = process.argv; -const connection = { - id: `probe-${label}`, - name: `libSQL ${label}`, - type: "libsql", - host, - ...(portRaw === "-" ? {} : { port: Number(portRaw) }), - ...(token && token !== "-" ? { password: token } : {}), - ...(portRaw === "-" ? { ssl: { mode: "require" } } : {}), - createdAt: new Date(), -} as unknown as DatabaseConnection; - -const provider = new LibSQLProvider(connection); -const results: Record = {}; - -async function surface(name: string, run: () => Promise): Promise { - try { - results[name] = { ok: true, value: await run() }; - } catch (error) { - results[name] = { ok: false, error: error instanceof Error ? error.message : String(error) }; - } -} - -await surface("connect", async () => { - await provider.connect(); - return "connected"; -}); -await surface("query", () => provider.query("SELECT id, name, country FROM probe_customers ORDER BY id")); -await surface("queryParams", () => provider.query("SELECT COUNT(*) AS c FROM probe_orders WHERE customer_id = ?", [1])); -await surface("explain", () => provider.query("EXPLAIN QUERY PLAN SELECT * FROM probe_customers WHERE country = 'tr'")); -await surface("write", () => provider.query("UPDATE probe_customers SET country = 'tr' WHERE id = 1")); -await surface("schema", () => provider.getSchema()); -await surface("overview", () => provider.getOverview()); -await surface("health", () => provider.getHealth()); -await surface("performance", () => provider.getPerformanceMetrics()); -await surface("slowQueries", () => provider.getSlowQueries()); -await surface("activeSessions", () => provider.getActiveSessions()); -await surface("tableStats", () => provider.getTableStats()); -await surface("indexStats", () => provider.getIndexStats()); -await surface("storageStats", () => provider.getStorageStats()); -await surface("maintenanceCheck", () => provider.runMaintenance("check")); -await surface("maintenanceReindex", () => provider.runMaintenance("reindex")); -await surface("maintenanceVacuum", () => provider.runMaintenance("vacuum")); -await surface("badStatement", () => provider.query("SELECT * FROM no_such_table")); -await surface("disconnect", async () => { - await provider.disconnect(); - return "disconnected"; -}); - -await Bun.write(`probe-results-${label}.json`, JSON.stringify(results, null, 2)); -for (const [name, outcome] of Object.entries(results)) { - const record = outcome as { ok: boolean; error?: string }; - console.log(record.ok ? "OK " : "ERR ", name.padEnd(20), record.ok ? "" : record.error); -} diff --git a/probe-results-cloud.json b/probe-results-cloud.json deleted file mode 100644 index c5c7727a..00000000 --- a/probe-results-cloud.json +++ /dev/null @@ -1,294 +0,0 @@ -{ - "connect": { - "ok": true, - "value": "connected" - }, - "query": { - "ok": true, - "value": { - "rows": [ - { - "id": 1, - "name": "Ada", - "country": "tr" - }, - { - "id": 2, - "name": "Linus", - "country": "de" - }, - { - "id": 3, - "name": "Grace", - "country": "us" - } - ], - "fields": [ - "id", - "name", - "country" - ], - "rowCount": 3, - "executionTime": 64, - "columnTypes": { - "id": "INTEGER", - "name": "TEXT", - "country": "TEXT" - } - } - }, - "queryParams": { - "ok": true, - "value": { - "rows": [ - { - "c": 666 - } - ], - "fields": [ - "c" - ], - "rowCount": 1, - "executionTime": 67 - } - }, - "explain": { - "ok": true, - "value": { - "rows": [ - { - "id": 3, - "parent": 0, - "notused": 62, - "detail": "SEARCH probe_customers USING INDEX idx_customers_country (country=?)" - } - ], - "fields": [ - "id", - "parent", - "notused", - "detail" - ], - "rowCount": 1, - "executionTime": 98 - } - }, - "write": { - "ok": true, - "value": { - "rows": [], - "fields": [], - "rowCount": 1, - "executionTime": 72 - } - }, - "schema": { - "ok": true, - "value": [ - { - "name": "probe_customers", - "rowCount": 3, - "size": "8 KB", - "columns": [ - { - "name": "id", - "type": "INTEGER", - "nullable": true, - "isPrimary": true - }, - { - "name": "name", - "type": "TEXT", - "nullable": false, - "isPrimary": false - }, - { - "name": "country", - "type": "TEXT", - "nullable": true, - "isPrimary": false, - "defaultValue": "'tr'" - } - ], - "indexes": [ - { - "name": "idx_customers_country", - "columns": [ - "country" - ], - "unique": false - } - ], - "foreignKeys": [] - }, - { - "name": "probe_orders", - "rowCount": 2000, - "size": "52 KB", - "columns": [ - { - "name": "id", - "type": "INTEGER", - "nullable": true, - "isPrimary": true - }, - { - "name": "customer_id", - "type": "INTEGER", - "nullable": true, - "isPrimary": false - }, - { - "name": "amount", - "type": "REAL", - "nullable": true, - "isPrimary": false - }, - { - "name": "note", - "type": "TEXT", - "nullable": true, - "isPrimary": false - } - ], - "indexes": [], - "foreignKeys": [ - { - "columnName": "customer_id", - "referencedTable": "probe_customers", - "referencedColumn": "id" - } - ] - } - ] - }, - "overview": { - "ok": true, - "value": { - "version": "SQLite 3.47.0", - "uptime": "N/A", - "maxConnections": 0, - "databaseSize": "64 KB", - "databaseSizeBytes": 65536, - "tableCount": 2, - "indexCount": 1 - } - }, - "health": { - "ok": true, - "value": { - "databaseSize": "64 KB", - "cacheHitRatio": "N/A", - "slowQueries": [ - { - "query": "Integrity: OK", - "calls": 0, - "avgTime": "N/A" - }, - { - "query": "Journal Mode: wal", - "calls": 0, - "avgTime": "N/A" - } - ], - "activeSessions": [] - } - }, - "performance": { - "ok": true, - "value": { - "deadlocks": 0 - } - }, - "slowQueries": { - "ok": true, - "value": [] - }, - "activeSessions": { - "ok": true, - "value": [] - }, - "tableStats": { - "ok": true, - "value": [ - { - "schemaName": "main", - "tableName": "probe_customers", - "rowCount": 3, - "tableSize": "4 KB", - "tableSizeBytes": 4096, - "indexSize": "4 KB", - "indexSizeBytes": 4096, - "totalSize": "8 KB", - "totalSizeBytes": 8192 - }, - { - "schemaName": "main", - "tableName": "probe_orders", - "rowCount": 2000, - "tableSize": "52 KB", - "tableSizeBytes": 53248, - "indexSize": "0 B", - "indexSizeBytes": 0, - "totalSize": "52 KB", - "totalSizeBytes": 53248 - } - ] - }, - "indexStats": { - "ok": true, - "value": [ - { - "schemaName": "main", - "tableName": "probe_customers", - "indexName": "idx_customers_country", - "columns": [ - "country" - ], - "isUnique": false, - "isPrimary": false, - "indexSize": "4 KB", - "indexSizeBytes": 4096, - "scans": 0 - } - ] - }, - "storageStats": { - "ok": true, - "value": [ - { - "name": "main", - "size": "64 KB", - "sizeBytes": 65536 - } - ] - }, - "maintenanceCheck": { - "ok": true, - "value": { - "success": true, - "executionTime": 67, - "message": "ok" - } - }, - "maintenanceReindex": { - "ok": true, - "value": { - "success": true, - "executionTime": 84, - "message": "REINDEX completed successfully" - } - }, - "maintenanceVacuum": { - "ok": false, - "error": "libSQL servers do not accept VACUUM: only REINDEX and PRAGMA integrity_check are allowed" - }, - "badStatement": { - "ok": false, - "error": "SQLite error: no such table: no_such_table" - }, - "disconnect": { - "ok": true, - "value": "disconnected" - } -} \ No newline at end of file diff --git a/probe-results-self.json b/probe-results-self.json deleted file mode 100644 index c792b262..00000000 --- a/probe-results-self.json +++ /dev/null @@ -1,294 +0,0 @@ -{ - "connect": { - "ok": true, - "value": "connected" - }, - "query": { - "ok": true, - "value": { - "rows": [ - { - "id": 1, - "name": "Ada", - "country": "tr" - }, - { - "id": 2, - "name": "Linus", - "country": "de" - }, - { - "id": 3, - "name": "Grace", - "country": "us" - } - ], - "fields": [ - "id", - "name", - "country" - ], - "rowCount": 3, - "executionTime": 1, - "columnTypes": { - "id": "INTEGER", - "name": "TEXT", - "country": "TEXT" - } - } - }, - "queryParams": { - "ok": true, - "value": { - "rows": [ - { - "c": 666 - } - ], - "fields": [ - "c" - ], - "rowCount": 1, - "executionTime": 1 - } - }, - "explain": { - "ok": true, - "value": { - "rows": [ - { - "id": 3, - "parent": 0, - "notused": 62, - "detail": "SEARCH probe_customers USING INDEX idx_customers_country (country=?)" - } - ], - "fields": [ - "id", - "parent", - "notused", - "detail" - ], - "rowCount": 1, - "executionTime": 1 - } - }, - "write": { - "ok": true, - "value": { - "rows": [], - "fields": [], - "rowCount": 1, - "executionTime": 4 - } - }, - "schema": { - "ok": true, - "value": [ - { - "name": "probe_customers", - "rowCount": 3, - "size": "8 KB", - "columns": [ - { - "name": "id", - "type": "INTEGER", - "nullable": true, - "isPrimary": true - }, - { - "name": "name", - "type": "TEXT", - "nullable": false, - "isPrimary": false - }, - { - "name": "country", - "type": "TEXT", - "nullable": true, - "isPrimary": false, - "defaultValue": "'tr'" - } - ], - "indexes": [ - { - "name": "idx_customers_country", - "columns": [ - "country" - ], - "unique": false - } - ], - "foreignKeys": [] - }, - { - "name": "probe_orders", - "rowCount": 2000, - "size": "52 KB", - "columns": [ - { - "name": "id", - "type": "INTEGER", - "nullable": true, - "isPrimary": true - }, - { - "name": "customer_id", - "type": "INTEGER", - "nullable": true, - "isPrimary": false - }, - { - "name": "amount", - "type": "REAL", - "nullable": true, - "isPrimary": false - }, - { - "name": "note", - "type": "TEXT", - "nullable": true, - "isPrimary": false - } - ], - "indexes": [], - "foreignKeys": [ - { - "columnName": "customer_id", - "referencedTable": "probe_customers", - "referencedColumn": "id" - } - ] - } - ] - }, - "overview": { - "ok": true, - "value": { - "version": "sqld 0.24.33 (f8fb14f3 2026-08-11) (SQLite 3.47.0)", - "uptime": "N/A", - "maxConnections": 0, - "databaseSize": "64 KB", - "databaseSizeBytes": 65536, - "tableCount": 2, - "indexCount": 1 - } - }, - "health": { - "ok": true, - "value": { - "databaseSize": "64 KB", - "cacheHitRatio": "N/A", - "slowQueries": [ - { - "query": "Integrity: OK", - "calls": 0, - "avgTime": "N/A" - }, - { - "query": "Journal Mode: wal", - "calls": 0, - "avgTime": "N/A" - } - ], - "activeSessions": [] - } - }, - "performance": { - "ok": true, - "value": { - "deadlocks": 0 - } - }, - "slowQueries": { - "ok": true, - "value": [] - }, - "activeSessions": { - "ok": true, - "value": [] - }, - "tableStats": { - "ok": true, - "value": [ - { - "schemaName": "main", - "tableName": "probe_customers", - "rowCount": 3, - "tableSize": "4 KB", - "tableSizeBytes": 4096, - "indexSize": "4 KB", - "indexSizeBytes": 4096, - "totalSize": "8 KB", - "totalSizeBytes": 8192 - }, - { - "schemaName": "main", - "tableName": "probe_orders", - "rowCount": 2000, - "tableSize": "52 KB", - "tableSizeBytes": 53248, - "indexSize": "0 B", - "indexSizeBytes": 0, - "totalSize": "52 KB", - "totalSizeBytes": 53248 - } - ] - }, - "indexStats": { - "ok": true, - "value": [ - { - "schemaName": "main", - "tableName": "probe_customers", - "indexName": "idx_customers_country", - "columns": [ - "country" - ], - "isUnique": false, - "isPrimary": false, - "indexSize": "4 KB", - "indexSizeBytes": 4096, - "scans": 0 - } - ] - }, - "storageStats": { - "ok": true, - "value": [ - { - "name": "main", - "size": "64 KB", - "sizeBytes": 65536 - } - ] - }, - "maintenanceCheck": { - "ok": true, - "value": { - "success": true, - "executionTime": 1, - "message": "ok" - } - }, - "maintenanceReindex": { - "ok": true, - "value": { - "success": true, - "executionTime": 4, - "message": "REINDEX completed successfully" - } - }, - "maintenanceVacuum": { - "ok": false, - "error": "libSQL servers do not accept VACUUM: only REINDEX and PRAGMA integrity_check are allowed" - }, - "badStatement": { - "ok": false, - "error": "SQLite error: no such table: no_such_table" - }, - "disconnect": { - "ok": true, - "value": "disconnected" - } -} \ No newline at end of file diff --git a/src/components/WireCompatibilityHint.tsx b/src/components/WireCompatibilityHint.tsx index 12836da3..116f58c9 100644 --- a/src/components/WireCompatibilityHint.tsx +++ b/src/components/WireCompatibilityHint.tsx @@ -11,7 +11,7 @@ interface WireCompatibilityHintProps { /** * Tells the user that the driver they just selected also serves other engines - * (issue #424, Phase 0). It exists because the connection dialog offers fourteen + * (issue #424, Phase 0). It exists because the connection dialog offers sixteen * driver buttons and none of them says "MariaDB", so a MariaDB user has no way to * know that MySQL is the right button. * diff --git a/src/components/login/hero-proof.tsx b/src/components/login/hero-proof.tsx index 1e7553aa..a64c5344 100644 --- a/src/components/login/hero-proof.tsx +++ b/src/components/login/hero-proof.tsx @@ -51,7 +51,7 @@ export const HERO_CLAIMS: readonly { key: string; value: number; unit: string; d // at, so counting it here claimed one external engine more than the product has and put // this page one out of step with the number README.md publishes. The pill for it stays - // it is a provider the connection picker offers - marked as embedded so the reader can - // see why fifteen pills sit under a claim of fourteen. + // see why sixteen pills sit under a claim of fifteen. value: EXTERNAL_DATABASE_TYPES.length, unit: "database engines", detail: "one client, one workspace, every one of them", diff --git a/src/hooks/use-connection-form.ts b/src/hooks/use-connection-form.ts index 004bb3e2..52ef18f2 100644 --- a/src/hooks/use-connection-form.ts +++ b/src/hooks/use-connection-form.ts @@ -512,7 +512,7 @@ export function useConnectionForm({ isOpen, onConnect, editConnection, onTestCon // ClickHouse there, so listing them would promise a paste this form cannot // honour. Trino's own `jdbc:trino://…` is a JDBC URL the parser does not read. message: - "Could not parse connection string. Supported formats: postgres://, mysql://, mongodb://, couchbase://, clickhouse://, http(s)://, redis://, oracle://, mssql://", + "Could not parse connection string. Supported formats: postgres://, mysql://, mongodb://, couchbase://, clickhouse://, libsql://, http(s)://, redis://, oracle://, mssql://", }); return; } diff --git a/src/lib/agent/investigation.ts b/src/lib/agent/investigation.ts index e7f42078..a6d40047 100644 --- a/src/lib/agent/investigation.ts +++ b/src/lib/agent/investigation.ts @@ -960,7 +960,7 @@ const PLAN_DELIVERABLES: Readonly> * `language` arrived with #414 and every sentence that assumed SQL is now written * twice. The tag does NOT vary with it and that is the point of taking the language * separately: the tag is the canonical type-id in both arms, because it is what - * `isQueryFenceTag` accepts (a total record over `DatabaseType`, so all fourteen pass) + * `isQueryFenceTag` accepts (a total record over `DatabaseType`, so all sixteen pass) * and what `rich-text.tsx` and `readPlanStatement` key the editor hand-off on. A draft * a model fenced as ```` ```javascript ```` produces no `plan-statement-drafted` event * at all — the run would be scored as having drafted nothing while the user is looking diff --git a/src/lib/db-showcase.ts b/src/lib/db-showcase.ts index 9adbab09..e5c92d01 100644 --- a/src/lib/db-showcase.ts +++ b/src/lib/db-showcase.ts @@ -69,7 +69,7 @@ export interface ShowcaseDatabase { /** * True for the embedded store, false for a database the user already runs. * - * The showcase shows all fifteen providers while the hero claims fourteen engines, + * The showcase shows all sixteen providers while the hero claims fifteen engines, * and this flag is how the page carries that difference without any surface typing * the word "libredb": the pill it marks is the one the count leaves out. */ diff --git a/src/lib/db/compatibility.ts b/src/lib/db/compatibility.ts index a4620284..1671cb9d 100644 --- a/src/lib/db/compatibility.ts +++ b/src/lib/db/compatibility.ts @@ -74,10 +74,10 @@ export const SHIPPED_DATABASE_TYPES: readonly DatabaseType[] = Object.freeze(Obj /** * Which shipped ids are databases a user already runs, and which one is not. * - * `libredb` is the embedded store this app carries with it; the other fourteen are + * `libredb` is the embedded store this app carries with it; the other fifteen are * external engines you point the product at. Everything published as a database - * count means the external fourteen - README.md's "fourteen drivers reach - * forty named engines", the login hero's engine claim - so the split needs a + * count means the external fifteen - README.md's "fifteen drivers reach + * forty-one named engines", the login hero's engine claim - so the split needs a * definition somewhere, and it belongs beside `SHIPPED` rather than in the UI that * prints it. That is the same reason `SHIPPED` itself lives here. * @@ -543,7 +543,7 @@ export function compatibleEnginesFor(type: DatabaseType): readonly WireCompatibl * app at it, so the embedded store is out of both halves of the sum. * * Still no runtime consumer: README.md and the docs table are markdown and quote the - * number as prose, and the login hero prints the two halves separately - fourteen in + * number as prose, and the login hero prints the two halves separately - fifteen in * the proof row, twenty-six in the relatives line - rather than their sum. This exists * so the arithmetic has one definition, and the unit test pins it. */ From 1086d6030ca1abdf4a592f1e86bae3f554878fa6 Mon Sep 17 00:00:00 2001 From: cevheri Date: Thu, 27 Aug 2026 13:12:50 +0300 Subject: [PATCH 4/5] chore(operator): regenerate the bundle so the CSV description carries operator-sdk's own wrapping `Helm Chart Lint` failed on the bundle freshness step, and the drift was purely mechanical: I hand-edited the CSV `description` in both `operator/config/.../bases` and `operator/bundle/manifests` to name libSQL, and my line breaks are not the ones `operator-sdk generate bundle` produces. The gate re-runs `make -C operator bundle` and diffs, so any hand-wrapped YAML folded scalar fails it even when the text is identical. `make -C operator bundle` run locally; the only content change is the wrapping, plus the `createdAt` stamp the gate ignores with `-I '^ *createdAt:'`. `operator-sdk bundle validate --select-optional suite=operatorframework` passes. Not a chart version question: `chart:check` is green (chart 0.1.53 / appVersion 0.13.4 / package.json 0.13.4), #167 concerns packaged files under `charts/`, and nothing under `charts/` changed here. Lesson for the next engine: the CSV description is generated, so edit `operator/config/manifests/bases/...` and re-run `make -C operator bundle` rather than editing the two copies by hand. --- .../libredb-studio-operator.clusterserviceversion.yaml | 10 +++++----- .../libredb-studio-operator.clusterserviceversion.yaml | 8 ++++---- 2 files changed, 9 insertions(+), 9 deletions(-) diff --git a/operator/bundle/manifests/libredb-studio-operator.clusterserviceversion.yaml b/operator/bundle/manifests/libredb-studio-operator.clusterserviceversion.yaml index 732b3503..5e57bc70 100644 --- a/operator/bundle/manifests/libredb-studio-operator.clusterserviceversion.yaml +++ b/operator/bundle/manifests/libredb-studio-operator.clusterserviceversion.yaml @@ -22,11 +22,11 @@ metadata: capabilities: Basic Install categories: Database, Developer Tools containerImage: ghcr.io/libredb/libredb-studio-operator:0.13.4 - createdAt: "2026-08-24T19:49:18Z" - description: Open-source web-based SQL IDE for fifteen engines - PostgreSQL, - MySQL, Oracle, SQL Server, SQLite, MongoDB, Redis, Couchbase, ClickHouse, Apache - Druid, Elasticsearch, OpenSearch, Apache Trino, Apache Cassandra and libSQL - - with AI-powered query assistance. + createdAt: "2026-08-27T10:12:23Z" + description: Open-source web-based SQL IDE for fifteen engines - PostgreSQL, MySQL, + Oracle, SQL Server, SQLite, MongoDB, Redis, Couchbase, ClickHouse, Apache Druid, + Elasticsearch, OpenSearch, Apache Trino, Apache Cassandra and libSQL - with + AI-powered query assistance. operators.operatorframework.io/builder: operator-sdk-v1.42.3 operators.operatorframework.io/project_layout: helm.sdk.operatorframework.io/v1 repository: https://github.com/libredb/libredb-studio diff --git a/operator/config/manifests/bases/libredb-studio-operator.clusterserviceversion.yaml b/operator/config/manifests/bases/libredb-studio-operator.clusterserviceversion.yaml index 0c4a907f..9d7b0f61 100644 --- a/operator/config/manifests/bases/libredb-studio-operator.clusterserviceversion.yaml +++ b/operator/config/manifests/bases/libredb-studio-operator.clusterserviceversion.yaml @@ -6,10 +6,10 @@ metadata: capabilities: Basic Install categories: Database, Developer Tools containerImage: ghcr.io/libredb/libredb-studio-operator:0.0.0 - description: Open-source web-based SQL IDE for fifteen engines - PostgreSQL, - MySQL, Oracle, SQL Server, SQLite, MongoDB, Redis, Couchbase, ClickHouse, Apache - Druid, Elasticsearch, OpenSearch, Apache Trino, Apache Cassandra and libSQL - - with AI-powered query assistance. + description: Open-source web-based SQL IDE for fifteen engines - PostgreSQL, MySQL, + Oracle, SQL Server, SQLite, MongoDB, Redis, Couchbase, ClickHouse, Apache Druid, + Elasticsearch, OpenSearch, Apache Trino, Apache Cassandra and libSQL - with + AI-powered query assistance. repository: https://github.com/libredb/libredb-studio support: LibreDB labels: From d9ad26ffaf5032d1798269bd9a711bd7059874d9 Mon Sep 17 00:00:00 2001 From: cevheri Date: Thu, 27 Aug 2026 13:15:14 +0300 Subject: [PATCH 5/5] docs(provider): the checklist never named the three storefronts, which is why they were stale The #511 review found the chart description, the chart keywords and the OLM CSV description all still naming the previous engine set after libSQL had landed everywhere the compiler looks. That was not an oversight in one PR - `docs/ADDING_A_PROVIDER.md` does not mention any of them. Its "Always" and "Conditionally" blocks are the surfaces a gate or the type system catches; this block is the one with neither, so it goes in the same list rather than in a commit message nobody re-reads. Added as a third block, and each line carries the reason it bites: - ArtifactHub shows `description` and SEARCHES `keywords`, so an engine missing from the keywords is an engine nobody finds - and #167 makes a keyword-only fix cost a chart version of its own, which is why the chart's own comment asks for it in the release that ships the engine. Two names are usually right, the type-id and the product a user types (`libsql` and `turso`). - The CSV description must be edited in `operator/config/manifests/bases/` and regenerated with `make -C operator bundle`. Editing `operator/bundle/manifests/` by hand is what failed `Helm Chart Lint` on this PR: the gate re-runs the generator and diffs, so hand-wrapped YAML fails it even when the text is byte-identical in meaning. - Numerals need their denominator separated first (type-ids, external drivers, relatives, the sum), with `connectableProductCount()` as the one definition. A mechanical replace is how a CORRECT number becomes wrong - the agent docs' "the other fourteen" counts type-ids minus the two `CATALOG_PLANS` dialects and moves for a different reason than the driver count does. Verified while writing it: `tests/unit/marketplace-copy.test.ts` exists and passes (13/13), and the three listing files named are the ones this PR had to touch. Gates: format, lint, typecheck, knip. --- docs/ADDING_A_PROVIDER.md | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) diff --git a/docs/ADDING_A_PROVIDER.md b/docs/ADDING_A_PROVIDER.md index 692ebcf8..113289c6 100644 --- a/docs/ADDING_A_PROVIDER.md +++ b/docs/ADDING_A_PROVIDER.md @@ -804,6 +804,34 @@ The integration points, all of which need an entry. This is the list the Strateg guess. `tests/unit/sql/grammar.test.ts` holds `Record` maps for both decisions, so the compiler will at least stop you from *forgetting* that a decision exists +**Published where a human reads it, and this is the block with the fewest gates.** `readme:check` +compares the translated READMEs against `README.md` and `chart:check` compares versions; nothing +counts the engines in a catalog listing, so an engine can ship while three storefronts still name the +previous set (measured in the #511 review, which found all of these stale after libSQL had already +landed everywhere the compiler looks): + +- [ ] `charts/libredb-studio/Chart.yaml` — the `description`, which is what **ArtifactHub** shows, AND + the `keywords` list, which is what ArtifactHub **searches**. An engine absent from the keywords is + an engine nobody finds; the chart's own comment says a new engine's keyword belongs in the release + that ships it, because #167 otherwise makes a keyword-only fix cost a chart version of its own. + Two names are often right — the type-id and the product a user would type (`libsql` and `turso`) +- [ ] `operator/helm-charts/libredb-studio/Chart.yaml` — the operator's embedded copy, same edit +- [ ] `operator/config/manifests/bases/libredb-studio-operator.clusterserviceversion.yaml` — the CSV + `description`, which is what **OperatorHub** shows. **Edit only this file and then run + `make -C operator bundle`**: `operator/bundle/manifests/...` is generated from it, and the + `Verify operator bundle is up to date` step re-runs the generator and diffs, so a hand-wrapped + YAML folded scalar fails the gate even when the text is identical to what it wants +- [ ] `README.md` + `README_zh.md` + `README_ja.md`, `DOCKERHUB.md`, `docs/BRAND_MESSAGING.md` — the + engine tables and every prose numeral. **Separate the denominators before touching a numeral**: + type-ids the factory builds, external drivers (that set minus the embedded store), wire-compatible + relatives, and their sum. `connectableProductCount()` is the arithmetic's one definition — derive + from it, and re-read each sentence to see which of the four it counts. A mechanical replace is + how a correct number becomes wrong: the agent docs' "the other fourteen" counts type-ids minus + the two `CATALOG_PLANS` dialects and moved for a different reason than the driver count did +- [ ] the marketplace listings under `deploy/` — a claim that enumerates engines is bound to the file + that proves it, and the `marketplace-copy` test fails when a plan-capable engine is missing from + one + **And the tests for every exhaustive map**, which are the real checklist — several are exhaustive *by construction* (`Record` in `db-ui-config`, `PICKER_COVERAGE` in the connection-form test), so the compiler and those tests refuse to pass until each is updated: