docs: state the 1.0 scope, the backend tiers and the versioning policy

README.md

@@ -16,6 +16,23 @@ with DynamoDB works with ExtendDB, unchanged.
 - **Multi-cloud** — use DynamoDB semantics on any cloud that runs an ExtendDB-supported database backend
 - **Air-gapped environments** — DynamoDB functionality with no internet connectivity
 
+## Scope
+
+ExtendDB is a server that implements the Amazon DynamoDB API on a database that you provide: PostgreSQL, SQLite, MongoDB or Cassandra. An application connects to it through the endpoint setting of an AWS SDK or the AWS CLI. The application signs each request with an access key stored in ExtendDB.
+
+ExtendDB 1.0 is for:
+
+- Local development and continuous integration (CI) test runs.
+- Self-hosted deployments: on premises, in a private or public cloud, at the edge, or on a network with no internet access.
+
+ExtendDB 1.0 is not:
+
+- Amazon DynamoDB: ExtendDB is a separate implementation of the API. [Differences from DynamoDB](docs/differences-from-dynamodb.md) lists the known differences and the operations that ExtendDB does not implement, for example global tables.
+- A managed service: you provide the database, and the database's own tools handle replication, failover and database backups.
+- A multi-server deployment: run one ExtendDB server for each catalog. In 1.0, more than one server on one catalog is not supported.
+
+[Storage backends](#storage-backends) says what 1.0 promises for each backend. The [versioning policy](docs/versioning.md) says what a release can change.
+
 ## Features
 
 - Full DynamoDB wire protocol: CRUD, Query, Scan, Batch, Transactions, Streams, TTL, Import/Export
@@ -27,6 +44,25 @@ with DynamoDB works with ExtendDB, unchanged.
 - JSON metrics endpoint with DynamoDB CloudWatch-style metric names and dimensions
 - Daemon mode with syslog logging, plus `--foreground` for container and supervisor environments
 
+## Storage backends
+
+An ExtendDB binary contains one storage backend, which the build selects. The tier of a backend says what the 1.x releases promise for it:
+
+- Stable: for production. The [versioning policy](docs/versioning.md) applies in full. Required CI checks test the backend, and a pull request cannot merge until they pass.
+- Beta: for production within the known limits in the table. The versioning policy applies with one exception. To remove a known limit, a minor release can change the behavior that the table lists, and its release notes name the change.
+- Experimental: for evaluation. The backend is not for production use. The versioning policy does not apply. A release can change the backend's behavior or configuration with no deprecation period.
+
+| Backend | Tier | Topology | Data across upgrades | API coverage | CI | Known limits |
+|---|---|---|---|---|---|---|
+| PostgreSQL | Stable | PostgreSQL 14 or later, on the same host, on another host or as a managed service. | `extenddb migrate --yes` migrates the catalog in place. CI upgrades catalogs that 0.1.x releases created. | The operations in [Supported Operations](#supported-operations). Vector search needs the pgvector extension. | `integration`, required, on PostgreSQL 16 | Time to Live (TTL) removes about 100 expired items per table per minute. |
+| SQLite | Stable (single node) | SQLite runs in the server process. A lock file allows one server for each database file. There is no production image, so build from source. | As PostgreSQL. `migrate --yes` copies the database file before its first change. | The operations in Supported Operations. | `integration`, required | One write transaction at a time. TTL as PostgreSQL. |
+| MongoDB | Beta | MongoDB 7.0 or later, as a replica set, also for one node. | As PostgreSQL. | No vector search. On a global secondary index (GSI) with more than one RANGE attribute, `Query` takes a key condition on the first one only. | `mongodb-integration`, required, on MongoDB 7 | A number in a key attribute has at most 34 significant digits. The database names are fixed, so one replica set holds one catalog. TTL as PostgreSQL. |
+| Cassandra | Experimental | Cassandra 4.1 or later, with `PasswordAuthenticator`. Each table takes its writes in one datacenter. | `migrate --yes` migrates the catalog. No rollback procedure is tested. | No vector search. CreateBackup keeps each backup in the catalog, with or without a backup store. The `Query` limit of MongoDB on a GSI with more than one RANGE attribute. | `cassandra-integration`, required, on Cassandra 4.1 | Some management operations are not atomic. A few known race conditions are open. TTL removes about 1,000 expired items per table per minute. See [Known Limitations](docs/cassandra-setup.md#known-limitations). |
+
+On every backend, run one ExtendDB server for each catalog. Only SQLite enforces this. [Differences from DynamoDB](docs/differences-from-dynamodb.md) lists the differences of each backend from the service.
+
+The `@extenddb/dev` npm package and the `extenddb/extenddb-dev` image run a dev-mode SQLite build, for local development and CI. Dev mode uses plain HTTP and seeds an example key that can do everything, so keep a dev-mode server on loopback. At startup it migrates a catalog that is behind it, after it copies the database file.
+
 ## Quick Start
 
 ```bash
@@ -260,6 +296,7 @@ docs/
 | [Security Model](docs/manuals/10-security-model.md) | Threat model, authentication, authorization |
 | [Deployment Guide](docs/manuals/11-deployment-guide.md) | Self-hosted, multi-cloud, air-gapped deployments |
 | [Differences from DynamoDB](docs/differences-from-dynamodb.md) | Behavioral differences and unsupported operations |
+| [Versioning and Support Policy](docs/versioning.md) | What each release can change, and how long a release gets fixes |
 | [Troubleshooting](docs/troubleshooting.md) | Common errors and solutions |
 | [Verifying Container Images](docs/verifying-images.md) | Build provenance and SBOM attestations for the published images |
 

crates/app/Cargo.toml

@@ -9,8 +9,8 @@ license.workspace = true
 
 [features]
 # Developer mode: plain HTTP on loopback, open authorization (SigV4 still
-# enforced), seeded dev credential. A build-time profile for local/CI only;
-# the thin bin enforces that it is paired with a dev/CI backend.
+# enforced), seeded dev credential. A build-time profile for local development
+# and CI. The thin bin enforces that it is paired with the `sqlite` backend.
 dev-mode = ["extenddb-config/dev-mode"]
 mongodb-test-hooks = ["extenddb-server/mongodb-test-hooks"]
 backup-test-hooks = ["extenddb-server/backup-test-hooks"]

crates/bin/Cargo.toml

@@ -26,9 +26,10 @@ mongodb-test-hooks = [
     "extenddb-app/mongodb-test-hooks",
 ]
 # Developer mode: plain HTTP on loopback, open authorization (SigV4 still
-# enforced), seeded dev credential. A build-time profile for local/CI only; it
-# requires a dev backend (`sqlite`/`sqlite-memory`) and fails to build with a
-# production backend like `postgres` (enforced at compile time in `main.rs`).
+# enforced), seeded dev credential. A build-time profile for local development
+# and CI. It compiles only with `sqlite` or `sqlite-memory`. A build of any other
+# backend with it, `postgres` among them, fails (`main.rs` checks this at compile
+# time). A production `sqlite` build leaves it off.
 dev-mode = ["extenddb-app/dev-mode"]
 
 [dependencies]

crates/bin/src/main.rs

@@ -11,10 +11,11 @@
 //!
 //! In-tree backends are selected by mutually exclusive Cargo features:
 //! `postgres` (the default production backend), `mongodb` (production, built with
-//! `--no-default-features --features mongodb`), `cassandra`, and `sqlite`/`sqlite-memory`
-//! (the dev/CI backend). Exactly one must be enabled: [`set_backend`] installs one
-//! backend per process, so a build with more than one would be ambiguous and is
-//! rejected at compile time.
+//! `--no-default-features --features mongodb`), `cassandra`, and `sqlite`/`sqlite-memory`.
+//! `sqlite` is for production on a single node, and `sqlite-memory` keeps the database
+//! in memory for development and CI. Exactly one must be enabled. [`set_backend`]
+//! installs one backend per process, so a build with more than one would be ambiguous
+//! and is rejected at compile time.
 
 // Exactly one backend feature must be enabled.
 #[cfg(any(
@@ -42,18 +43,17 @@ compile_error!(
 );
 
 // Developer mode relaxes the security posture (plain HTTP on loopback, open
-// authorization). It is a dev/CI-only profile and must be built only with a
-// dev/CI-suitable backend. Rather than denying each production backend by name
-// (every backend is a production backend unless proven otherwise, so a deny-list
-// would have to grow with each new one), require a known dev backend: dev-mode
-// compiles only when `sqlite` is enabled. `sqlite-memory` enables `sqlite`, so it
-// is covered too; postgres, mongodb — or any future production backend — fail the
-// build, so there is no path by which a production deployment can serve in dev mode.
+// authorization) for development and CI. It compiles only when the build enables
+// `sqlite`. `sqlite-memory` enables `sqlite`, so dev mode also compiles with
+// `sqlite-memory`. The check names the one backend that dev mode allows, because a
+// deny-list of the other backends would have to grow with each new backend. A build
+// of any other backend with `dev-mode` fails. A production `sqlite` build leaves
+// `dev-mode` off.
 #[cfg(all(feature = "dev-mode", not(feature = "sqlite")))]
 compile_error!(
-    "the `dev-mode` feature requires a dev/CI backend such as `sqlite`; it must \
-     not be built with a production backend like `postgres` or `mongodb` (build \
-     with `--no-default-features --features sqlite-memory,dev-mode`)"
+    "the `dev-mode` feature requires the `sqlite` backend and does not build with \
+     `postgres`, `mongodb` or `cassandra` (build with \
+     `--no-default-features --features sqlite-memory,dev-mode`)"
 );
 
 fn main() -> anyhow::Result<()> {

docs/manuals/07-upgrade-manual.md

@@ -8,7 +8,7 @@ This release versions the PostgreSQL, SQLite, Cassandra and MongoDB catalogs by
 
 Migration 9, `warm_throughput`, adds a nullable `warm_throughput` column to the `tables` and `indexes` catalog tables on PostgreSQL, SQLite and Cassandra. It holds the `WarmThroughput` a client sets on a table or a global secondary index, which `DescribeTable` reports; nothing acts on the value (see [Differences from DynamoDB](../differences-from-dynamodb.md)). Tables created before the migration report the defaults for their billing mode. On MongoDB it changes nothing, since the catalog collections have no validator, and the ledger records it as not applicable. Migration 9 is breaking: to go back to a release without it, restore the backup taken before the upgrade (see [Rollback Procedure](#rollback-procedure)).
 
-Configure `[backup.store]` before the release that stops creating in-catalog backups. From this release CreateBackup writes to the configured store on PostgreSQL, SQLite and MongoDB, and is asynchronous there as on the service: the response is `CREATING` and the backup becomes `AVAILABLE` once the upload ends, so a client that deletes a backup it just created has to poll DescribeBackup first (the previous release returned `AVAILABLE`). A store-backed backup is listed, described, deleted and restored like any other. `RestoreTableFromBackup` of a backup in the configured store runs the restore job from the store on every backend (PostgreSQL, SQLite, MongoDB and Cassandra); in-catalog backups, those of the previous release included, restore as before. An S3 store needs `spool_path` set (the restore's download area) on PostgreSQL, MongoDB and Cassandra, and that must be a directory of its own, not the database directory; a filesystem store needs no spool. Migration 8 adds the `restore_progress` table to the PostgreSQL data database; run `extenddb migrate --yes` before starting the new binary. The progress rows of a restore in flight live in the data database, so a data database restored from a dump taken before migration 8 has none of them and no `restore_progress` table: the new binary refuses to serve it until `extenddb migrate --yes` has run again, and its reconciler then finds no progress for the restore and resumes it from its first data file into the (empty) target, or `FAILED`s it as `target_not_empty` if the target already holds items, in which case `extenddb backup remove-target --account <account> <table>` (the admin guide's `backup` section) removes the target and its record and the backup restores again. The same `target_not_empty` state is what a server stopped inside a restore's first data file on MongoDB, or inside its first batch of 10,000 items or 32 MiB on Cassandra, leaves on restart, since both record progress after the write; the command is the remedy there too. `backup remove-target` writes the catalog, so on SQLite the server is stopped for it, as for `purge`. The previous binary resumes no restore (a store-backed restore in flight at a rollback is lost with the catalog rows, as the rollback paragraph below says). Quiesce restores before a downgrade. `kind = "disabled"` is no longer needed to keep a deployment able to restore the backups it takes. Without a store CreateBackup writes the in-catalog format and logs a deprecation warning naming `[backup.store]` on each call; a later release's CreateBackup refuses without one. Cassandra writes the in-catalog format with or without a store. `kind = "disabled"` under `[backup.store]` is the opt-out that keeps a deployment on the in-catalog format on purpose and silences the warning. On file-backed SQLite an absent section is a filesystem store at `backups/` beside the database file, created `0700` at startup; that root must be owned by the extenddb user with mode `0700`, under directories owned by root or that user which others cannot write to. A default root that breaks the rule (a database file in a group-writable directory, say), or whose `extenddb-store.json` names another catalog (a catalog re-created by `destroy` and `init` under the old root), is logged at startup as a warning (`backup store: the default root ... failed its check`) and leaves the store unconfigured, not unhealthy, so the upgrade never stops a server that never configured backups and CreateBackup keeps working on it; the same root configured explicitly is a startup error or an unhealthy store. The warning applies only while the catalog has no backup rows for the default root (an upgrade, a fresh deployment, a catalog re-created by `destroy` and `init`): once a server has written rows for it, the default root is handled as a configured one, so its backups never vanish behind a warning (a failed identity check is an unhealthy store, reported on `/health`; a root that can no longer be opened is a startup error naming it). Instances that share a catalog must share one store root, an S3 bucket and prefix or a shared mount: a second instance pointed at a host-local directory under the same configured path starts with the store unhealthy (`/health` reports `backup_store` unhealthy; the admin guide's `[backup.store]` section says what the identity file and the manifest probe check). `extenddb destroy` leaves the store; `extenddb backup purge --confirm` is what deletes its backups, and a catalog re-created with `init` needs the old root's `extenddb-store.json` removed or a new root, since the file names the old catalog.
+Configure `[backup.store]` before the release that stops creating in-catalog backups. From this release CreateBackup writes to the configured store on PostgreSQL, SQLite and MongoDB, and is asynchronous there as on the service: the response is `CREATING` and the backup becomes `AVAILABLE` once the upload ends, so a client that deletes a backup it just created has to poll DescribeBackup first (the previous release returned `AVAILABLE`). A store-backed backup is listed, described, deleted and restored like any other. `RestoreTableFromBackup` of a backup in the configured store runs the restore job from the store on every backend (PostgreSQL, SQLite, MongoDB and Cassandra); in-catalog backups, those of the previous release included, restore as before. An S3 store needs `spool_path` set (the restore's download area) on PostgreSQL, MongoDB and Cassandra, and that must be a directory of its own, not the database directory; a filesystem store needs no spool. Migration 8 adds the `restore_progress` table to the PostgreSQL data database; run `extenddb migrate --yes` before starting the new binary. The progress rows of a restore in flight live in the data database, so a data database restored from a dump taken before migration 8 has none of them and no `restore_progress` table: the new binary refuses to serve it until `extenddb migrate --yes` has run again, and its reconciler then finds no progress for the restore and resumes it from its first data file into the (empty) target, or `FAILED`s it as `target_not_empty` if the target already holds items, in which case `extenddb backup remove-target --account <account> <table>` (the admin guide's `backup` section) removes the target and its record and the backup restores again. The same `target_not_empty` state is what a server stopped inside a restore's first data file on MongoDB, or inside its first batch of 10,000 items or 32 MiB on Cassandra, leaves on restart, since both record progress after the write; the command is the remedy there too. `backup remove-target` writes the catalog, so on SQLite the server is stopped for it, as for `purge`. The previous binary resumes no restore (a store-backed restore in flight at a rollback is lost with the catalog rows, as the rollback paragraph below says). Quiesce restores before a downgrade. `kind = "disabled"` is no longer needed to keep a deployment able to restore the backups it takes. Without a store CreateBackup writes the in-catalog format and logs a deprecation warning naming `[backup.store]` on each call. A later major release's CreateBackup refuses without one. Cassandra writes the in-catalog format with or without a store. `kind = "disabled"` under `[backup.store]` is the opt-out that keeps a deployment on the in-catalog format on purpose and silences the warning. On file-backed SQLite an absent section is a filesystem store at `backups/` beside the database file, created `0700` at startup; that root must be owned by the extenddb user with mode `0700`, under directories owned by root or that user which others cannot write to. A default root that breaks the rule (a database file in a group-writable directory, say), or whose `extenddb-store.json` names another catalog (a catalog re-created by `destroy` and `init` under the old root), is logged at startup as a warning (`backup store: the default root ... failed its check`) and leaves the store unconfigured, not unhealthy, so the upgrade never stops a server that never configured backups and CreateBackup keeps working on it; the same root configured explicitly is a startup error or an unhealthy store. The warning applies only while the catalog has no backup rows for the default root (an upgrade, a fresh deployment, a catalog re-created by `destroy` and `init`): once a server has written rows for it, the default root is handled as a configured one, so its backups never vanish behind a warning (a failed identity check is an unhealthy store, reported on `/health`; a root that can no longer be opened is a startup error naming it). Instances that share a catalog must share one store root, an S3 bucket and prefix or a shared mount: a second instance pointed at a host-local directory under the same configured path starts with the store unhealthy (`/health` reports `backup_store` unhealthy; the admin guide's `[backup.store]` section says what the identity file and the manifest probe check). `extenddb destroy` leaves the store; `extenddb backup purge --confirm` is what deletes its backups, and a catalog re-created with `init` needs the old root's `extenddb-store.json` removed or a new root, since the file names the old catalog.
 
 A backup that a 0.1.x release took can restore without its global and local secondary indexes: a SQLite backup taken by 0.1.14 records only the table's key schema and attribute definitions, and its restore creates the table with its items and no index. Take a new backup of each table after upgrading to v1.0, and keep the 0.1.x backups for their items only.