[hitesh96db]

What

A deleted user, group, role or account now leaves nothing behind for one created later with the same name or ID, on all four backends.

Inline policies and permissions boundaries are stored by principal name, with no foreign key to the principal. On MongoDB and Cassandra, no IAM record is linked to its principal or account at all. Before this change, each delete path removed a different subset (read from main at 35edb778):

Delete	SQLite / PostgreSQL left	MongoDB left	Cassandra left
user	inline policies, boundary	boundary, tags	boundary
group	inline policies	inline policies, members	inline policies, members
role	inline policies, boundary	inline policies, boundary, tags, sessions	inline policies, boundary, tags, sessions
account	nothing (foreign keys)	every IAM record of the account	every IAM record of the account

Changes:

• Deletes remove everything the principal owns. SQLite and PostgreSQL delete a principal's inline policies and boundary in the same transaction as its row. MongoDB and Cassandra have no transaction across tables, so they delete what a principal owns first, credentials first, and its own record last. A failure then leaves the principal with less than it had, never a credential or permission without it, and a retry finishes. MongoDB's delete_user now returns errors from these deletes; before, it ignored them.
• Account deletes on MongoDB and Cassandra remove the account's users, groups and roles, and everything they own, before the account record. The "refuse while tables exist" check stays.
• Creates start clean. After its own insert succeeds, a user, group or role create deletes anything stored under its name. A create that is refused as a duplicate touches nothing. On MongoDB and Cassandra, an account create does the same for its ID. Records that an earlier release left behind are cleared when the name or ID is used again, without a catalog migration. If the cleanup fails, MongoDB and Cassandra remove the new record again, so a retry is not refused as a duplicate.
• MongoDB refuses a taken account ID. Its accounts collection is unique on account_name only, so create_account now checks the ID first. Before, a second account with the same ID could be created, and the cleanup on create must never reach a live account.
• A deleted account grants nothing (MongoDB and Cassandra). The access key and session lookups, the console password hash read, the management-API password check and the AssumeRole trust-policy read treat a principal whose account no longer exists as unknown. This covers records that an earlier release's account delete left behind. On Cassandra the account delete now commits at quorum, which the console's quorum reads need to see it.
• A policy or boundary on a missing principal is refused. The management API and the console answer HTTP 404 IAM <user|group|role> not found instead of 204, as IAM answers NoSuchEntity. The check is in the server: the storage methods stay plain data access, and the cleanup on create covers a put that races a delete.
• SQLite: no transaction left open on a pooled connection. Dropping a SQLite transaction only queues its rollback. A BEGIN IMMEDIATE on that connection before the rollback ran failed with attempted to call begin_with at non-zero transaction depth. That already happened on create_user's error paths; the new NotFound and AlreadyExists returns made it frequent. The creates and deletes now commit or roll back explicitly before they return.

I kept cascade rather than IAM's DeleteConflict refusal: ExtendDB already cascades access keys, tags and memberships, and refusing would break existing callers. docs/differences-from-dynamodb.md records the difference.

Why

Closes #693.
Closes #694.

A principal created with a deleted principal's name got its permissions: an inline policy or boundary on every backend, and on MongoDB and Cassandra also its access keys, sessions or group members. On MongoDB and Cassandra, a deleted account's users stayed in the catalog with their password hashes.

On SQLite, on main (35edb778), I reproduced the role, user and group cases with extenddb manage. A recreated role listed the old policy and boundary, and a new session of it was allowed ListTables. A recreated user and a recreated group behaved the same way.

Testing done

• New shared check extenddb_storage::iam_leftovers_conformance (feature test-util). Each backend's suite runs it against a real database. It checks that a delete leaves no policy or boundary, that a create over rows stored under its name starts empty, that a refused duplicate create keeps the live principal's policies and boundary, and that an account created again with the same ID has no principals or policies.
  • SQLite (in-memory) and PostgreSQL 17 (tests/iam_leftovers.rs): pass.
  • Negative controls: the check fails with main's users.rs, groups.rs and roles.rs on both backends. With only the cleanup in SQLite's create_user removed, it fails at "created over rows left under its name".
  • MongoDB (management_store::live_tests, added to devtools/run-mongodb-tests --rust-live) and Cassandra (tests/direct/iam_leftovers.rs): not run locally, because this machine has neither. Both also test that a user and role of a deleted account (only the account record removed) cannot log in or be assumed. The credential stores' new checks have no unit test: MongoDB's hard-codes the extenddb_catalog database, and Cassandra's is private to the crate.
• New wire test tests/test_iam_delete_leftovers.py (9 tests, all backends). For a role, user, group and account, it gives each everything it can own, deletes it, checks that its key or session is refused, creates it again, and checks that it starts empty and denied. It also checks that the account's user can no longer log in through the management API or the console. Two tests check the 404 on a missing principal.
  • On main's SQLite build, 8 of the 9 fail. The account test passes there, as expected: the foreign keys already cascade an account delete on SQLite. Its failure would show on MongoDB and Cassandra, which I could not run.
  • On this branch, the 27 existing pytest files that create, delete or attach to principals, plus the new file, pass on SQLite release builds (668 passed, 8 skipped) and PostgreSQL release builds (666 passed, 10 skipped), on the head with main merged.
• cargo +1.97.0 fmt --all -- --check: clean.
• cargo +1.97.0 clippy --all-targets -- -D warnings on the six changed crates, with both feature sets of clippy.yml: clean. The workspace-wide run stops at an unused variable in crates/core/tests/data_file_decode_peak.rs, which this branch does not touch.
• cargo +1.88.0 check on the changed crates: OK.
• cargo test --lib for storage, storage-sqlite, storage-mongodb, storage-cassandra and server: pass.
• cargo test -p extenddb-storage-postgres --tests against PostgreSQL 17: 395 passed. 4 failed, and they fail the same way on main here: the 3 migrate_data_db_name tests need a password in the connection string, and a_role_that_does_not_own_the_tables_is_told_what_to_run depends on the local superuser setup.

Checklist

• I have read CONTRIBUTING.md
• All tests pass (cargo test --workspace)
• Code is formatted (cargo fmt --check)
• Clippy is clean (cargo clippy -- -W clippy::pedantic)
• I have added or updated tests for new functionality
• I have updated documentation if behavior changed
• Breaking changes are noted below (if any)
• If this changes the wire protocol, Storage trait, auth model, on-disk
  format, or public CLI surface, an RFC has been accepted or is linked
  below. Otherwise, an ADR captures the decision (link below).

ADR / RFC: n/a. No schema, trait or wire change. The management API now answers 404 where it answered 204 for a policy or boundary on a missing principal.

Breaking changes

• PUT .../users|groups|roles/{name}/policy/{policy} and PUT .../users|roles/{name}/permissions-boundary on a principal that does not exist answer 404 IAM <type> not found, where they answered 204. A script that attaches before it creates must create first.
• Deleting a principal removes its inline policies and boundary (and, on MongoDB and Cassandra, everything else it owns). A deployment that relied on them coming back when a principal with the same name was created again must attach them again.
• On MongoDB, create-account with an account ID that is already in use answers Account already exists.

The upgrade manual has a v0.1.15 entry. No catalog migration is required.

---

By submitting this pull request, I confirm that my contribution is made under
the terms of the Apache License 2.0 and I agree to the Developer Certificate of
Origin (DCO). See CONTRIBUTING.md for details.

[navareamol]

No blocking findings. Two things to settle:

1. CI is red and the branch is behind main. run-tests fails on panicking_refresh_does_not_leak_inflight_flag in crates/cache, which this PR does not touch. It passed 100 of 100 runs on main here, so I think it is a flake. Could you update the branch and rerun it? CodeQL also reports 4 "hard-coded cryptographic value" alerts on the test passwords in iam_leftovers.rs and live_tests.rs. Could you generate the password in the test, or dismiss the alerts with a comment?
2. The description says a deleted principal leaves nothing behind, but the failed-login history in login_attempts is keyed by name and follows it. Measured on SQLite and Cassandra: 6 wrong passwords for a user, delete the user, create the same name again, then the correct password from the same address gets 429 for up to 15 minutes. It is the same on main, so this PR does not cause it. Could you clear that name's login_attempts rows in the delete and the create, or narrow the claim in the description? Filing it as its own issue also works for me.

The wire tests pass on SQLite, PostgreSQL and Cassandra. On main, 8 of the 9 fail on SQLite, as the description says. The two Cassandra direct tests pass too.

I'll approve once 1 is green and 2 is settled.

[navareamol]

If the clear fails and the delete of the new row fails too, the second error is only logged and the create returns the first. The principal then exists with the old rows still under it. A retry of the create gets 409 and skips the clear, so nothing removes them. Is anything else clearing them in that case? I read this from the code and did not inject the failure.