Inflight ledger is a bare per-segment counter — not safe under async/batched/retried admit·complete

[susheem-k]

Problem

ledger_inflight is a single integer per segment_key:

• ledger_admit(segment_key) → count = count + 1
• ledger_complete(segment_key) → count = MAX(0, count - 1)
• ledger_inflight(segment_key) → count

(control_plane/store.py ledger_admit / ledger_complete; mirrored in tokenops HttpStore.)

It is read by concurrency_cap at pre_call to decide admit vs reject/queue.

This works today only because every admit/complete is synchronous and serialized
(one governed call at a time per process, SqliteStore serializes on an RLock). It is
not safe once admit/complete are batched, buffered, retried, or a process crashes —
which is exactly the direction of the remote-only / events:batch work.

Failure examples

A. Lost complete on crash → permanent phantom in-flight

1. Agent process admits call X → count = 1.
2. LLM call returns; process crashes before complete flushes.
3. count stays 1 forever. concurrency_cap(max_concurrent=1) now rejects every
   future call on that segment, for every process, until someone manually clears it.
   MAX(0, count-1) only guards the low side; nothing caps or repairs the high side.

B. Retried batch → double count (no idempotency)

1. Client buffers [admit(seg)], flush POST times out, client retries.
2. Plane applied it twice → count = 2 for one real call.
3. concurrency_cap rejects a call that should have been admitted.
   A retried complete is the mirror: count goes too low, masking a real in-flight call,
   so concurrency_cap admits past the limit.

C. Reordering across a buffer

1. Call X complete is buffered; call Y admit flushes first.
2. Between the two flushes, a third process reads inflight and sees an inflated count.
   Bounded by flush interval, but the value is transiently wrong in the unsafe direction
   for a reject decision.

D. Idempotency key on a bare counter is ambiguous
To dedupe, the plane must remember, per idempotency_key, whether that key was a +1
or a -1, and refuse to reapply — extra state that a counter does not naturally carry.

Sketch of a fix (for later — NOT this plan)

Model inflight as a set of active call-ids:

• admit(segment_key, call_id) → INSERT OR IGNORE
• complete(segment_key, call_id) → DELETE
• inflight(segment_key) → SELECT COUNT(*)
• TTL sweep for call-ids older than N minutes (crash cleanup).

Naturally idempotent, crash-tolerant, and safe to read slightly stale. Alternative:
keep the counter but make complete and reject-mode admit hard flush barriers that
never ride a buffer.

Scope

Explicitly out of scope for the remote-only control-plane plan
(scratch/remote-only-control-plane-plan.md). That plan assumes synchronous
admit/complete for now and only leaves room for batching. This issue tracks making
the inflight model actually safe before that room is used.

Related: #55, #60, plan Part 4.