Repository navigation
[P1B][T11] 实现 Redis 无 MQ 轮询领取、过期恢复与唤醒索引修复 #8
Description
目标
Worker 从权威 run 状态领取任务，确保索引发布前崩溃也不会永久遗漏已接受运行。
优先级与归属
- 优先级：P1B；任务标识：T11。
- 责任仓库：
agentic-ai-java/argi-extensions。 - 总览：https://github.com/agentic-ai-java/argi/issues/117。
- 这是单一交付任务；以中立接口/reference tests 隔离依赖，可单独提交评审。
范围与落地方案
- 实现 runnable/due-retry 索引、容量感知 claim、Redis 时间 heartbeat/过期扫描。
- 同 slot durable outbox 发布到 global advisory index；提供可发现、可限流的修复扫描，索引不是状态真相。
- 重复/乱序候选按当前 revision/status 丢弃，terminal run 不重跑，unsafe recovery 转核对流程。
前置依赖
间接或集成依赖（不单独阻塞）
- [P1B][T08] 实现 Redis 持久化运行记录与原子提交去重 #6 (T08)：已由 T09 的前置关系传递覆盖，不重复设置 blocker。
验收条件
- outboxIndexRepairAfterCrash：提交 run/outbox 后发布索引前崩溃可修复。
- 重复候选、过期索引和两个 polling Worker 不产生第二个有效执行 owner。
- due retries 仅在 Redis 到期后进入 runnable；scanner 与其他 scanner 并发安全。
- Redis 异常停止领取且不丢权威状态；明确轮询 batch/backoff 和扫描容量上限。
非目标
- 不引入 MQ/Redis Streams，不将 heartbeat 视为外部副作用完成证明。
兼容与验证约束
- Java 17；新增路径显式启用，保留既有 API、默认行为、checkpoint/Store 格式及配置前缀。
- Core 不引入 Redis/Extensions 依赖；存储实现归 Extensions，应用接线归可选运行层。
- 首阶段无 MQ、整 Run Worker，不做节点分片，不承诺通用 exactly-once 或 Redis HA RPO0。
- 先用模型无关单测/reference 验证本任务，再按范围运行现有 API/格式门禁；涉及 Redis 用真实存储测试，涉及接管用独立 JVM。
当前实现依据
- Redis leased saver storage/atomicity, Namespace hash-tag key.
- Expired-turn orphan reconciliation.
- Work-item coordination contract, Claim/active acknowledgement/heartbeat, Hands durable enqueue. Work ack here means active/start acknowledgement, not durable graph completion.
分析基线：Core 6324f53、Extensions ed07835；AgentScope 参考 c95eb1f。本 Issue 是待完成任务，不代表能力已经实现。
Activity
Merge pull request #8 from GTyingzi/feature-mcp-nacos-refactor-distri…
PoisonGodv commentedon Nov 6, 2026
Context. Issue #8 asks (among other criteria) that duplicate candidates, expired
indexes and two polling Workers never produce a second valid execution owner, and
that a Redis outage stops claiming without losing authoritative state. The merged
#69 adds RedisLeasedProgressStore: a session lease (claim/renew/release) whose
owner+fencing token guards run/journal/outbox/checkpoint writes; every mutation is
one atomic Lua script that re-validates owner, fence, Redis server TIME and
revisions/entry-ids before its first write, with client-side script retries
disabled. C1 (outboxIndexRepairAfterCrash) belongs to the still-open #74
(cf8daf5) and is deliberately not claimed here.
What was run (all against real Valkey 8.1.2, AOF appendfsync always, and real
separate JVM processes — no in-process simulation of multiple JVMs):
claim ttl parameter) expires -> a second JVM takes over with fence 1->2 ->
the killed owner is resurrected in a third JVM from its recorded lease:
renew and commit throw LeaseLostException, release returns false, and the
authoritative hashes are byte-identical (no zombie writes). Takeover observed
2313-2506 ms after the kill across two independent executors (3+2 runs).
over, SIGCONT: the resumed process keeps using its in-memory lease and every
write path is fenced; it exits via the fenced path. 5 runs total.
epoch-ms barrier and claim simultaneously: exactly one winner, N-1
LeaseBusyException, winner fence=1, release then re-claim gives fence=2.
Duplicate candidates are dropped by the store: replaying a commit with a stale
expected revision -> ProgressConflictException "Run revision does not match";
correct revision but an existing append-only entry id -> "entry id already
exists"; a further claim while the lease is live -> BUSY.
every claim/renew attempt failed with connection exceptions (0 successes, no
local degradation); after restarting the same container, AOF recovery
reproduced revision/state/checkpoint/journal/outbox field-for-field; the
takeover then proceeded (fence 2) and the old owner was fenced.
expiry, first success 4003-4099 ms after the claim, fence monotonic.
Adversarial additions from the independent review (same store, public API only):
commit with the CORRECT expected revision -> LeaseLost (the time gate alone
rejects a matching owner+fence), fence retained, next claim fence+1.
first commit; new owner claims but does not commit; the resumed owner's
commit matches the stored revision exactly, so only the owner/fence gate can
reject it - and it does, with zero writes.
the killed owner's own later claim is BUSY (no second lease for anyone,
including the previous owner).
Honest mapping to #8's acceptance criteria on today's main:
rejection on every write path, single winner under concurrent claims,
revision/entry-id dedup). Advisory-index-level candidate dedup is feat(persistence): bounded directory discovery and advisory index repair #74 scope.
resumes"; polling batch/backoff were explicit fixture parameters.
due-retry index and scanner are feat(persistence): bounded directory discovery and advisory index repair #74 components and were not tested.
Known non-blocking nits in our own harness reporting (not store defects), found during independent review: the S4
"connection-type" assertion matches any *Exception class name (actual logged
classes were all WriteRedisConnectionException), one timing figure in the
frozen report has a typo (2447/2436/2506 ms), and the S3 barrier-lag note
understates observed scheduler jitter (winner uniqueness is unaffected because
the server serializes CLAIM atomically).
Nothing is claimed about HA failover/RPO0, multi-primary deployments, or
exactly-once external effects; the store's Javadoc itself scopes these out.
This message reports local verification evidence only; no fix PR is attached.