Skip to content

Move req_llm to 1.24.0 and update the other outdated dependencies - #138

Merged
deepfates merged 5 commits into
mainfrom
claude/reqllm-1-24
Sep 24, 2026
Merged

deepfates merged 5 commits into
mainfrom
claude/reqllm-1-24

Conversation

@deepfates

@deepfates deepfates commented Sep 23, 2026

Copy link
Copy Markdown
Owner

Supersedes #110.

req_llm 1.17.1 to 1.24.0

#110 stopped because the multimodal benchmark manifests declared an exact ReqLLM build (version, package hash, upstream revision) and MultimodalManifest.runtime_dependency!/1 refused to dispatch under any other, so each ReqLLM release meant editing the module constant, both manifest files and their checksums. The hold on #110 was that this made the admitted multimodal run unreproducible from the tree. That run is no longer admitted: MULTIMODAL_FIDELITY.md records it as invalidated, and c02e91f removed its record and the validator that pinned its manifest SHA. The declaration protected nothing and blocked the library's own dependency.

What changes:

  • The manifests no longer carry the ReqLLM dependency (schema_version 3; payload checksums recomputed). MultimodalManifest.runtime_dependency!/0 reads the loaded ReqLLM version and its Hex package checksum from mix.lock and raises when they disagree. The runner binds that dependency into the provider, so the checkpoint identity, every request audit and the artifact record the ReqLLM that actually serialized and sent the requests, and a checkpoint cannot resume under a different ReqLLM package. New test: "the campaign records the loaded ReqLLM package and a checkpoint cannot resume under another". Removing the binding from the runner fails it and two resume tests.
  • ReqLLM 1.18.0 honours a caller's max_retries: 0 (upstream #819). The transport guard in Imp.Clients.ReqLLM stays because it emits the attempt event that campaign budgets count; its comment and test/campaign_budget_test.exs now say that instead of describing the old reset.
  • benchmarks/authorities.json pins 1.24.0 (tag commit fd9e079f). test/authority_inventory_test.exs now compares the pin's version and Hex checksums with mix.lock, so the next bump fails that test until the pin moves too.

Changelog 1.17.1 to 1.24.0, read for anything affecting Imp: no removed or renamed API that Imp calls. Relevant fixes: caller retry limits honoured (above); OpenAI Responses response.failed stream events decode as terminal errors; Anthropic thinking signatures kept with tool calls; OpenAI replayed reasoning items kept at their own turn. Under an Imp.Deadline, Imp now caps ReqLLM's :total_timeout (new in 1.18) as well as :receive_timeout. The receive timeout bounds one attempt's wait for the next bytes, and ReqLLM retries a timed-out attempt and waits out a 429's retry-after with the full timeout each time, so a call under a deadline could run several times past it (measured through Dwell.Deadline: a silent provider with 800 ms left took 3.4 s and 4 requests; a 429 with retry-after 20 s took 60 s; with the cap, 0.8 s and 1 request). ReqLLM runs a call with a total timeout in an unlinked task, so a request step in that task ends it when the caller dies and makes its telemetry count as the caller's (bind_to_caller/1, Imp.Telemetry.act_for/2); without it, a cancelled call kept making requests whose cost was never recorded, and campaign budgets lost the usage. Transitive: jsv 0.24.0, llm_db 2026.9.5, texture 2.0.0 (via jsv), dotenvy, splode, zoi patch releases.

Other outdated dependencies

Updated: earmark_parser 1.4.46, erlexec 2.5.0 (fixes kill_group killing the port program when a child cannot join its requested group), ex_doc 0.40.4, mox 1.3.2. Removed from the lock: ex_aws_auth and goth, optional ReqLLM dependencies nothing here declares (mix deps.unlock --check-unused).

Left as is: dialyxir stays at 1.4.7. 1.4.8 matches ignore entries on the line alone, so every {line, column} entry in .dialyzer_ignore.exs stops matching and mix dialyzer.check fails on unused filters (the first CI run of this PR showed it). Moving means rewriting those entries, which is its own change. After this, dialyxir is the only dependency mix hex.outdated lists.

.dialyzer_ignore.exs: the lib/req_llm.ex entry is removed because req_llm 1.24.0 no longer produces that warning, and the multimodal_runner.ex entry moves up one line with the runner change.

Checks

  • mix format --check-formatted, mix compile --warnings-as-errors: clean
  • mix check: 54 doctests, 9 properties, 2866 tests, 0 failures, 13 skipped (220 excluded)
  • mix dialyzer.check: Total errors: 149, Skipped: 149, Unnecessary Skips: 0, passed
  • mix quality.check: credo found no issues; mix_audit found no vulnerabilities (the two cowlib advisories remain on the existing ignore list; cowlib 2.20.0 is the latest release)

Dwell's lock moves to the same req_llm in its own PR.

…oads

The multimodal manifests declared an exact ReqLLM build and the runner
refused to dispatch under any other, so every ReqLLM release meant editing
the manifest constant, both manifest files and their checksums. The run
those manifests described was invalidated and its record removed, so the
declaration protected nothing.

The manifests no longer carry the dependency (schema_version 3). The runner
reads the loaded ReqLLM version and its Hex package checksum from mix.lock,
refuses to run when they disagree, and binds that dependency into the
checkpoint identity, request audits and artifact, so a checkpoint cannot
resume under a different ReqLLM package.

ReqLLM now honours a caller's max_retries: 0. The transport guard stays,
because it is where the attempt event that campaign budgets count is
emitted; its comment and test describe that instead of the old reset.

benchmarks/authorities.json pins 1.24.0 (tag commit fd9e079f), and its
test now reads version and Hex checksums from mix.lock rather than a copy.

.dialyzer_ignore.exs: the lib/req_llm.ex entry is gone because req_llm 1.24.0
no longer produces that warning, and the multimodal_runner.ex entry moves up
one line with the runner change.
erlexec 2.5.0 stops kill_group from killing the port program when a child
cannot join its requested process group. ex_aws_auth and goth are optional
ReqLLM dependencies that nothing here declares; mix deps.unlock
--check-unused reported both.

dialyxir stays at 1.4.7: 1.4.8 matches ignore entries on the line alone, so
every {line, column} entry in .dialyzer_ignore.exs stops matching and the
check fails on unused filters.
Imp.Deadline capped :receive_timeout, which bounds one attempt's wait. ReqLLM
retries a timed-out attempt, and waits out a 429's retry-after, each attempt
with the full receive timeout, so a call under a deadline could run several
times past it. :total_timeout (ReqLLM 1.18+) bounds the call as a whole.
With :total_timeout set, ReqLLM runs the call in a task under its own
supervisor, not linked to the caller. Three things broke there:

- A caller killed mid-call (a cancelled Imp.Run) left the task running. It
  kept retrying against the provider until its timeouts ran out; against a
  silent provider, three more requests arrived after the caller was gone.
- The campaign budget's usage handler keeps only events emitted by its
  owner, so usage ReqLLM emitted from the task was dropped: a budgeted call
  under a deadline recorded 0 input tokens instead of 7. Every GEPA trial
  runs under a deadline.
- The transport attempt event lost the caller's trace, so Imp.trace/2 of a
  call under a deadline did not contain it.

A request step now runs first in that task: it ends the task when the
caller goes down, and makes the task emit as the caller
(Imp.Telemetry.act_for/2). Handlers that keep only their owner's events ask
Imp.Telemetry.emitted_for?/1, which accepts that task but still excludes
other processes the owner started.

The two ReqLLM client ignore entries in .dialyzer_ignore.exs move to the
lines the code now occupies; the previous commit left them stale and
dialyzer.check failed on unused filters.
Imp now requires req_llm ~> 1.18, and this lock still held 1.17.1. CI's
differential job runs mix deps.get in examples/deployment, which rewrote the
tracked lock, and the hover/papillon pilot tests then refused the dirty
tree ("Imp candidate tracked tree is dirty"). This is the lock that
deps.get produces.
@deepfates
deepfates merged commit ae9aa9f into main Sep 24, 2026
10 checks passed
@deepfates
deepfates deleted the claude/reqllm-1-24 branch September 24, 2026 03:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant