fix(livekit-agents): cancel in-flight recovery in the LLM fallback adapter's aclose

[LHMQ878]

Summary

llm.FallbackAdapter's recovery task outlives aclose().

When a provider fails, FallbackLLMStream._try_recovery starts a background task that re-issues the request against it to see whether it has come back. The task is stored on _LLMStatus.recovering_task, but FallbackAdapter.aclose() only detached the metrics handlers:

async def aclose(self) -> None:
    for llm_instance in self._llm_instances:
        llm_instance.off("metrics_collected", self._on_metrics_collected)

The two sibling adapters already cancel theirs, which is what makes this look like an oversight rather than a deliberate difference:

adapter	cancels recovery in aclose
stt.FallbackAdapter (stt/fallback_adapter.py:292)	yes — both recovering_recognize_task and recovering_stream_task
tts.FallbackAdapter (tts/fallback_adapter.py:133)	yes
llm.FallbackAdapter	no

Measured on bbf163f

A primary that raises APIConnectionError on its first request and then blocks on the recovery request, so the attempt is still in flight when the adapter is closed:

	before	after
recovery task pending after await adapter.aclose()	yes	no
recovery request runs to completion once released	yes	no
_status[0].available flipped back to True post-aclose	yes	no
llm_availability_changed emitted on the closed adapter	yes	no
tasks left on the loop after closing the adapter and every provider	3	0

That last row is the practical one. Closing the adapter and then each LLM — the ordinary teardown order — still left FallbackLLMStream._try_recovery.<locals>._recover_llm_task pending, along with the LLMStream main and metrics tasks it owns. So a real provider request stays open past shutdown, and the task keeps the chat context it was constructed with alive. It also means a _status mutation and an event can land after the caller has stopped listening.

Change

Cancel the recovery tasks in aclose(), the same way the STT and TTS adapters do:

async def aclose(self) -> None:
    # a recovery attempt outlives the stream that started it, so cancel it here rather
    # than leaving an in-flight request that can still flip availability and emit on a
    # closed adapter (the STT and TTS fallback adapters do the same)
    for llm_status in self._status:
        if llm_status.recovering_task is not None:
            await aio.cancel_and_wait(llm_status.recovering_task)

    for llm_instance in self._llm_instances:
        llm_instance.off("metrics_collected", self._on_metrics_collected)

Test

test_aclose_cancels_in_flight_recovery in tests/test_llm_fallback.py, confirmed red on the unmodified tree before being kept (AssertionError: expected aclose to cancel the in-flight recovery request, it outlives the stream that started it).

It drives the real adapter: the primary fails, the fallback answers, the recovery attempt blocks, then aclose() is called. Beyond asserting the task is done it also releases the blocked request afterwards and asserts the attempt did not resurrect — no available flip, no extra llm_availability_changed. Cancelling the task alone would satisfy a done() assertion even if the underlying request completed anyway, so the post-release assertions are what pin the actual behaviour.

Verification

• ruff check and ruff format --check on both changed files — clean.
• mypy (repo config, strict) on livekit.agents.llm — 1 diagnostic, identical with and without this change (a pre-existing import-untyped for livekit.plugins.google.utils, from that plugin not being installed in my environment).
• uv run pytest --unit: I could not install the full workspace on this machine — bithuman 2.7.0 publishes no Windows wheel, so uv sync --all-extras --dev fails outright. I ran the suite against an editable livekit-agents instead, which leaves 15 modules erroring on absent plugin packages. Captured the sorted FAILED/ERROR set with the change stashed and unstashed: byte-identical (15 collection errors + 4 failures in test_ivr_activity.py and test_tokenizer_xml_markup.py, all pre-existing and unrelated), 1567 passed both ways. tests/test_llm_fallback.py is green: 3 passed.

I did not touch CHANGELOG.md or any package manifest, per CONTRIBUTING.

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[devin-ai-integration]

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

[LHMQ878]

Closing this as a duplicate of #4921, which has been open since February with the same one-file change to llm/fallback_adapter.py and its CLA already signed. I searched issues before filing but not open PRs, which is how I missed it — my apologies for the noise.

@bilalahmed14's PR should be the one that lands. For whatever it's worth to a reviewer, I did measure the behaviour before writing mine, and it may help move #4921 along:

• With a primary that fails once and then blocks on the recovery request, adapter._status[0].recovering_task is still pending after await adapter.aclose().
• Releasing it lets it run to completion, flip available back to True, and emit llm_availability_changed on an adapter the caller had already closed.
• Closing the adapter and then every provider — the documented teardown order — still leaves the task pending on the loop.

If it's useful, I'm happy to contribute the regression test (test_aclose_cancels_in_flight_recovery, confirmed failing on an unmodified tree) as a follow-up to #4921 rather than as a competing PR.

Separately, while comparing the three adapters I found what looks like a distinct bug in the TTS adapter: FallbackSynthesizeStream._try_recovery bails on if not retry_text: return before starting a probe, so a provider marked unavailable is never re-probed on the streamed path — the chunked path recovers it on the next request. Filing that separately rather than folding it in here.