Skip to content

chore(rust): replace LEECH_DISABLE_RUST with --backend on data prepare - #208

Open
jayhesselberth wants to merge 1 commit into
mainfrom
chore/drop-leech-disable-rust
Open

chore(rust): replace LEECH_DISABLE_RUST with --backend on data prepare#208
jayhesselberth wants to merge 1 commit into
mainfrom
chore/drop-leech-disable-rust

Conversation

@jayhesselberth

Copy link
Copy Markdown
Member

Closes #177.

LEECH_DISABLE_RUST was a temporary workaround for #176, added because "Rust is available" and "Rust is faster here" are different claims and nothing could measure the second. They came apart badly — 13 reads/s against 130–234 on the Python pool. #176 is fixed (#178: concurrent batch dispatch, the process-global POD5 reader cache, and py.detach over the I/O), so the hatch has outlived its reason.

Why this isn't a straight delete

#177 offered three options. (a) "delete it" turned out to need a piece of (b) first: --backend existed only on predict, so the env var was the only way to force the Python path on prepare — and docs/troubleshooting.md told people to use it for exactly that, in both the throughput and read-yield sections. Removing it would have taken the diagnostic with it.

So the hatch moves to the seam #169 built rather than disappearing.

Changes

  • data prepare gains --backend auto|rust|python, matching predict.
  • One resolver, preparation.parallel._select_prepare_backend(), ANDs the choice with availability and rust_prepare_unsupported_reason(config). One place decides, so the startup log line and the dispatch cannot disagree.
  • Scoped to prepare. The old variable raised inside the try: in _rust_accel.py, so it killed leech_core process-wide — move-table parsing and inference included, where the extension is a real win. That was far broader than the one step it was meant to switch.
  • --backend rust raises instead of falling back, whether the cause is a missing leech_core, an unsupported config, or --workers 1 (the sequential path is Python-only). A forced run that quietly took the other path measures nothing, which is the whole reason the switch exists.
  • LEECH_DISABLE_RUST and the DISABLE_RUST branch are gone from _rust_accel.py (the os import with them); both docs references now say --backend python.

The two backends produce identical chunks — that is what tests/test_backend_parity.py enforces field by field — so this flag only ever changes throughput. The help text says so.

Tests

Seven new cases in tests/test_prepare_dispatch.py::TestBackendSelection, covering each choice against available/unavailable and supported/unsupported configs, plus the raise-don't-fall-back property. They need no POD5/BAM fixtures, matching the rest of that file.

1015 passed, 27 skipped. ruff format --check, ruff check, and ty check clean.

Downstream

rnabioco/escapepod-modelsworkflow/rules/charging.smk, rule charging_leech_prepare — sets LEECH_DISABLE_RUST=1. That should now be dropped outright, since #176 is fixed and the default path is the fast one. If a pin is still wanted, it is --backend python.

🤖 Generated with Claude Code

LEECH_DISABLE_RUST was a temporary workaround for #176, added because
"Rust is available" and "Rust is faster here" are different claims and
nothing could measure the second. #176 is fixed (#178), so the hatch has
outlived its reason (#177).

Deleting it outright would have cost the diagnostic: --backend existed
only on predict, so the env var was the only way to force the Python
path on prepare, and docs/troubleshooting.md said to use it for exactly
that. So it moves to the seam #169 built instead of just disappearing.

- data prepare gains --backend auto|rust|python, matching predict.
- One resolver, _select_prepare_backend(), ANDs the choice with
  availability and rust_prepare_unsupported_reason(config). Scoped to
  prepare: it no longer kills leech_core process-wide for move-table
  parsing and inference, where the extension is a real win.
- --backend rust raises rather than falling back, whether the cause is a
  missing leech_core, an unsupported config, or --workers 1 (the
  sequential path is Python-only). A forced run that quietly took the
  other path measures nothing.
- LEECH_DISABLE_RUST and the DISABLE_RUST branch are gone from
  _rust_accel.py; both docs references now say --backend python.

Closes #177
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

LEECH_DISABLE_RUST landed undocumented in #175 — document, relocate, or delete it with #176

1 participant