✨ Add RL truncation

[flowerthrower]

Description

This PR adds a maximum steps (per episode) argument for Predictor/PredictorEnv initialization. As opposed to tracer_path this is not supposed to be changed after training. It defaults to None, keeping previous implementation behavior consistent.

I decided against implementing the pass timeout (for now), since I did not find an issue with overly long passes during training (might change with additional passes in the future), and this often raises issues when running multiple worker instances.

Fixes #678

Assisted-by: GPT-5.5 via Codex

Checklist

• The pull request only contains commits that are focused and relevant to this change.
• I have added appropriate tests that cover the new/changed functionality.
• I have updated the documentation to reflect these changes.
• I have added entries to the changelog for any noteworthy additions, changes, fixes, or removals.
• I have added migration instructions to the upgrade guide (if needed).
• The changes follow the project's style guidelines and introduce no new warnings.
• The changes are fully tested and pass the CI checks.
• I have reviewed my own code changes.

If PR contains AI-assisted content:

• I have disclosed the use of AI tools in the PR description as per our AI Usage Guidelines.
• AI-assisted commits include an Assisted-by: [Model Name] via [Tool Name] footer.
• I confirm that I have personally reviewed and understood all AI-generated content, and accept full responsibility for it.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[flowerthrower]

@coderabbitai review

[coderabbitai]

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

[coderabbitai]

📝 Walkthrough

Walkthrough

This PR introduces a configurable max_steps parameter to Predictor and PredictorEnv, enabling episode truncation once a step limit is reached. PredictorEnv.step now returns truncated=True with a truncation_reason when the limit is hit. A test and changelog entry accompany the change.

Changes

Add RL truncation

Layer / File(s)	Summary
PredictorEnv max_steps truncation logic src/mqt/predictor/rl/predictorenv.py	PredictorEnv.__init__ accepts and stores max_steps; step checks whether num_steps reaches max_steps, traces the terminal step, and returns terminated=False, truncated=True with info["truncation_reason"] = "max_steps_exceeded".
Predictor wiring, test, and changelog src/mqt/predictor/rl/predictor.py, tests/compilation/test_predictor_rl.py, CHANGELOG.md	Predictor.__init__ gains a documented max_steps parameter forwarded to PredictorEnv; a new test verifies truncation at max_steps=1; CHANGELOG.md records the "Add RL truncation" entry with PR #697 link.

Estimated code review effort: 2 (Simple) | ~10 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Predictor
  participant PredictorEnv
  participant Tracer

  Predictor->>PredictorEnv: __init__(max_steps)
  Predictor->>PredictorEnv: step(action)
  PredictorEnv->>PredictorEnv: increment num_steps
  alt num_steps >= max_steps
    PredictorEnv->>Tracer: trace terminal step
    PredictorEnv-->>Predictor: truncated=True, info.truncation_reason
  else
    PredictorEnv->>Tracer: trace normal step
    PredictorEnv-->>Predictor: normal terminated/truncated result
  end

Loading

Possibly related PRs

• munich-quantum-toolkit/predictor#641: Both PRs modify PredictorEnv.step's terminal-step tracing behavior alongside the PredictorEnv/Predictor integration.
• munich-quantum-toolkit/predictor#714: Both PRs modify PredictorEnv.step around terminal/truncated handling and tracing.

Suggested labels: enhancement

Suggested reviewers: burgholzer

Poem

A rabbit hops with steps to spare,
But now a limit's set with care.
When max_steps runs its course so fast,
Truncation saves the day at last! 🐇
No more loops that never end—
Training speeds up, my furry friend!

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Linked Issues check	⚠️ Warning	The PR adds max-step truncation, but issue #678 also asks for failed/timeout pass truncation, which is not implemented.	Extend the implementation to handle failed/timeout pass truncation as requested in #678, or clarify that this behavior is intentionally deferred.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Out of Scope Changes check	✅ Passed	The changes stay focused on RL truncation, with matching tests, docs, and changelog updates.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Title check	✅ Passed	The title is concise and accurately highlights the main change: RL truncation support.
Description check	✅ Passed	The description includes the summary, motivation, linked issue, and checklist items required by the template.

✨ Finishing Touches

✨ Simplify code

• Create PR with simplified code
• Commit simplified code in branch 678-add-rl-truncation

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}