Skip to content

Fix HEX slit values, EDXD recovery and manual shutter exclusion - #126

Merged
Anthony Sligar (sligara7) merged 21 commits into
NSLS2:mainfrom
sligara7:hex-operator-fixes
Oct 29, 2026
Merged

Anthony Sligar (sligara7) merged 21 commits into
NSLS2:mainfrom
sligara7:hex-operator-fixes

Conversation

@sligara7

@sligara7 Anthony Sligar (sligara7) commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

Scope

Follow-up to Nghia's operator reports and Jakub's review, based on merged
#125 (1518646). No QueueServer or process architecture changes.

Delivered

  • 6 — GeRM statistics: enable callbacks and statistics computation in
    GeRMStatsDataLogic.prepare_single, for TIFF and HDF data paths. No
    device-specific enabling in EDXD plans.
  • 7 — EDXD recovery: restore scan-start motor positions, not requested
    scan-range starts, on success/failure/abort. Grid recovery uses the
    reset_positions_decorator on _body, as requested. Preserve reset opt-outs
    and pause semantics; Halt intentionally bypasses cleanup.
  • 8 — Manual shutters: reserve both controls synchronously before deferred
    submission and disable them until completion; guard against busy/paused RE.
    Keep PR125's zero extra manual settling sleep, explicitly approved by AJ;
    acquisition-plan settling is unchanged. Supply device discovery and command
    policy from a HEX adapter rather than the generic widget.

Explicitly deferred

At AJ's direction, remove the device-specific plan-widget special cases.
5 (slit prefill) and 9 (Two theta override) are not delivered here.
Generic forms use plan signatures as before.

11 is now delivered at the plan layer: edxd_scan and
edxd_custom_pos_list_grid default to save_as_tiff=True. The unchanged generic
GUI derives its checked checkbox from that signature. Python callers omitting
the argument also select TIFF; explicit False still selects HDF5.
A later, separate PR may extract the current GUI/config boundary; it will have
nothing to do with QueueServer.

10 — p4p/exporter: unresolved workstation p4p cleanup error. Local
p4p4.3.0 has p4p.server and ordinary shutdown/explicit cleanup exit cleanly.
The installed launcher references the removed gui environment; its deployment
owner and exact workstation state need operator diagnostics. Document supported
exporter task and read-only diagnostics; do not claim a p4p fix.

PR125 cleanup

  • Fix CLI example to Pass Through, update constants-format docs and remove
    obsolete filter schema. The separate migration document was removed at AJ's request.
  • DCLM/filter restructuring is removed at AJ's request: those source files,
    profile construction and existing related tests match merged PR125 exactly.
    Dataclass/config validation, independent filter settings keys and selected-only
    resolution are deferred to a separate later proposal, not delivered here.
  • Pin Pyright, remove unnecessary ignore/conversion, correct misleading profile
    timeout comment without altering device timeouts.
  • Do not restore magics, Sample Small display labels, auto-tune plots,
    between-aperture behavior, or compatibility aliases removed by PR125.

Verification

Own worktree's locked Pixi environment, with CA/PVA automatic discovery disabled
and addresses restricted to loopback. No borrowed environment/PYTHONPATH override.

  • Prior expanded review suite: 263 passed (saved execution log).
  • Final reviewer-fix regression run:
    pytest tests/test_review_followup.py tests/test_edxd_operator_recovery.py tests/gui/test_generic_widget_boundary.py tests/gui/test_shutter_status.py:
    45 passed.
  • pixi run --locked lint: passed.
  • pixi run --locked type-check: Pyright, 0 errors.
  • Actually reran pixi run --locked hex-gui-mock: inspected generic slit/theta
    defaults, exercised both shutter actions and mutual exclusion, captured
    screenshots, and exited the GUI. No removed special-case behavior claimed.
  • TIFF follow-up: 54 tests passed, including actual emitted TIFF resources
    when the argument is omitted, explicit True/False formats, EDXD recovery and
    generic checkbox/boundary tests. Pyright/lint passed; actual mock GUI rerun
    confirms checked TIFF and user opt-out, with no GUI production-code change.
  • After removing DCLM/filter restructuring: 85 tests passed, covering restored
    DCLM/filter/stop tests plus retained GeRM, EDXD recovery, shutter and GUI-default
    regressions. Configured Pyright and lint passed.

The boundary tests cover the touched generic widgets, not complete architectural
separation of the entire current GUI. Known device-specific policy is not added
to generic widget code.

Mock-only verification: no beamline hosts, live PVs, IOC restart or hardware
validation. No Docker resources used or pruned.

Load selected slit readbacks asynchronously, prepare GeRM statistics, restore initial scan positions, and serialize manual shutter actions without changing settling delays. Label the tilt parameter Two theta while retaining theta. Add full-plan failure/abort/pause regressions and document the removed gui environment and deployment-side exporter launcher diagnostics.

Assisted-by: copilot:gpt-6.1-sol
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings October 29, 2026 15:08
@sligara7
Anthony Sligar (sligara7) marked this pull request as draft October 29, 2026 15:10
Keep zero delay for manual shutter buttons as confirmed by AJ, alongside mutual exclusion. Acquisition-plan timing remains unchanged.

Assisted-by: copilot:gpt-6.1-sol
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The review reports no active issues requiring resolution.

No open issues 🎉

What changed in this PR

Addresses HEX operator issues in slit controls, EDXD acquisition recovery, and manual shutter operation, with documentation for launcher troubleshooting.

Changes:

  • Loads selected slit values asynchronously, rejects stale responses, and labels the tilt input Two theta.
  • Re-enables GeRM statistics before acquisition, restores pre-scan positions, and prevents overlapping manual shutter actions.
  • Adds regression coverage and documents recovery behavior, supported exporter launch commands, and workstation diagnostics.
File Description
tests/​test_edxd.py Corrects a comment describing position restoration.
tests/​test_edxd_operator_recovery.py Tests statistics recovery, position restoration, interruptions, and reset opt-outs.
tests/​gui/​test_shutter_status.py Tests shutter exclusion and busy/paused RunEngine gating.
tests/​gui/​test_plan_widget_slits.py Tests slit readbacks, asynchronous loading, stale responses, and errors.
tests/​gui/​test_edxd_tilt_label.py Verifies the tilt label preserves the submitted theta argument.
src/​hextools/​gui/​shutter_status.py Reserves shutter controls during deferred execution and gates them on RunEngine state.
src/​hextools/​gui/​plan_widget.py Adds asynchronous slit-value loading and the tilt label override.
src/​hextools/​gui/​device_sources.py Adds local device-path resolution for slit reads.
src/​hextools/​edxd.py Enables GeRM statistics and restores initial positions in 1D/grid scans.
docs/​gui.md Documents slit loading, shutter exclusion, and EDXD recovery behavior.
docs/​getting_started.md Documents exporter launching and local launcher/p4p diagnostics.

Review effort: Balanced

Comment thread src/hextools/gui/plan_widget.py Resolved
Comment thread src/hextools/edxd.py Resolved
Comment thread src/hextools/edxd.py Resolved
Keep Python plan defaults unchanged; narrow slit input types for Pyright and verify manual shutter zero-delay exclusion.

Assisted-by: copilot:gpt-6.1-sol
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@sligara7

Copy link
Copy Markdown
Contributor Author

Integration update — 2026-10-29

This draft now incorporates #125 at
b2f5550f5b10ab76cd163f9b02593d1e72e0e806, as requested by AJ.
Do not merge until PR125 has merged. Until then, the comparison to main
also includes PR125's changes. PR125 itself is untouched; no published history
was rewritten.

  • Item11: EDXD scan/custom-position grid GUI Save as TIFF starts checked and
    remains toggleable. Python plan defaults are unchanged.
  • AJ explicitly approved PR125's zero extra manual-button settling sleep,
    combined with our mutual-exclusion guard. Acquisition-plan timing is unchanged.
  • PR125 switches the configured type checker to Pyright; corrected the slit
    field types accordingly.

Verification after integration:

  • Expanded suite: 221 passed, one newly added TIFF test initially failed due
    to a missing mandatory positions-list fixture. Corrected that fixture.
  • Final focused slit/TIFF/shutter tests: 12 passed.
  • Configured lint and Pyright: passed.
  • Actual pixi run --locked hex-gui-mock rerun: slit and tilt submissions,
    shutter exclusion and visibly checked TIFF checkbox verified; GUI exited.
  • Local p4p4.3.0 has p4p.server; ordinary shutdown and explicit cleanup each
    exited0 with empty stderr. The reported workstation p4p error remains
    unresolved
    , not fixed by this PR. No live host/PV access.

This update supersedes the original PR body's main-only base and retained
manual-button settling-delay statements below.

Retain approved manual shutter mutual exclusion and zero manual delay.

Assisted-by: copilot:gpt-6.1-sol
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Move GeRM statistics setup into data logic, use grid reset decorator, remove deferred GUI special cases and clean up PR125 configuration/docs without restoring removed features.

Assisted-by: copilot:gpt-6.1-sol
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Preserve Jakub’s merged set_fe_aperture API unchanged.

Assisted-by: copilot:gpt-6.1-sol
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@sligara7

Copy link
Copy Markdown
Contributor Author

At AJ’s request, removed docs/migration-pr125.md and its docs index entry. Clarification: merged PR125 defines set_fe_aperture, and this PR does not change that name or add the old name back. Jakub’s merged API remains authoritative.

Assisted-by: copilot:gpt-6.1-sol
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Explicit save_as_tiff=False retains HDF5. Verify emitted resource formats and generic checkbox defaults.

Assisted-by: copilot:gpt-6.1-sol
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Restore merged PR125 implementation, profile construction and coupled tests; keep agreed operator fixes. Restructuring remains in history for a separate later proposal.

Assisted-by: copilot:gpt-6.1-sol
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@sligara7

Copy link
Copy Markdown
Contributor Author

At AJ’s request, removed the DCLM/filter restructuring and coupled profile/test changes. These files now match merged PR125 exactly. Kept GeRM lifecycle statistics enabling, EDXD recovery, manual shutter exclusion and plan-level TIFF defaults. Restoration verification:85 tests passed; configured Pyright/lint passed. Restructuring is deferred to a separate later proposal.

@sligara7
Anthony Sligar (sligara7) marked this pull request as ready for review October 29, 2026 16:34
@sligara7
Anthony Sligar (sligara7) requested review from Jakub Wlodek (jwlodek) and a balanced review from Copilot October 29, 2026 16:34

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The GUI tests import optional dependencies unconditionally, causing pytest collection to fail instead of skipping them when GUI extras are absent.

2 open issues

Review effort: Balanced

Comment thread tests/gui/test_edxd_plan_defaults.py Resolved
Comment thread tests/gui/test_shutter_status.py Resolved
Reproduced collection failures with qtpy absent; verify both modules skip with qtpy or bluesky_widgets unavailable and still execute with GUI extras.

Assisted-by: copilot:gpt-6.1-sol
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings October 29, 2026 16:47

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The changes control live beamline hardware, but verification is mock-only, requiring a beamline operator to confirm safe motor recovery and shutter-completion timing.

No open issues 🎉

2 resolved since last review

Review effort: Balanced

Use identical upstream review wording and example; leave PR127 DCLM implementation untouched.

Assisted-by: copilot:gpt-6.1-sol
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@sligara7

Copy link
Copy Markdown
Contributor Author

Checked compatibility with Jakub’s PR127 at720210f. The only merge conflict was overlapping CLI filter documentation; ebafcd6 adopts PR127’s exact wording/example. git merge-tree now succeeds without conflicts. Both PRs delete the obsolete filter schema consistently. PR126 does not modify DCLM or its tests, so PR127’s DCLMConstants/.constants implementation is preserved. No PR127 code was merged into this branch; this is a source/merge compatibility check, not combined runtime validation.

Copilot AI balanced review requested due to automatic review settings October 29, 2026 17:51
@sligara7
Anthony Sligar (sligara7) merged commit d8efa5c into NSLS2:main Oct 29, 2026
6 checks passed
@sligara7
Anthony Sligar (sligara7) deleted the hex-operator-fixes branch October 29, 2026 17:56

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Human assessment is needed because a critical cleanup-ordering risk can leave shutters open during motor recovery, prolonging sample exposure after success, failure, or abort.

No open issues 🎉

Review effort: Balanced

Anthony Sligar (sligara7) added a commit to sligara7/hextools that referenced this pull request Oct 29, 2026
Preserve NSLS2#126 operator behavior and NSLS2#127 DCLMConstants/.constants without restructuring. Keep the extracted HEX shutter adapter outside the generic GUI; boundary tests failed on the duplicate merge-added adapter before removal. Validate the actual locked mock launcher, technique tabs, aperture/filters, TIFF defaults and shutter actions; record screenshots and full GUI, operator, lint and type-check results.

Assisted-by: copilot:gpt-6.1-sol
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants