Skip to content

slicecamd: anchor the fine-acquire settle to the executed TCS move, not a timer - #478

Closed
cfremling wants to merge 1 commit into
mainfrom
feat/fineacquire-wait-for-move
Closed

slicecamd: anchor the fine-acquire settle to the executed TCS move, not a timer#478
cfremling wants to merge 1 commit into
mainfrom
feat/fineacquire-wait-for-move

Conversation

@cfremling

@cfremling cfremling commented Aug 14, 2026

Copy link
Copy Markdown
Collaborator

THIS FIXES A CRITICAL BUG (measured, UT 2026-08-06 07:50, BD+28d4211)

Fine-acquire corrections do not move the telescope: offset_acam_goal sends
offsetgoal <dRA> <dDEC> fineguiding to acamd, which only shifts the guide
GOAL — the physical pt_offset happens at acamd's next guide solve, up to
one full guide cycle (ACAM exptime + ~few s) later. slicecamd's post-correction
settle is currently a fixed 3 s sleep anchored at the send (FINE_ACQUIRE_SETTLE_SEC,
slicecam_interface.cpp:576-578), followed by 2 skipped frames and 3 fresh
samples. This works fine most of the time, but very short slicecam exposure times in combination with longer exposure times set on acam WILL lead to a situation where all of the slicecam safety and wait logic
completes before the telescope actually moves. This WILL result in a double offset by fineacquire, which can in a very unfortunate case also lead to a fineacquire 'lock' in between these two guide goal changes. Result is that the star is NOT centered on exposure.

On 2026-08-06 the per-frame log shows this end to end:

  • 07:50:25.320 — measurement 1: dRA=−1.035 dDEC=−2.429 (r=2.64″) → goal shift 1 sent
  • 07:50:31.013 — tcsd receives offset -0.983478 -2.485729 40 (move 1, 5.7 s after the send); DONE 31.893
  • 07:50:30.630 / 31.408 / 32.173 — measurement 2's three frames: the first two show the star at the identical pre-move pixel (160,112)
  • 07:50:32.173 — measurement 2: dRA=−0.931 dDEC=−2.388 (r=2.56″) — the same error re-measured from stale sky → goal shift 2 sent
  • 07:50:38.358 — tcsd receives offset -0.902148 -2.568997 40 (move 2 — the double application)
  • 07:50:39.059 — "converged r=0.249″" (star genuinely on the aimpoint after move 1 alone) — 0.13 s before move 2 completes and drags it off
  • During the following three exposures acamd's during-exposure plate solve puts the field at DEC 28.877943 — 0.068″ from the doubly-shifted goal, i.e. the guide loop held the star ~2.6″ off the slit, locked on the wrong spot.
  • All three frames of the V=10.5 standard extracted at SNR −1.5 to −2.5 (expected ~400–500).

Prevalence: 1 confirmed double in 109 fine sessions scanned — rare because the
typical guide cadence (~5 s) beats the ~6 s fineacquire re-measure time, but the odds rise
toward certainty in the combination when ACAM exposures lengthen (poor transparency) while bright
stars keep slicecam frames short.

The fix (add a real check for guide goal change instead of the fixed wait in fineacquire)

  1. acamd: after each successful tcsd.pt_offset in Target::do_acquire,
    increment a monotonic ptoffset_seq (+ timestamp) and include it in the
    already-published status.
  2. slicecamd: do_fineacquire records the counter before sending a goal
    shift; on the guiding path it then waits (existing acamd-status
    subscription + condition variable) until the counter advances — i.e. the
    move was executed — before starting FINE_ACQUIRE_SETTLE_SEC (which
    thereby becomes a true post-move mount settle) and the existing
    reset/settle-frames logic. Timeout FINE_ACQUIRE_MOVE_TIMEOUT (default
    60 s, cfg.in knob; 0 disables): on timeout, log a WARNING and fall through
    to today's behavior — fail-open, never a hang. A stop interrupts the
    wait; the direct-TCS path (guide loop not running) is synchronous and
    needs no wait.
  3. This also removes the converged-while-a-shift-is-in-flight defect for
    free: no sample any decision uses can predate the move it follows.
  4. Ride-along log fix: offset_acam_goal's "requested offsets … arcsec"
    line printed degrees while guiding (the ×3600 sits in the non-guiding
    branch only, slicecam_interface.cpp:2619-2636). Log now prints arcsec on
    both paths; nothing sent to acamd changes.

Notes for the reviewer

  • Config lives in slicecamd.cfg.in (the deployed .cfg is cmake output and
    is regenerated on rebuild — the June 22 aimpoint/gain revert).
  • Not compiled locally (no build environment); build on the instrument.
  • Acceptance after deployment: scrape a night's fine sessions and check every
    post-correction sampling gap sits after its move-DONE; the robot's
    completion-audit race detector stays armed as the regression alarm either
    way.

🤖 Generated with Claude Code

While ACAM guides, a fine-acquire correction only shifts acamd's goal;
the telescope moves at acamd's next guide solve. The settle was timed
from the send, so with short slicecam exposures the next sample median
could be built from pre-move frames and the same correction applied
twice (UT 2026-08-06, BD+28d4211: exposed 2.6 arcsec off slit).

acamd publishes a counter incremented after each executed pt_offset.
do_fineacquire waits for it to advance before settling; a stop
interrupts the wait; on timeout (FINE_ACQUIRE_MOVE_TIMEOUT, default
60 s, 0 disables) it proceeds on the old timer. Also fixes the
offset_acam_goal log line, which printed degrees labelled arcsec on
the guiding path.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 591ec67b43

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +585 to +586
return this->acam_ptoffset_seq.load(std::memory_order_acquire) > seq_before
|| !this->is_fineacquire_running.load(std::memory_order_acquire);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Correlate the sequence with this goal shift

When a regular guide solve is already in progress as the fine-acquire correction is sent, its pt_offset may have been calculated from the old goal but complete after seq_before was sampled. That unrelated completion advances the generic counter and makes this predicate succeed, so slicecamd begins settling while its requested goal shift still awaits the next guide cycle; short slicecam exposures can therefore reproduce the double-offset failure this change is intended to prevent. The acknowledgement needs to identify a move incorporating this specific goal revision, rather than merely any later pt_offset.

Useful? React with 👍 / 👎.

Comment on lines +582 to +586
const bool moved = this->acam_cv.wait_for( lock,
std::chrono::duration<double>( this->fineacquire_state.move_timeout_sec ),
[this, seq_before]() {
return this->acam_ptoffset_seq.load(std::memory_order_acquire) > seq_before
|| !this->is_fineacquire_running.load(std::memory_order_acquire);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Notify the waiter when fine-acquire stops

If ACAM stalls, stops guiding, or otherwise produces no further status publication, an operator's fineacquire stop only clears is_fineacquire_running and never notifies acam_cv. This wait_for therefore cannot reevaluate its stop predicate until the configured 60-second timeout, leaving the framegrab thread blocked even though the stop command has returned. Notify acam_cv when clearing the running flag so stop actually interrupts the wait.

Useful? React with 👍 / 👎.

@cfremling

Copy link
Copy Markdown
Collaborator Author

Retracted for rework.

@cfremling cfremling closed this Aug 14, 2026
@cfremling
cfremling deleted the feat/fineacquire-wait-for-move branch August 14, 2026 05:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant