[dnth]

Intent

Repair the Boat second mate's wake path so it can take routed work again. The captain chose fixing Boat first after the main Firstmate could not route builds there. This changes Firstmate shared tracked code; follow firstmate-coding-guidelines. The main home is /Desktop/firstmate and the existing secondmate is boatmate (SSH alias boat-boatmate). Observed failure: an initial remote-doctor SSH connection timed out while the sandbox was suspended; fm-compute-wake.sh boatmate then failed after about 58 seconds with credential acquisition did not become ready; compensation confirmed. Find and fix the root cause from concrete logs, the auth-run service journal, and endpoint facts, not speculation. Cover the actual failure mode with a regression using existing bin/tests conventions. Do not merely suppress the error or weaken readiness.\n\nProven root cause and implementation decision: auth-run unit fm-boat-61eff1a1e9791eacdd85-9aa78d4f5cdec5e28a5c7ca5ea398737.service logged canonical broker unavailable at 2026-10-15 21:19:37 +0800. The workstation's conventional address 127.0.0.1:8765 was occupied by unrelated Python process /tmp/cdp-probe/refstub.py, whose /v1/healthz returned HTTP 404. An exact installed OMP broker bind probe failed with EADDRINUSE. Credential installation had not begun. SSH endpoint refresh and verified host-key handling were already working; they were not the cause. The fix selects a free service-private workstation loopback broker port for each default credential lease, retains the explicit FM_BOAT_BROKER_URL shared-broker override, and leaves the remote bind 127.0.0.1:8765 unchanged. Never kill the unrelated listener. The regression starts two default credential leases using real systemd services, fixture HTTP brokers and the real scoped facade: each must vend its own distinct fixture credential, and stopping one must not interrupt the other. It failed before the fix with the reported readiness error and passes after.\n\nOperational constraints: owner scripts may diagnose and prove the fix against the real existing boatmate from the main home (fm-compute-wake.sh boatmate; fm-on.sh boatmate fm-remote-doctor.sh; fm-boat.py status/wake/sleep boatmate). Wakes cost compute, so keep them few and deliberate. Never destroy, recreate, or re-provision the sandbox. Never relaunch or steer the boatmate agent itself; Firstmate owns the post-merge relaunch. Never print, manually copy, export, or commit a credential, including the sandbox gh login. Do not hand-edit main-home data or state; owner scripts own operational mutations. Never run doctor --fix or any Herdr lifecycle action. Leave boatmate either asleep or awake with doctor passing; it has already been safely returned asleep, with the scoped bearer shredded, and must not be left awake with failed doctor readiness. Do not run paid live smoke as part of automated fixture tests or create a smoke sandbox. No agent launch, steer, --fix, Herdr lifecycle action, sandbox recreation, or credential disclosure occurred in the implementation smoke.\n\nAcceptance criteria, with current accepted AC2 rescope: AC1: identify the wake root cause with concrete log/unit-journal/SSH or address evidence and write it in the PR, with a receipt citing that evidence. AC2: after the fix, fm-compute-wake.sh boatmate succeeds from the main home, and fm-on.sh boatmate fm-remote-doctor.sh reaches the host. Every remaining failing doctor check must be named, and the PR must show the wake path does not cause it. Remote services normally restored by the second mate's relaunch are Firstmate's post-merge responsibility, not part of this task. Specifically name remote-job-worker, remote-job-probe and the Herdr server for fm-remote; never claim doctor was fully green. Prove through code/docs that ordinary relaunch starts them, without running --fix or any Herdr lifecycle action. AC3: a regression test or deterministic check covers the failure mode where feasible; otherwise explain why and document the manual check in the PR. AC4: no credential printed, manually copied or committed and no sandbox recreation; provide secret-scan evidence and state this in completion.\n\nObserved implementation proof: one real wake using this isolated branch's owner scripts with FM_HOME=/Desktop/firstmate and the main home as current directory returned ready: boatmate. Its lease reached ready using workstation broker http://127.0.0.1:46651 and an authenticated reverse-forward check. The subsequent remote doctor reached the Linux host and exited 1 with exactly remote-job-worker, remote-job-probe and fm-remote herdr-server missing. These are expected separate post-resume restoration prerequisites, not a remaining credential-wake failure. fm-spawn.sh:775-796 orders compute wake before fm_remote_readiness_ensure; fm-remote-readiness-lib.sh:24-43 checks doctor, repairs, then checks again as the relaunch gate; fm-remote-doctor.sh:469-475,775-785,801-842 owns remote worker/probe and Herdr startup; docs/remote-secondmates.md:175-201 confirms compute stop loses the agent and ordinary launch/recovery restores through readiness. Cite these boundaries in the PR. After smoke, the authorized fm-boat.py sleep command returned suspended: boatmate; the same sandbox bx_cbbk3thq is stopped, lease retired, bearer shredded, and the unrelated listener remains untouched.\n\nPrivate evidence to read and carry into the PR's root-cause and verification sections: ~/Desktop/firstmate/data/fm-boatmate-wake-credential/artifacts/wake-evidence.md . Do not publish credentials or incidental private operational paths in the PR. All AC1-AC4 receipts are complete after Firstmate's explicit AC2 rescope; the former AC2 failure remains append-only history, not an accepted-blocked exception. Existing implementation commit is e7475ca. Already exercised: targeted new regression red before and green after; credential suite 15 tests OK, lifecycle suite 24 tests OK, routing 11 checks pass; changed-shell lint with pinned ShellCheck 0.11.0 and documentation audience/link check pass. A full canonical lint invocation hit its 300-second caller deadline; do not misreport that as passing full lint. A targeted ten-family secret signature scan of all added lines found zero findings; gitleaks/detect-secrets/trufflehog were unavailable, so do not imply they ran. Keep the pipeline full: no requested step skips, no bypass of findings, no merge. Ask-user findings must be escalated to Firstmate and never self-answered or bypassed with --yes. Finish at a PR with verified green checks, not at mere implementation completion.

What Changed

• Give each default Boat credential lease a free, service-private workstation loopback broker port, avoiding readiness failures when the conventional port is occupied. Preserve FM_BOAT_BROKER_URL and the remote 127.0.0.1:8765 bind.
• Add fixture HTTP brokers and a regression covering two concurrent default leases, distinct credentials, and continued service after one lease stops.
• Document default broker isolation, the shared-broker override, and concurrent-lease coverage.

Risk Assessment

✅ Low: The change narrowly isolates default broker ports while preserving the explicit override, remote binding, scoped authorization, and cleanup behavior.

Testing

Inspected the change, passed the focused credential suite and isolated product checks, reproduced the failure before the fix, and retained credential-free transcripts. Existing boatmate smoke was not repeated; its accepted operational receipt was supplied in context.

• Live validation: ✅ go - 5 of 6 scenarios driven live against the product

Scenario	Result	Live	Evidence
Start default credential leases while port 8765 returns HTTP 404; acquisition succeeds using private broker ports	✅ pass	live	Product behavior transcript; Before-fix failure reproduction
Run two leases with distinct credentials, then stop one; the other retains its own credential	✅ pass	live	Product behavior transcript
Select an explicit shared broker; the lease serves its scoped credential	✅ pass	live	Product behavior transcript
Select an unhealthy shared broker; readiness fails and cleanup retires the lease without installing a remote bearer	✅ pass	live	Product behavior transcript
Release disposable leases; services and helpers stop while the unrelated conventional listener remains available	✅ pass	live	Product behavior transcript; Targeted credential suite
Wake the existing boatmate and reach its remote doctor	⏸️ untested	no	Not repeated: this phase forbids accessing operator data or mutating the main home, and automated tests must not perform paid smoke. Disposable homes exercised the real credential services and facade…

Evidence: Product behavior transcript

Conventional port already occupied; using read-only health probe
Conventional broker health: HTTP 404
Lease 1: ready; correct distinct fixture credential; private broker http://127.0.0.1:47559; remote bind unchanged
Lease 2: ready; correct distinct fixture credential; private broker http://127.0.0.1:57645; remote bind unchanged
Stopped first lease; second still serves its own credential
Both default leases retired; conflicting listener still responds HTTP 404
Explicit shared-broker override: ready and serves expected scoped fixture credential
Explicit unhealthy shared broker: readiness rejected; lease retired; no remote bearer installed
Disposable homes cleaned; service and helper retirement verified

Evidence: Before-fix failure reproduction

Conventional port already occupied; using read-only health probe
Conventional broker health: HTTP 404
Disposable homes cleaned; service and helper retirement verified
Traceback (most recent call last):
  File "~/.no-mistakes/worktrees/dd71c22cc6d7/01M4ZX9V3EA2DE742F52H002RT/.boat-test-tmp/drive.py", line 27, in <module>
    result=lab.start()
  File "~/.no-mistakes/worktrees/dd71c22cc6d7/01M4ZX9V3EA2DE742F52H002RT/.boat-test-tmp/before/tests/boat-auth-cases.py", line 105, in start
    return self.call('start', 'fixture', str(self.config), 'openai-codex/fixture', ok=ok)
  File "~/.no-mistakes/worktrees/dd71c22cc6d7/01M4ZX9V3EA2DE742F52H002RT/.boat-test-tmp/before/tests/boat-auth-cases.py", line 101, in call
    if ok and result.returncode: raise AssertionError(result.stderr)
AssertionError: error: credential acquisition did not become ready

Evidence: Targeted credential suite

...............
----------------------------------------------------------------------
Ran 15 tests in 70.351s

OK

Evidence: Reproducible isolated product driver

import json, os, runpy, shutil, socket, subprocess, sys, threading, urllib.request, urllib.error
from pathlib import Path
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
root=Path(sys.argv[1]).resolve()
sys.argv=['boat-auth-cases.py',str(root)]
m=runpy.run_path(str(root/'tests/boat-auth-cases.py'),run_name='boat_validation')
Lab=m['Lab']; rows=m['rows']; labs=[]; server=None
class Conflict(BaseHTTPRequestHandler):
    def log_message(self,*args): pass
    def do_GET(self): self.send_error(404)
def health():
    try:
        with urllib.request.urlopen('http://127.0.0.1:8765/v1/healthz',timeout=2) as r: return r.status
    except urllib.error.HTTPError as e: return e.code
try:
    try:
        server=ThreadingHTTPServer(('127.0.0.1',8765),Conflict)
        threading.Thread(target=server.serve_forever,daemon=True).start()
        print('Started disposable conflicting listener on conventional port',flush=True)
    except OSError:
        print('Conventional port already occupied; using read-only health probe',flush=True)
    before=health(); assert before==404, 'Need HTTP-404 conflict to reproduce reported failure'
    print('Conventional broker health: HTTP 404',flush=True)
    for index in range(2):
        lab=Lab(); labs.append(lab); lab.env.pop('FM_BOAT_BROKER_URL')
        lab.update(broker_access='fixture_distinct_'+str(index))
        result=lab.start()
        assert rows(lab.lease)['state']=='ready'
        assert lab.request()['credentials'][0]['credential']['access']=='fixture_distinct_'+str(index)
        settings=json.loads((lab.lease.parent/'settings.json').read_text())
        assert settings['remote']=='127.0.0.1:8765'
        print('Lease '+str(index+1)+': ready; correct distinct fixture credential; private broker '+settings['broker']+'; remote bind unchanged',flush=True)
    a,b=[json.loads((x.lease.parent/'settings.json').read_text())['broker'] for x in labs]
    assert a!=b and ':8765' not in (a,b)
    labs[0].call('stop')
    assert labs[1].request()['credentials'][0]['credential']['access']=='fixture_distinct_1'
    print('Stopped first lease; second still serves its own credential',flush=True)
    for lab in labs: lab.clean()
    labs=[]
    assert health()==before
    print('Both default leases retired; conflicting listener still responds HTTP 404',flush=True)
    lab=Lab(); labs.append(lab); lab.start()
    settings=json.loads((lab.lease.parent/'settings.json').read_text())
    assert settings['broker']==lab.env['FM_BOAT_BROKER_URL']
    assert lab.request()['credentials'][0]['credential']['access']==m['ENTRY']['credential']['access']
    print('Explicit shared-broker override: ready and serves expected scoped fixture credential',flush=True)
    lab.clean(); labs=[]
    lab=Lab(); labs.append(lab); lab.health_status=503
    result=lab.start(ok=False)
    assert 'credential acquisition did not become ready' in result.stderr
    assert rows(lab.lease)['state']=='retired'
    assert not (lab.path/'remote/omp-auth-broker.token').exists()
    print('Explicit unhealthy shared broker: readiness rejected; lease retired; no remote bearer installed',flush=True)
finally:
    for lab in labs: lab.clean()
    if server: server.shutdown(); server.server_close()
    shutil.rmtree(m['BASE'])
    print('Disposable homes cleaned; service and helper retirement verified',flush=True)

Pipeline

Updates from git push no-mistakes

✅ **intent** - passed

✅ No issues found.

✅ **Rebase** - passed

✅ No issues found.

✅ **Review** - passed

✅ No issues found.

✅ **Test** - passed

✅ No issues found.

• Live validation: ✅ go - 5 of 6 scenarios driven live against the product

Scenario	Result	Live	Evidence
Start default credential leases while port 8765 returns HTTP 404; acquisition succeeds using private broker ports	✅ pass	live	Product behavior transcript; Before-fix failure reproduction
Run two leases with distinct credentials, then stop one; the other retains its own credential	✅ pass	live	Product behavior transcript
Select an explicit shared broker; the lease serves its scoped credential	✅ pass	live	Product behavior transcript
Select an unhealthy shared broker; readiness fails and cleanup retires the lease without installing a remote bearer	✅ pass	live	Product behavior transcript
Release disposable leases; services and helpers stop while the unrelated conventional listener remains available	✅ pass	live	Product behavior transcript; Targeted credential suite
Wake the existing boatmate and reach its remote doctor	⏸️ untested	no	Not repeated: this phase forbids accessing operator data or mutating the main home, and automated tests must not perform paid smoke. Disposable homes exercised the real credential services and facade…

• git diff 76de0c5e3512d16769a14b17d1b1d3449e057bce e7475ca8863ec31520d37699548247ed187722bd
• TMPDIR="$PWD/.boat-test-tmp" PYTHONDONTWRITEBYTECODE=1 bash tests/fm-boat-omp-auth.test.sh — 15 tests passed
• python3 .boat-test-tmp/drive.py "$PWD" — real systemd credential services and scoped facade against isolated fixture endpoints
• python3 .boat-test-tmp/drive.py "$PWD/.boat-test-tmp/before" — baseline module reproduced credential acquisition did not become ready
• Removed disposable homes and baseline copy; verified service/helper retirement and clean git status

✅ **Document** - passed

✅ No issues found.

✅ **Lint** - passed

✅ No issues found.

✅ **Push** - passed

✅ No issues found.