Make a FOS failure visible: a Failed task state, and a tab that shows the task log - #1211
Merged
Merged
Conversation
A FOS error report is recorded and announced (#1207, #1208), but the task itself was left exactly where it was: Queued or In-Progress, forever. The task list still said the machine was working on it, and because Host:: loadTask() and getActiveTaskCount() count those states as live, the host could not be re-tasked until somebody noticed and cancelled it by hand. Adds taskStates row 6, Failed, and moves the task to it when FOS reports an error. A warning never does -- a warning means FOS carried on. Not Cancelled, which was the alternative. Cancelled means an administrator stopped it; losing the difference between "somebody stopped this" and "this broke" costs the operator the one fact they open the task list to find. Adding a sixth state is additive because every "is this task live" test in the tree is an allowlist -- getQueuedStates() plus getProgressState() -- so a state nobody listed is inactive by construction. No call site had to learn about it, and Complete and Cancelled rows already persist in `tasks` (20 and 33 of them on the reference install), so a Failed row lingering is the existing behavior rather than new litter. The state is written for every task type, not just imaging ones: a Memtest the host died on is as finished as a deploy. Only HOST_IMAGE_FAIL stays imaging-specific. Guarded on the taskStates row existing, so a web tree updated ahead of its database leaves the task alone rather than pointing it at a state that is not there. Verified end to end against an isolated copy of this server's database (podman MariaDB, shadow web tree, throwaway host on a locally-administered MAC): an error on a deploy task moved it to 6 and dropped the host's active task count to 0; an error on a wipe task did the same; a warning left the task Queued; every path still answers an identical `##`. Step 339 re-runs clean. All seven new assertions mutation-tested. Co-Authored-By: Claude <noreply@anthropic.com>
A Failed task is in no pane otherwise. It is not an active state, so the active pane excludes it by construction, and the Recent pane -- the only view of finished tasks there is -- asked for Complete and Cancelled only. The state would have existed with nothing in the UI able to show it. Recent's state filter becomes All / Complete / Cancelled / Failed, and the default includes Failed. 'both' becomes 'all' because there are three finished states now and a two-way label was about to start lying; getRecentTasks() still reads any unrecognised value as all-of-them, so a page rendered before this keeps working. FOGBase gains the getFailedState() passthrough its five siblings already have, so pages reach it the same way they reach the others. Verified against the isolated lab database: an error report moved the fixture task to 6 with taskStateChangedTime stamped (Task::set() already does that for any state change), and the pane's own WHERE returns 54 rows where the old one returned 53. Three new assertions mutation-tested. Co-Authored-By: Claude <noreply@anthropic.com>
taskLog has been written since 1.2 -- a row per task state transition -- and there has never been a way to read it short of SQL. That stopped being merely untidy at schema 338, when FOS reports started landing in the same table: the text a machine sends when a deploy dies was being stored and shown to nobody. Adds a Logs tab to Task Management: time, host, task type, state at the time, entry type, message, who recorded it. Errors and warnings are badged (danger/warning) because the type is what tells an operator whether the machine stopped or carried on. Defaults to reports rather than everything. State rows outnumber reports several-fold -- 52 against 2 on the reference install after two test reports -- and would bury the thing the tab exists to surface; All is one click away, as are errors, warnings and state changes on their own. Read through a derived table: taskLog and tasks both have `taskID` and `taskStateID` columns, and complex() builds its select list as bare backticked names, so an unqualified name across that join is ambiguous and the query dies. Aliasing inside the subquery gives every column a name of its own; MariaDB merges a derived table with no aggregate, so this is not a materialisation. tests/task-log-view.test.php pins the three things that fail silently: a grid column asking for a key the endpoint does not emit, a filter radio posting a value the switch does not handle (which shows the default and looks like it worked), and the table id drifting between the pane and its builder. Six mutations, all caught. Verified against the isolated lab database: the endpoint returns the mapped row shape through complex() with the joins resolving host name, task type and state icons, and each filter's WHERE returns the expected split (2 reports, 1 error, 1 warning, 52 state, 54 all). FOG_BCACHE_VER bumped -- fog.task.list.js changed. Co-Authored-By: Claude <noreply@anthropic.com>
mastacontrola
added a commit
that referenced
this pull request
Aug 19, 2026
Ported from 1.6 #1211, completing the port of #1210. A FOS error report is recorded there, and the task itself was left exactly where it was: Queued or In-Progress, forever. The task page still said the machine was working on it, and because Host::loadTask() and getActiveTaskCount() count those states as live, the host could not be re-tasked until somebody noticed and cancelled it by hand. Adds taskStates row 6, Failed, and moves the task to it when FOS reports an error. A warning never does -- a warning means FOS carried on. Not Cancelled, which was the alternative. Cancelled means an administrator stopped it; losing the difference between "somebody stopped this" and "this broke" costs the operator the one fact they open the task list to find. Additive on a maintenance branch, which is why it is portable at all: every "is this task live" test in the tree is an allowlist -- getQueuedStates() plus getProgressState() -- so a state nobody listed is inactive by construction, and no call site had to learn about it. 1.5 has no history pane, so a Failed task is as invisible in the UI as a Complete or Cancelled one already is; what changes is that the host is free and the task has stopped claiming otherwise. The state is written for every task type, not just imaging ones: a Memtest the host died on is as finished as a deploy. Only HOST_IMAGE_FAIL stays imaging-specific. Guarded on the taskStates row existing, so a web tree updated ahead of its database leaves the task alone rather than pointing it at a state that is not there. Verified against an isolated copy of the 1.5 database (podman MariaDB, shadow web tree, throwaway host on a locally-administered MAC, 2079 real hosts): a warning left the task Queued and wrote a row typed warning; an error moved it to 6 and dropped the host's active task count to zero; error, warning and unknown-MAC all still answer an identical `##`. Step 281 applies clean on a database at 278. Seven assertions mutation-tested. Co-authored-by: JJ Fullmer <7743340+darksidemilk@users.noreply.github.com> Co-authored-by: Claude <noreply@anthropic.com>
mastacontrola
added a commit
that referenced
this pull request
Aug 19, 2026
v1.6.11 carries FOGProject/fog-plugins#21, which was merged the morning after v1.6.10 was cut and so has never reached a server. That change matters more than its own PR made it sound, because the FOS reporting work landed in between. #1206/#1211/#1217/#1223 give a failed task a stored, multi-line report of what FOS actually said, and TaskError sends the flattened opening of it as HOST_IMAGE_FAIL's Reason -- but every bundled listener on v1.6.10 ignores that key and pushes the fixed string "This host has failed to image". Confirmed live before cutting the release. A report whose stored row read fog.download: failed to restore partition 2 partclone.ntfs: /dev/sda2 is busy ERROR: win11-split part 2 checksum mismatch exit code 1 (fog.download) pushed, in its entirety: "fos-deploy-test Failed" / "This host has failed to image". So the whole point of storing the trace stopped at the server. One line, because the sha256 is fetched from the release alongside the tarball rather than pinned here; bin/fetch-plugins.sh reads this constant and verifies what it downloads against it. Verified end to end on this pin: "Fetching plugins v1.6.11" -> "Plugins at v1.6.11", and all three imagefail listeners in the fetched tree read Reason. The 1.5 line keeps its plugins in-tree and has no pin, so its equivalent is a code change: #1226. Co-authored-by: JJ Fullmer <7743340+darksidemilk@users.noreply.github.com> Co-authored-by: Claude <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #1207 / #1208. FOS reports an error, FOG records it — and then nothing about the system changed: the task stayed where it was, and the row it wrote landed in a table nothing has ever displayed.
Two commits, one job: make a FOS failure something you can see.
1. A Failed task state (schema 339)
A task the host died on stayed Queued or In-Progress forever.
Host::loadTask()andHost::getActiveTaskCount()count those states as live, so the host could not be re-tasked until somebody noticed and cancelled the task by hand.taskStatesrow 6, Failed — "Host reported that the task could not be completed."TaskState::getFailedState(), with aFAILED_STATEhook like every other state getter, plus theFOGBasepassthrough its five siblings already haveTaskErrormoves the task to it on an error. A warning never does — a warning means FOS carried on.Not Cancelled, which was the alternative: Cancelled means an administrator stopped it, and losing the difference between "somebody stopped this" and "this broke" costs the operator the one fact they open the task list to find.
Adding a sixth state is additive because every "is this task live" test in the tree is an allowlist —
getQueuedStates()plusgetProgressState()— so a state nobody listed is inactive by construction. Complete and Cancelled rows already persist intasks(20 and 33 of them on the reference install), so a Failed row lingering is existing behavior, not new litter.The state is written for every task type, not just imaging ones — a Memtest the host died on is as finished as a deploy. Only
HOST_IMAGE_FAILstays imaging-specific._markFailed()is guarded on thetaskStatesrow existing, so a web tree updated ahead of its database leaves the task alone rather than pointing it at a state that is not there. Apart fromstateIDthe endpoint writes nothing about the task — it is unauthenticated, matched by MAC like every FOS endpoint, so what it may change stays a list of one.Recent pane: the state would otherwise be invisible — not active, and Recent asked for Complete and Cancelled only. Its filter becomes All / Complete / Cancelled / Failed with Failed in the default. (
both→all, since a two-way label was about to start lying; the endpoint still reads any unrecognised value as all-of-them.)2. A tab that shows the task log
taskLoghas been written since 1.2 and displayed never. Schema 338 started putting FOS report text in the same table, so the message a machine sends when a deploy dies was being stored and shown to nobody.New Logs tab on Task Management: time, host, task type, state at the time, entry type, message, recorded by. Errors and warnings are badged, because the type is what tells you whether the machine stopped or carried on. Defaults to reports — state rows outnumber them several-fold and would bury the thing the tab exists to surface — with All / Errors / Warnings / State changes one click away.
Read through a derived table:
taskLogandtasksboth carrytaskIDandtaskStateID, andcomplex()builds its select list as bare backticked names, so an unqualified name across that join is ambiguous and the query dies. Aliasing inside the subquery gives every column a name of its own; MariaDB merges a derived table with no aggregate in it.FOG_BCACHE_VERbumped —fog.task.list.jschanged.Verification
Against an isolated copy of this server's database (podman MariaDB on :13306, shadow web tree, throwaway host on a locally-administered MAC — nothing live was touched):
getActiveTaskCount()→ 0,loadTask()invalidtaskLogrow typedwarning##bodytaskStatesstill 6 rowsgetTaskLogs()throughcomplex()Ten new assertions across
tests/task-error-report.test.phpand the newtests/task-log-view.test.php, every one mutation-tested. Full suite green (56 files).Downstream
None — no route classes added or removed.