Skip to content

sysupgrade: ignore --no_reboot when the live flash was rewritten (#2231) - #2236

Merged
widgetii merged 1 commit into
masterfrom
sysupgrade-no-reboot-live-flash
Jul 28, 2026
Merged

sysupgrade: ignore --no_reboot when the live flash was rewritten (#2231)#2236
widgetii merged 1 commit into
masterfrom
sysupgrade-no-reboot-live-flash

Conversation

@widgetii

Copy link
Copy Markdown
Member

Fixes #2231.

The problem

On a camera that flashes its own rootfs, sysupgrade … -x finished the flash and then
declined to reboot — leaving a corpse. free_resources() drops the page cache to free RAM for
the download, so once flashcp has rewritten the partition backing the mounted squashfs, every
read that misses the cache comes back as the new image at stale offsets: SSH key auth, libc
and /etc return garbage. The closing "please reboot manually" was advice nobody could act
on, because the shell that would issue the reboot was one of the things that stopped working.
The camera still answered ping, and only a power cycle recovered it.

Confirmed on the lab hi3516ev300 — the partitions sysupgrade erases are the live mount:

/dev/root      /rom      squashfs ro    <- lowerdir, = "rootfs" mtd3 (root=/dev/mtdblock3)
/dev/mtdblock4 /overlay  jffs2    rw    <- upperdir, = "rootfs_data" mtd4
overlay        /         overlay  lowerdir=/,upperdir=/overlay/root

The fix

Decide the reboot on what was actually written and whether this camera runs from it,
rather than on skip_reboot alone.

  • get_system_info() records root_on_flash by probing /proc/cmdline for a root= naming
    something other than flash. Anchored to the root= token rather than matching a bare
    nfs/mmcblk/ram anywhere in the line as general/overlay/init does: a false "not on
    flash"
    is the one error here that bricks a camera, and the stock command line already carries
    mmz=anonymous and an mtdparts label ending -(rootfs_data). Anything not positively proven
    to live elsewhere — NAND root=ubi0:rootfs, an unreadable /proc/cmdline — counts as flash,
    which at worst costs a reboot.
  • do_update_rootfs, do_update_firmware and do_wipe_overlay mark the run before the
    write, not after: flashcp erases before it writes, so a half-erased partition is just as
    unreadable, and do_update_firmware reaches reboot_system through || die by exactly that
    path. do_wipe_overlay counts because the jffs2 it erases is the upperdir of the running
    overlay. do_update_kernel does not — that partition is not mounted, which is what keeps
    -x useful.
  • reboot_system() reboots anyway when the live flash was rewritten, and says why. A pre-flight
    notice lands before the trap that shields flashcp from a dying TTY, so the operator still
    has Ctrl-C (window is abort_wait, default 5s).

So -x still means what it says for a kernel-only update, an NFS/SD/ram root, or a same-version
run that writes nothing — and no longer promises a working camera it cannot deliver.

Also: reboot_system() now returns instead of exiting on the honoured path, so the main flow's
exit 0 is reached and die() keeps its own exit 1. A deliberate --no_reboot run used to
report failure.

Verified on hardware (lab hi3516ev300, NOR, overlay root, squashfs on mtd3)

run result
-z --build=nightly-20260724-04fe2e1 -x (real upgrade) notice + abort window, kernel+rootfs flashed, --no_reboot ignored, rebooted, came back on the new build with key auth intact
-z --kernel=… --force_ver -x (forced kernel write) flashed, -x honoured, box stayed up, exit 0
-z --rootfs=… -x (same version) nothing written, -x honoured, box stayed up

The script also parse-checks under the camera's own busybox ash.

Tests

.github/scripts/test_sysupgrade.sh grows a Part 1b (20 behaviour cases) and 12 Part 2 drift
assertions: the override itself, kernel-only/same-version/NFS/mmcblk/ram honoured paths, both
combined-image paths, a write that fails after it started, --wipe_overlay, a decoy command
line carrying ramdisk_size=/mmcblkish to pin the anchoring, and a no--x regression guard.

Every one of them fails against 1.0.55:

$ SRC=<1.0.55> bash .github/scripts/test_sysupgrade.sh
FAIL -x + live rootfs -> expected flash then reboot, rc=1 log='flashcp -v …/rootfs.squashfs.ssc338q /dev/mtd3'
…
23 check(s) failed.

🤖 Generated with Claude Code

On a camera that flashes its own rootfs, `-x` finished the flash and then
declined to reboot -- leaving a corpse. free_resources() drops the page cache
to free RAM for the download, so once flashcp has rewritten the partition
backing the mounted squashfs, every read that misses the cache comes back as
the new image at stale offsets: SSH auth, libc and /etc return garbage. The
closing "please reboot manually" was advice nobody could act on, because the
shell that would issue the reboot was one of the things that stopped working.
The camera still answered ping and only a power cycle recovered it.

Decide the reboot on what was actually written and whether this camera runs
from it, instead of on skip_reboot alone:

  - get_system_info() records root_on_flash, probing /proc/cmdline for a root=
    that names something other than flash. Anchored to the root= token rather
    than matching a bare nfs/mmcblk/ram anywhere in the line as init(1) does:
    a false "not on flash" is the one error here that bricks a camera, and the
    stock command line already carries mmz=anonymous and an mtdparts label
    ending -(rootfs_data). Anything not positively proven to live elsewhere --
    NAND root=ubi0:rootfs, an unreadable /proc/cmdline -- counts as flash,
    which at worst costs a reboot.

  - do_update_rootfs, do_update_firmware and do_wipe_overlay mark the run
    before the write, not after: flashcp erases before it writes, so a
    half-erased partition is just as unreadable, and do_update_firmware
    reaches reboot_system through `|| die` by exactly that path.
    do_wipe_overlay counts because the jffs2 it erases is the upperdir of the
    running overlay. do_update_kernel does not -- that partition is not
    mounted, which is what keeps -x useful.

  - reboot_system() reboots anyway when the live flash was rewritten, and says
    why. A pre-flight notice lands before the trap that shields flashcp from a
    dying TTY, so the operator still has Ctrl-C.

So -x still means what it says for a kernel-only update, an NFS/SD/ram root,
or a same-version run that writes nothing -- and no longer promises a working
camera it cannot deliver.

reboot_system() now returns instead of exiting on the honoured path, so the
main flow's `exit 0` is reached and die() keeps its own `exit 1`. A deliberate
--no_reboot run used to report failure.

Verified on the lab hi3516ev300 (NOR, overlay root, squashfs on mtd3): a real
upgrade with -x prints the notice, flashes, overrides -x and comes back on the
new build with key auth intact; a forced kernel-only write with -x is honoured
and the box stays up, exit 0. 20 new behaviour cases and 12 drift assertions in
test_sysupgrade.sh, all of which fail against 1.0.55.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@widgetii
widgetii merged commit 0010f73 into master Jul 28, 2026
101 checks passed
@widgetii
widgetii deleted the sysupgrade-no-reboot-live-flash branch July 28, 2026 16:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

sysupgrade --no_reboot (-x) corrupts the running rootfs and locks out SSH after an on-device flash

1 participant