feat(observability): surface Kubernetes scheduling status while a sandbox pod is Pending/unscheduled #4746

User Story

As a user creating a sandbox on the Kubernetes driver, I want to see that the sandbox pod is waiting to be scheduled and why (insufficient resources, taints, GPU availability, quota), so that I can distinguish "still progressing" from "will never schedule", fix the cluster or cancel, and not stare at an unexplained spinner.

Problem Statement

With the Kubernetes driver, a sandbox pod can stay Pending indefinitely because the scheduler cannot place it (insufficient CPU/memory, node taints, no matching GPU node, resource quota). During that entire time:

• The gateway record stays at phase Provisioning, and sandbox list / sandbox get show nothing beyond that phase — no condition, no message, no hint that scheduling is the blocker.
• The sandbox create provisioning display never advances past the first step ("Requesting sandbox..."). The kube Scheduled event is what completes the "Sandbox allocated" step (attach_kube_progress_metadata in crates/openshell-driver-kubernetes/src/driver.rs, ~L2284, maps only Scheduled, Pulling, and Pulled reasons), so a pod that never schedules leaves the display on step 1 indefinitely.
• kube-scheduler emits FailedScheduling warning events carrying the actual reason (for example 0/3 nodes are available: 3 Insufficient memory), and the driver forwards them to the gateway as platform events with the raw reason and message. But they carry no progress metadata, so the CLI treats them as non-progress events: they do not extend the provisioning idle deadline and are not represented in the step state (is_provisioning_progress_event, crates/openshell-cli/src/commands/common.rs ~L510; fallthrough in crates/openshell-cli/src/run.rs ~L783). At best the most recent message transiently appears on the interactive spinner's detail line and is replaced by the next event.

A sandbox pending forever therefore conveys almost no information to the end user.

Impact / Why This Matters

• A permanently unschedulable sandbox is indistinguishable from slow provisioning. The user waits, eventually hits the CLI provisioning timeout, and still has no idea why — or whether retrying would help.
• The current workaround is cluster access: kubectl describe pod / kubectl get events to read the scheduler's reason. This requires Kubernetes knowledge and credentials that sandbox users of a hosted gateway do not have, and it defeats the purpose of the openshell CLI surface.
• After the create command exits (timeout or cancellation) or from another terminal, the scheduling blocker is not observable at all: the sandbox just reads Provisioning.

Proposed Design

• While the sandbox pod is Pending and unscheduled, sandbox create shows an explicit waiting-to-be-scheduled state on the provisioning display, with the scheduler's most recent reason as the detail (for example 0/3 nodes are available: 3 Insufficient memory).
• sandbox list / sandbox get (including --output json) expose the pending/unscheduled state and the latest scheduling reason for a sandbox in this condition, so the blocker is observable after the create command exits and by other users/operators.
• The information is derived from the scheduler's events and the pod's scheduling state, never from the workload.
• Once the pod schedules, the normal provisioning steps (sandbox allocated, image pull, start) resume.
• Whether the scheduling wait extends the create command's idle deadline is left open as an implementation decision; the requirement is only that the wait state and reason are visible while it lasts.

Acceptance Criteria

• On a cluster where the sandbox pod cannot schedule (for example, memory requests exceeding every node's capacity), sandbox create shows a waiting-to-be-scheduled state with the scheduler's reason instead of an unexplained step-1 spinner.
• The reason shown matches the message Kubernetes reports for the same pod (e.g. the FailedScheduling event seen via kubectl get events).
• sandbox list and sandbox get --output json expose the pending/unscheduled state and latest scheduling reason while the pod is unscheduled, on the Kubernetes driver.
• Once the pod schedules, provisioning completes normally and the wait indication clears.
• Behavior of the other compute drivers is unchanged.

Alternatives Considered

• Rely on the raw platform events already streamed over WatchSandbox. The FailedScheduling events reach the gateway with their reason and message, but only the interactive spinner's transient detail line ever shows them, they do not participate in progress step state, they do not extend the idle deadline, and nothing persists them to sandbox status. Insufficient for observability after the fact.
• Fold this into bug: sandbox create leaves stale resource on ImagePullFailed, blocking retry #2675's failure-event mapping. bug: sandbox create leaves stale resource on ImagePullFailed, blocking retry #2675 (which absorbed bug(k8s): sandbox create shows 'Image pulled' for the bootstrap image and hides the pull failure reason #4415) maps terminal pod Failed events such as image-pull failures to progress output and cleans up stale resources. An unscheduled pod is not a terminal failure — it may schedule once capacity frees — so it needs a distinct "waiting" representation rather than a failure reason. The mapping work is related but the semantics differ.
• A fixed provisioning timeout that errors out. Tells the user provisioning failed without saying why, and would kill pods that would have scheduled shortly after. It also compounds bug(kubernetes): sandbox stuck at Provisioning with replicas: 0 has no recovery path (start/stop both refuse) #4371, where a sandbox stuck at Provisioning has no recovery path short of delete.
• Tell users to use kubectl. Not available to users without cluster access, and it defeats the CLI/gateway observability surface.

Agent Investigation

• attach_kube_progress_metadata (crates/openshell-driver-kubernetes/src/driver.rs ~L2284, at 7a3afe7b1) maps only Scheduled, Pulling, and Pulled reasons to progress metadata. There is no handling for FailedScheduling events or for the pod Pending phase / PodScheduled: False condition anywhere in the tree (grep for FailedScheduling / PodScheduled across crates/ returns nothing).
• The Scheduled mapping completes PROGRESS_STEP_REQUESTING_SANDBOX ("Sandbox allocated") and activates the image-pull step, so an unscheduled pod leaves the CLI on the first step indefinitely.
• Kubernetes events are watched cluster-wide and forwarded to the gateway as DriverPlatformEvents with raw reason/message (map_kube_event_to_platform, ~L2232), including kube-scheduler's FailedScheduling warnings for the sandbox pod. But only events carrying progress metadata keys affect the CLI provisioning display and its idle deadline (is_provisioning_progress_event, crates/openshell-cli/src/commands/common.rs ~L510).
• The gateway derives no scheduling signal into SandboxStatus; the record remains phase Provisioning.
• Related open work: bug: sandbox create leaves stale resource on ImagePullFailed, blocking retry #2675 (image-pull failure reasons and stale-resource cleanup), feat(observability): expose the supervisor version and resolved image digests in sandbox status and events #4272 (supervisor version and resolved digests in status/events — extends the status surface, not failure or wait states), bug(kubernetes): sandbox stuck at Provisioning with replicas: 0 has no recovery path (start/stop both refuse) #4371 (stuck Provisioning with replicas: 0 and no pod — a different failure mode, focused on recovery rather than reporting).