fix(hosted): name a suspended account instead of re-running sign-in

CHANGELOG.md

@@ -7,6 +7,15 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+### Fixed
+
+- A suspended account is named as such. When the server refuses a token with
+  `account_suspended`, the CLI prints its message — "Your account has been suspended.
+  Contact support@evalshift.dev." `evalshift login` with a stored token for that account
+  exits 1 with it instead of calling the token no longer valid and starting a browser
+  sign-in, and push auto-create shows it without the token-access hint. Service account
+  keys are not affected by a user's suspension.
+
 ## [1.3.0] - 2026-10-09
 
 ### Added

DOCS.md

@@ -794,7 +794,7 @@ evalshift logout
 ```
 
 - Credentials live in `~/.evalshift/credentials` (owner-only permissions). Precedence: CLI flags (`--host`/`--token`) > env (`EVALSHIFT_HOST`/`EVALSHIFT_TOKEN`) > credentials file. `--no-browser` prints the approval URL for remote shells.
-- Re-running `login` (without `--token`) while the stored token for that host still works reuses it — no new token is minted — and prints `already logged in as <email>`. To switch accounts, run `evalshift logout` first.
+- Re-running `login` (without `--token`) while the stored token for that host still works reuses it — no new token is minted — and prints `already logged in as <email>`. To switch accounts, run `evalshift logout` first. A suspended account gets the server's message instead (`Your account has been suspended. Contact support@evalshift.dev.`, exit 1) — signing in again cannot fix it, and a suspended account cannot approve a fresh browser login, which then waits until `--timeout`. Service account keys in the org keep working.
 - **`login` issues a personal token — don't use one in CI.** A personal token belongs to you and stops working when your membership does, which is correct on a workstation and fatal in a pipeline. For CI, mint a **service account key** in the web app (Settings → API tokens → Service accounts), scope it to the permissions the job needs, and pass it as `EVALSHIFT_TOKEN` from an encrypted secret instead of running `login` on the runner. Service accounts are org-owned, so the key survives the person who created it leaving.
 - Projects are `org/project` slugs — from `--project`, or the `project:` key in config. Missing projects are auto-created when permissions allow (`--no-create-project` disables; project-scoped tokens can't auto-create).
 - `evalshift bundle <run-id>` builds the upload artefact without uploading; `push --bundle <path>` uploads a prebuilt one. `run_bundle.json.gz` carries the manifest, per-example rows (inputs, both outputs, per-evaluator scores and the cost/latency deltas), each example's `traces` — the replay's own tool-call traces, one stream per model side with the ordered tool calls (names, arguments, call ids), round markers, any final text and refusal messages (no `model_call` events and no tool results; a stream over 256 KB keeps its leading events and is flagged `truncated`; imported agent traces from `traces import` stay local and are not uploaded) — the aggregate, `analysis`, the policy `decision` (whose `policy` field is the resolved `migration_policy` this run's verdict was computed under, or `null` when none is configured — see [Migration policy and CI gating](#migration-policy-and-ci-gating)), a run-level `economics` rollup (per-role calls, tokens, cost, latency), `methodology_notes`, the [insights](#run-insights) narrative, the evaluator config and the dataset snapshot. **`report.html` is not uploaded** — it is still written to the run directory for local viewing, and the hosted app renders the run from the data instead. Bundle bytes are deterministic: the same run always compresses identically. Pushes are idempotent on run id. On GitHub Actions, git metadata (`GITHUB_SHA`, branch refs) is baked into the bundle so the server can pair PR runs with base-branch baselines; elsewhere it comes from git, and `bundle` fails with `could not determine a valid 40-character git SHA; run inside git or set GITHUB_SHA` outside a git checkout. `push <run-id>` builds a bundle only when `run_bundle.json.gz` is missing and otherwise uploads the existing one as-is.

docs/hosted.md

@@ -25,6 +25,12 @@ Re-running `login` while the stored token for that host still works reuses it
 instead of minting a new one, and prints `already logged in as <email>`. To
 switch accounts, run `evalshift logout` first, then `evalshift login`.
 
+If your account has been suspended, `login` with its stored token prints the
+server's message — `Your account has been suspended. Contact support@evalshift.dev.` —
+and exits 1 instead of starting a new browser sign-in. A suspended account cannot
+approve a CLI login either, so a fresh `login` on a machine with no stored token
+waits until `--timeout` runs out. See [Troubleshooting](#troubleshooting).
+
 You can still paste an existing hosted API token manually:
 
 ```bash
@@ -373,5 +379,6 @@ credential file locally and repository secrets in CI.
 | `could not determine a valid 40-character git SHA` | `bundle` (or `push <run-id>` building a bundle) ran outside a git checkout, or in one with no commits, and `GITHUB_SHA` is unset. | Run inside a git repository with at least one commit, or set `GITHUB_SHA`. |
 | `project was not found` | The project does not exist and auto-create is disabled or not allowed. | Ask an owner to create it, use an org-scoped owner token, or enable auto-create. |
 | `cannot auto-create <slug> at <host>` | The message names the host it talked to and the server's status. Most often the host is not the one you meant: with no `--host` and no `EVALSHIFT_HOST`, an unset credentials file falls back to `https://api.evalshift.dev`, where your org does not exist. | Run `evalshift whoami` and check the host it prints. If it is wrong, `evalshift login --host <hosted-api-url>`. If the host is right and the status is 403, the token lacks org access — see [Project auto-create](#project-auto-create). |
+| `Your account has been suspended` | An EvalShift admin suspended the account this token belongs to. Its browser sessions and personal tokens are refused; service-account keys in your org keep working. | Contact support@evalshift.dev. Running `evalshift login` again does not help. |
 | `this run needs a paid plan` | The org's plan does not cover this push, its 30-day trial ended without a subscription, or the subscription has stopped paying. | Open the upgrade URL printed with the message; for the monthly quota you can also wait for the reset and push the same run id again. See [Plan limits](#plan-limits). |
 | `this run carries no migration policy` warning | No `migration_policy` is configured in `evalshift.yaml`, so the bundle has no `decision.policy`. | Add `migration_policy` to `evalshift.yaml` (see [Configuration](configuration.md#migration_policy)). Until then the gate has only whatever old web-app policy the project still has; with none, it reports `inconclusive` and never blocks the pull request unless the GitHub Action sets `require-policy: true`. |

llms-full.txt

@@ -220,7 +220,8 @@ evalshift login [--token es_...] [--host URL] [--no-browser] [--timeout SECS=900
   Device-code browser flow, or --token (verified via GET /me). Credentials stored at
   ~/.evalshift/credentials (owner-only perms). Without --token, a still-valid stored token for
   the host is reused (no new token minted; "already logged in as <email>"); a 401/403 falls
-  through to the browser flow. To switch accounts: `evalshift logout`, then `login`.
+  through to the browser flow, except a suspended account (403 account_suspended), which exits 1
+  with the server's message. To switch accounts: `evalshift logout`, then `login`.
   Issues a PERSONAL token: tied to your membership, dies with it. Correct for a workstation,
   wrong for CI. For CI mint a SERVICE ACCOUNT KEY in the hosted web app (Settings -> API
   tokens -> Service accounts): org-owned machine identity, never owner-equivalent (role is
@@ -1494,5 +1495,10 @@ Hosted 401 -> check evalshift whoami; precedence flags > env > credentials file;
 start with es_; a key past its rotation grace window authenticates as nobody.
 Hosted 403 `Permission denied: <key>` -> the token authenticated but its scopes (or its
 principal's role) don't cover that permission. Widen the scope or mint a key that holds it.
+Hosted 403 "Your account has been suspended" (code account_suspended) -> the user behind a
+personal token was suspended; whoami/push print that message and exit 1. login with a stored
+token does too, instead of starting a browser sign-in; a fresh login waits until --timeout (a
+suspended user cannot approve it). Re-login cannot fix it: contact support@evalshift.dev.
+Service account keys are unaffected.
 Owner-only keys a service account can never hold: project:create (auto-create -> use
 --no-create-project on a pre-created project).

src/evalshift_cli/cli/commands/login.py

@@ -10,7 +10,12 @@
 import typer
 from rich.console import Console
 
-from evalshift_cli.hosted.client import HostedClient, HostedError, HostedHTTPError
+from evalshift_cli.hosted.client import (
+    HostedAccountSuspendedError,
+    HostedClient,
+    HostedError,
+    HostedHTTPError,
+)
 from evalshift_cli.hosted.credentials import (
     CredentialsError,
     is_insecure_host,
@@ -96,8 +101,8 @@ def _reuse_stored_credentials(*, host: str, console: Console) -> bool:
     Every browser approval mints a fresh personal token on the server, and the CLI can
     only hold one at a time, so re-running ``login`` with a working credential would
     strand the previous token. A rejected token (revoked, or for another account) falls
-    through to the browser flow; any other failure is reported as-is because the browser
-    flow would hit the same wall.
+    through to the browser flow. A suspended account is reported and stops here. Any other
+    failure is reported as-is because the browser flow would hit the same wall.
     """
     try:
         stored = load_credentials()
@@ -107,6 +112,10 @@ def _reuse_stored_credentials(*, host: str, console: Console) -> bool:
         return False
     try:
         me = HostedClient(host=host, token=stored.token).me()
+    except HostedAccountSuspendedError as exc:
+        # A 403 like the rejected-token case below, but a fresh sign-in cannot fix it.
+        console.print(f"[red]✗[/red] {exc}")
+        raise typer.Exit(code=1) from exc
     except HostedHTTPError as exc:
         if exc.status_code in (401, 403):
             console.print("[yellow]![/yellow] stored token is no longer valid; signing in again")

src/evalshift_cli/hosted/client.py

@@ -37,6 +37,17 @@ def __init__(
         super().__init__(message)
 
 
+ACCOUNT_SUSPENDED = "account_suspended"
+
+
+class HostedAccountSuspendedError(HostedHTTPError):
+    """The account behind the token is suspended (HTTP 403, code ``account_suspended``).
+
+    The message is the server's sentence and names who to contact, so callers print it as-is.
+    It is not an expired token: signing in again cannot help.
+    """
+
+
 class HostedClient:
     """Typed-ish wrapper around the hosted API endpoints used by the CLI."""
 
@@ -163,7 +174,10 @@ def _request(self, method: str, path: str, **kwargs: Any) -> Any:
             ) from exc
         if response.status_code >= 400:
             code, message, details = _extract_error(response)
-            raise HostedHTTPError(response.status_code, message, code=code, details=details)
+            error_type = (
+                HostedAccountSuspendedError if code == ACCOUNT_SUSPENDED else HostedHTTPError
+            )
+            raise error_type(response.status_code, message, code=code, details=details)
         if not response.content:
             return None
         return response.json()

src/evalshift_cli/hosted/push.py

@@ -25,6 +25,7 @@
     validate_bundle,
 )
 from evalshift_cli.hosted.client import (
+    HostedAccountSuspendedError,
     HostedClient,
     HostedError,
     HostedHTTPError,
@@ -515,6 +516,9 @@ def _auto_create_project(client: HostedClient, *, project_slug: str) -> None:
         raise PushError(f"invalid project slug {project_slug!r}; expected org/project") from exc
     try:
         projects = client.list_projects(org_slug)
+    except HostedAccountSuspendedError as exc:
+        # The server's sentence already says what to do; an access hint would mislead.
+        raise PushError(str(exc)) from exc
     except HostedHTTPError as exc:
         raise PushError(
             f"cannot auto-create {project_slug!r} at {client.host}: "
@@ -528,6 +532,9 @@ def _auto_create_project(client: HostedClient, *, project_slug: str) -> None:
         return
     try:
         client.create_project(org_slug, slug=project, name=_name_from_slug(project))
+    except HostedAccountSuspendedError as exc:
+        # The server's sentence already says what to do; an access hint would mislead.
+        raise PushError(str(exc)) from exc
     except HostedHTTPError as exc:
         if exc.status_code == _PAYMENT_REQUIRED:
             # An expired org is read-only: creating a project is refused with the same 402 a

tests/unit/test_hosted_cli.py

@@ -14,7 +14,7 @@
 
 from evalshift_cli.cli.main import app
 from evalshift_cli.hosted.bundle import BUNDLE_FILENAME, BundleError, build_bundle
-from evalshift_cli.hosted.client import HostedNetworkError
+from evalshift_cli.hosted.client import HostedAccountSuspendedError, HostedNetworkError
 from evalshift_cli.hosted.credentials import (
     CredentialsError,
     load_credentials,
@@ -37,6 +37,8 @@
 
 runner = CliRunner()
 
+SUSPENDED_MESSAGE = "Your account has been suspended. Contact support@evalshift.dev."
+
 
 def _read_bundle(path: Path) -> dict[str, Any]:
     with gzip.open(path, "rt", encoding="utf-8") as fh:
@@ -384,6 +386,30 @@ def test_login_reauthenticates_when_stored_token_is_rejected(
     assert stored.token == "es_device_plaintext"
 
 
+def test_login_reports_a_suspended_account_instead_of_signing_in_again(
+    tmp_path: Path,
+    monkeypatch: pytest.MonkeyPatch,
+) -> None:
+    credentials_path = tmp_path / "credentials"
+    monkeypatch.setenv("EVALSHIFT_CREDENTIALS_PATH", str(credentials_path))
+    save_credentials("https://api.evalshift.test", "es_suspended", path=credentials_path)
+    fake_client, calls = _device_flow_client(
+        me_for_stored=HostedAccountSuspendedError(403, SUSPENDED_MESSAGE, code="account_suspended"),
+    )
+    monkeypatch.setattr("evalshift_cli.cli.commands.login.HostedClient", fake_client)
+    monkeypatch.setattr(
+        "evalshift_cli.cli.commands.login.webbrowser.open",
+        lambda _url: pytest.fail("a suspended account must not start a browser sign-in"),
+    )
+
+    result = runner.invoke(app, ["login", "--host", "https://api.evalshift.test"])
+
+    assert result.exit_code == 1
+    assert calls == []
+    assert SUSPENDED_MESSAGE in result.output
+    assert "no longer valid" not in result.output
+
+
 def test_login_ignores_stored_credentials_for_a_different_host(
     tmp_path: Path,
     monkeypatch: pytest.MonkeyPatch,
@@ -1047,6 +1073,28 @@ def test_auto_create_failure_names_the_host_and_the_servers_answer(
     assert "forbidden: org not visible to this token" in text
 
 
+def test_auto_create_repeats_a_suspension_without_the_permission_hint(
+    tmp_path: Path,
+    monkeypatch: pytest.MonkeyPatch,
+) -> None:
+    bundle_path = _build_bundle_for_push(tmp_path, monkeypatch)
+    fake = _fake_client(
+        responses=[{"raise_404": True}],
+        list_projects_error=HostedAccountSuspendedError(
+            403, SUSPENDED_MESSAGE, code="account_suspended"
+        ),
+        host="http://localhost:8080",
+    )
+    monkeypatch.setenv("EVALSHIFT_HOST", "http://localhost:8080")
+    monkeypatch.setenv("EVALSHIFT_TOKEN", "es_secret")
+    monkeypatch.setattr("evalshift_cli.hosted.push.HostedClient", lambda **_: fake)
+
+    with pytest.raises(PushError) as excinfo:
+        _push(bundle_path, tmp_path, create_project=True)
+
+    assert str(excinfo.value) == SUSPENDED_MESSAGE
+
+
 def test_create_project_failure_names_the_host_and_the_servers_answer(
     tmp_path: Path,
     monkeypatch: pytest.MonkeyPatch,

tests/unit/test_hosted_client.py

@@ -9,7 +9,7 @@
 import httpx
 import pytest
 
-from evalshift_cli.hosted.client import HostedClient, HostedHTTPError
+from evalshift_cli.hosted.client import HostedAccountSuspendedError, HostedClient, HostedHTTPError
 from tests.conftest import FakeHostedClient
 
 
@@ -345,3 +345,41 @@ def test_fake_hosted_client_mirrors_hosted_client() -> None:
         assert inspect.signature(getattr(FakeHostedClient, name)) == inspect.signature(real), (
             f"FakeHostedClient.{name}() no longer matches HostedClient.{name}()"
         )
+
+
+SUSPENDED_MESSAGE = "Your account has been suspended. Contact support@evalshift.dev."
+
+
+def test_account_suspended_maps_to_its_own_error(monkeypatch: pytest.MonkeyPatch) -> None:
+    client = _client_with_response(
+        monkeypatch,
+        httpx.Response(
+            403,
+            json={
+                "error": {
+                    "code": "account_suspended",
+                    "message": SUSPENDED_MESSAGE,
+                    "details": {"support_email": "support@evalshift.dev"},
+                }
+            },
+        ),
+    )
+
+    with pytest.raises(HostedAccountSuspendedError) as excinfo:
+        client.me()
+
+    assert isinstance(excinfo.value, HostedHTTPError)
+    assert excinfo.value.status_code == 403
+    assert str(excinfo.value) == SUSPENDED_MESSAGE
+
+
+def test_other_403s_stay_plain_http_errors(monkeypatch: pytest.MonkeyPatch) -> None:
+    client = _client_with_response(
+        monkeypatch,
+        httpx.Response(403, json={"error": {"code": "forbidden", "message": "nope"}}),
+    )
+
+    with pytest.raises(HostedHTTPError) as excinfo:
+        client.me()
+
+    assert not isinstance(excinfo.value, HostedAccountSuspendedError)