fix(push): upgrade prompt for 402 after project auto-create; v6 plan copy

CHANGELOG.md

@@ -7,6 +7,17 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+### Fixed
+
+- `evalshift push` printed a traceback instead of the upgrade prompt when hosted EvalShift
+  refused the run with HTTP 402 right after auto-creating the project (over the monthly run
+  or concurrent-run limit, or a subscription that stopped paying). It now prints the
+  server's message and the billing link and exits 1. A 402 while auto-creating the
+  project — what an organization whose trial has ended gets — now shows the same prompt
+  instead of an access-permission hint.
+- `evalshift push` printed a traceback when the connection failed while auto-creating the
+  project; it now prints the error and exits 1.
+
 ### Changed
 
 - The EvalShift repositories moved from the `babaliauskas` GitHub account to
@@ -19,6 +30,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
   `evalshift/evalshift-action` steps as well as the old
   `babaliauskas/evalshift-action` name, matching owner and repository names
   case-insensitively as GitHub does.
+- Docs and the `init --ci` workflow comment describe hosted EvalShift's current plans: a
+  30-day Pro trial for every new account, then Pro per organization (the hosted Free and
+  Team plans are gone). Local runs stay free and unlimited.
 
 ## [1.2.1] - 2026-10-01
 

DOCS.md

@@ -760,15 +760,15 @@ The full field-by-field data contract lives in [docs/hosted.md — Privacy model
 
 ### Plan limits
 
-Local runs are always unlimited — plan limits apply only to what you push. When a push exceeds your org's plan (monthly runs, seats, retention) or the subscription has stopped paying, the server answers `402` and the CLI prints exactly what it said:
+Local runs are always unlimited — plan limits apply only to what you push. Every new hosted account gets a 30-day Pro trial (no card) that covers every org it creates; after that each org needs its own Pro subscription. When a push exceeds your org's plan (monthly runs, seats, concurrent runs), the org's trial ended without a subscription, or the subscription has stopped paying, the server answers `402` and the CLI prints exactly what it said:
 
 ```
 ✗ EvalShift: this run needs a paid plan.
-  Monthly run limit reached on the Free plan (50 of 50 runs used).
+  The free trial for this organization ended on 2026-11-02. Subscribe to Pro to push runs and keep the CI gate — existing runs stay readable.
   Upgrade: https://app.evalshift.dev/app/acme/settings/billing
 ```
 
-Exit code is 1 and nothing is uploaded. The CLI never decides entitlements itself and never retries a payment error — retrying would not change the answer. Your run and its `report.html` are already on disk under `.evalshift/runs/`; push it again after upgrading, or wait for the monthly reset. Transient upload failures (429, 5xx) *are* retried with backoff, which is a different thing entirely.
+Exit code is 1 and nothing is uploaded. The CLI never decides entitlements itself and never retries a payment error — retrying would not change the answer. Your run and its `report.html` are already on disk under `.evalshift/runs/`; push it again once the org is subscribed (or, for the monthly quota, after the reset). An expired org stays readable — its runs, diffs and baselines still load. Transient upload failures (429, 5xx) *are* retried with backoff, which is a different thing entirely.
 
 ---
 
@@ -777,7 +777,7 @@ Exit code is 1 and nothing is uploaded. The CLI never decides entitlements itsel
 `evalshift init --ci` scaffolds `.github/workflows/evalshift.yml` — a production-shaped, self-documenting workflow (the setup checklist lives in its header comment) with three jobs:
 
 - `discover` lists committed suites under `.evalshift/suites/*/golden.jsonl` — a suite added by `capture sync` is evaluated on the next run with no workflow edit, and a project with no suites yet skips green. Suites must be committed for CI to see them: keep `.evalshift/*` ignored but un-ignore `.evalshift/suites/` and `.evalshift/toolsets/`.
-- `eval <suite>` is a matrix job per suite (the action evaluates one suite per invocation) with `fail-on: policy` and `evalshift-version` pinned to the CLI that scaffolded the project. Each suite is selected with `suite-name:` — see [Selecting a suite: name, not path](#selecting-a-suite-name-not-path). `max-parallel` defaults to 1; raise it toward the hosted plan's in-flight ceiling (Free 1, Pro 5, Team 10). Only the first matrix job posts the PR comment — the comment marker is a constant, so multiple suites would overwrite one another.
+- `eval <suite>` is a matrix job per suite (the action evaluates one suite per invocation) with `fail-on: policy` and `evalshift-version` pinned to the CLI that scaffolded the project. Each suite is selected with `suite-name:` — see [Selecting a suite: name, not path](#selecting-a-suite-name-not-path). `max-parallel` defaults to 1; raise it toward the hosted plan's concurrent-run limit (Pro: 5; Enterprise: by contract). Only the first matrix job posts the PR comment — the comment marker is a constant, so multiple suites would overwrite one another.
 - `evalshift gate` is the single check to require in branch protection: it fails if any suite failed and passes when evaluation was skipped (fork PR, no suites, or `EVALSHIFT_TOKEN` not yet set). Don't require the per-suite jobs (dynamic names) or the `evalshift/regression` commit status (last writer wins across suites).
 
 Runs on pushes to main create the base-branch baselines PRs diff against, so the workflow cancels superseded runs on PRs only, never on main.

docs/github-action.md

@@ -30,8 +30,8 @@ checklist) with three jobs:
   to the CLI that scaffolded the project: the CLI that *reads* the config in CI must be at least as new
   as the CLI that *wrote* it locally (`extra: forbid` rejects newer keys), and
   the CLI warns when the pin falls behind (or runs ahead of the local CLI) — see [Pin drift](#pin-drift). `max-parallel` defaults to 1 —
-  raise it toward your hosted plan's in-flight-run ceiling (Free 1, Pro 5,
-  Team 10). The PR comment is posted by the first matrix job only: the
+  raise it toward your hosted plan's concurrent-run limit (Pro: 5;
+  Enterprise: by contract). The PR comment is posted by the first matrix job only: the
   comment marker is a constant, so multiple suites would overwrite one
   another's summary.
 - **`evalshift gate`** — the join job to require in branch protection. It

docs/hosted.md

@@ -237,20 +237,26 @@ the wrong one fails exactly like a permissions problem.
 
 Running locally is always unlimited. Plan limits apply only to what you push.
 
-When a push exceeds your organization's plan — monthly runs, seats, retention —
-or the subscription has stopped paying, the server refuses the push with HTTP
-402 and the CLI prints exactly what the server said:
+Every new hosted account gets a 30-day Pro trial (no card) that covers every
+org it creates; after that each org needs its own Pro subscription.
+
+When a push exceeds your organization's plan — monthly runs, seats, concurrent
+runs — the org's trial ended without a subscription, or the subscription has
+stopped paying, the server refuses the push with HTTP 402 and the CLI prints
+exactly what the server said:
 
 ```
 ✗ EvalShift: this run needs a paid plan.
-  Monthly run limit reached on the Free plan (50 of 50 runs used).
+  The free trial for this organization ended on 2026-11-02. Subscribe to Pro to push runs and keep the CI gate — existing runs stay readable.
   Upgrade: https://app.evalshift.dev/app/acme/settings/billing
 ```
 
 `push` exits 1 and nothing is uploaded. The CLI does not evaluate entitlements
 itself and does not retry a payment error — retrying cannot change the answer.
 Your run and its `report.html` stay on disk under `.evalshift/runs/`, so you can
-push the same run id again after upgrading, or after the monthly counter resets.
+push the same run id again once the org is subscribed (or, for the monthly
+quota, after the counter resets). An expired org stays readable — its runs,
+diffs and baselines still load.
 
 Transient upload failures (HTTP 429 and 5xx from object storage) are a separate
 case and *are* retried with exponential backoff.
@@ -355,5 +361,5 @@ credential file locally and repository secrets in CI.
 | `could not determine a valid 40-character git SHA` | `bundle` (or `push <run-id>` building a bundle) ran outside a git checkout, or in one with no commits, and `GITHUB_SHA` is unset. | Run inside a git repository with at least one commit, or set `GITHUB_SHA`. |
 | `project was not found` | The project does not exist and auto-create is disabled or not allowed. | Ask an owner to create it, use an org-scoped owner token, or enable auto-create. |
 | `cannot auto-create <slug> at <host>` | The message names the host it talked to and the server's status. Most often the host is not the one you meant: with no `--host` and no `EVALSHIFT_HOST`, an unset credentials file falls back to `https://api.evalshift.dev`, where your org does not exist. | Run `evalshift whoami` and check the host it prints. If it is wrong, `evalshift login --host <hosted-api-url>`. If the host is right and the status is 403, the token lacks org access — see [Project auto-create](#project-auto-create). |
-| `this run needs a paid plan` | The org's plan does not cover this push, or the subscription has stopped paying. | Open the upgrade URL printed with the message, or wait for the monthly reset and push the same run id again. See [Plan limits](#plan-limits). |
+| `this run needs a paid plan` | The org's plan does not cover this push, its 30-day trial ended without a subscription, or the subscription has stopped paying. | Open the upgrade URL printed with the message; for the monthly quota you can also wait for the reset and push the same run id again. See [Plan limits](#plan-limits). |
 | `this run carries no migration policy` warning | No `migration_policy` is configured in `evalshift.yaml`, so the bundle has no `decision.policy`. | Add `migration_policy` to `evalshift.yaml` (see [Configuration](configuration.md#migration_policy)). Until then the gate has only whatever old web-app policy the project still has; with none, it reports `inconclusive` and never blocks the pull request unless the GitHub Action sets `require-policy: true`. |

llms-full.txt

@@ -267,8 +267,9 @@ evalshift push [RUN_ID] [--bundle PATH] [--project] [--host] [--token]
   limit warning naming the server's 100 MB hard limit and uploads anyway (the hard limit is
   server-configurable, so the CLI quotes it rather than enforcing a stale copy).
   Plan limits: server answers 402 when the org's plan does not cover the push (monthly runs,
-  seats, retention) or the subscription stopped paying. CLI prints the server's sentence plus
-  `Upgrade: <url>` and exits 1; nothing uploaded, local run untouched. Never retried.
+  seats, concurrent runs), the org's 30-day trial ended without a subscription, or the
+  subscription stopped paying. CLI prints the server's sentence plus `Upgrade: <url>` and exits 1;
+  nothing uploaded, local run untouched. Never retried.
   Two policy notices, both before the success line: (1) a bundle with no decision.policy
   (no migration_policy configured) prints, before any HTTP call, "this run carries no
   migration policy; unless this project still has an old web-app policy, the hosted gate
@@ -1262,7 +1263,7 @@ evalshift compare --suite-name <suite> --to gemini-3.1-pro-preview --gate critic
 #   eval <suite> -> matrix job per suite (action evaluates ONE suite per invocation);
 #     fail-on: policy; evalshift-version pinned to the scaffolding CLI (extra="forbid" config
 #     from a newer CLI fails loudly on an older one -- reader >= writer; see CI pin drift); max-parallel: 1 by default — raise
-#     toward the hosted plan's in-flight ceiling (Free 1, Pro 5, Team 10); PR comment only
+#     toward the hosted plan's concurrent-run limit (Pro: 5; Enterprise: by contract); PR comment only
 #     from the first matrix job (constant comment marker, multiple suites would overwrite);
 #     provider key env matches init's --provider; skips on fork PRs (no secret access) and
 #     no-ops green with a notice while EVALSHIFT_TOKEN is unset.

src/evalshift_cli/cli/commands/_scaffold.py

@@ -231,8 +231,8 @@
     strategy:
       # One suite regressing must not cancel the others' results.
       fail-fast: false
-      # Hosted EvalShift refuses uploads past your org plan's in-flight run
-      # ceiling (Free 1, Pro 5, Team 10). Raise this toward your plan's limit
+      # Hosted EvalShift refuses uploads past your org's concurrent-run
+      # limit (Pro: 5; Enterprise: by contract). Raise this toward it
       # to evaluate suites in parallel.
       max-parallel: 1
       matrix:

src/evalshift_cli/hosted/bundle_manifest.schema.json

@@ -1018,7 +1018,7 @@
       },
       "GuardrailEvent": {
         "additionalProperties": false,
-        "description": "A guardrail or policy check. Imported agent traces only.",
+        "description": "A guardrail or policy check.\n\nReserved for a bring-your-own-agent trace; no current producer emits it \u2014\n``evalshift traces import`` writes such traces only to a local artifact,\nnot into a pushed bundle.",
         "properties": {
           "name": {
             "minLength": 1,
@@ -1405,7 +1405,7 @@
       },
       "RetrievalEvent": {
         "additionalProperties": false,
-        "description": "A retrieval step. Imported agent traces only.",
+        "description": "A retrieval step.\n\nReserved for a bring-your-own-agent trace; no current producer emits it \u2014\n``evalshift traces import`` writes such traces only to a local artifact,\nnot into a pushed bundle.",
         "properties": {
           "documents": {
             "items": {

src/evalshift_cli/hosted/push.py

@@ -108,6 +108,39 @@ def _upgrade_prompt(exc: HostedHTTPError) -> PushError:
     return PushError("\n".join(lines))
 
 
+def _initiate_after_create(
+    client: HostedClient, manifest: dict[str, Any], *, size_bytes: int
+) -> dict[str, Any]:
+    """The ``POST /runs`` retried after auto-creating the project, mapped like the first.
+
+    It runs inside the first attempt's ``except`` clause, so its own errors would skip the
+    sibling handlers and surface as a traceback. ``POST /runs`` answers 404 for a missing
+    project before any entitlement check, so a 402 for the monthly run or concurrent-run
+    limit, seat overage, or a subscription that stopped paying arrives on this retry, not
+    on the first call.
+
+    Args:
+        client: The hosted client the first attempt used.
+        manifest: The bundle manifest sent as the run's create payload.
+        size_bytes: Size of the bundle file that will be uploaded.
+
+    Returns:
+        The server's ``POST /runs`` response.
+
+    Raises:
+        PushError: On any hosted error — the upgrade prompt for a 402, the server's
+            message otherwise.
+    """
+    try:
+        return client.initiate_run(manifest, size_bytes=size_bytes)
+    except HostedHTTPError as exc:
+        if exc.status_code == _PAYMENT_REQUIRED:
+            raise _upgrade_prompt(exc) from exc
+        raise PushError(str(exc)) from exc
+    except (HostedNetworkError, HostedError) as exc:
+        raise PushError(str(exc)) from exc
+
+
 @dataclass(frozen=True, slots=True)
 class PushResult:
     """Outcome of a hosted push.
@@ -239,7 +272,7 @@ def push_bundle(
             raise PushError(str(exc)) from exc
         _auto_create_project(client, project_slug=str(manifest["project_slug"]))
         project_created = True
-        response = client.initiate_run(manifest, size_bytes=upload_size_bytes)
+        response = _initiate_after_create(client, manifest, size_bytes=upload_size_bytes)
     except (HostedNetworkError, HostedError) as exc:
         raise PushError(str(exc)) from exc
 
@@ -489,17 +522,25 @@ def _auto_create_project(client: HostedClient, *, project_slug: str) -> None:
             f"permission — check `evalshift whoami` reports this host and an org "
             f"named {org_slug!r}",
         ) from exc
+    except (HostedNetworkError, HostedError) as exc:
+        raise PushError(str(exc)) from exc
     if any(item.get("slug") == project for item in projects):
         return
     try:
         client.create_project(org_slug, slug=project, name=_name_from_slug(project))
     except HostedHTTPError as exc:
+        if exc.status_code == _PAYMENT_REQUIRED:
+            # An expired org is read-only: creating a project is refused with the same 402 a
+            # push gets, and it deserves the same prompt rather than an access-permission hint.
+            raise _upgrade_prompt(exc) from exc
         raise PushError(
             f"cannot auto-create {project_slug!r} at {client.host}: "
             f"{_server_said(exc)}. Creating a project needs owner access to "
             f"{org_slug!r}; a scoped service-account key cannot do it — create the "
             f"project by hand, or push with an owner token",
         ) from exc
+    except (HostedNetworkError, HostedError) as exc:
+        raise PushError(str(exc)) from exc
 
 
 def _server_said(exc: HostedHTTPError) -> str:

tests/conftest.py

@@ -40,7 +40,7 @@
 
 from evalshift_cli.evaluators.base import EvalRecord
 from evalshift_cli.hosted.bundle import BundleBuildResult, build_bundle
-from evalshift_cli.hosted.client import HostedHTTPError
+from evalshift_cli.hosted.client import HostedError, HostedHTTPError
 from evalshift_cli.runner.checkpoint import append_call, write_state
 from evalshift_cli.runner.models import Call, RunModels, RunState
 
@@ -264,8 +264,8 @@ class FakeHostedClient:
     projects: list[dict[str, Any]] | None = None
     created_project: dict[str, Any] | None = None
     finalize_error: HostedHTTPError | None = None
-    list_projects_error: HostedHTTPError | None = None
-    create_project_error: HostedHTTPError | None = None
+    list_projects_error: HostedError | None = None
+    create_project_error: HostedError | None = None
     host: str = "https://api.evalshift.test"
     on_finalize: Callable[[], None] | None = None
     initiate_calls: int = 0

tests/unit/test_docs_currency.py

@@ -199,3 +199,31 @@ def test_no_doc_shows_a_removed_top_level_key(name: str) -> None:
 
     hits = [m.group(0) for m in pattern.finditer(text)]
     assert not hits, f"{name} shows removed top-level key(s) {hits}; that config would not load"
+
+
+#: Exact substrings that described the v5 hosted plans. v6 (2026-10) has no hosted Free or
+#: Team plan; the CLI itself is still free. Retiring another plan claim? Append it here.
+RETIRED_PLAN_TERMS: tuple[str, ...] = ("Free 1, Pro 5, Team 10", "on the Free plan")
+
+_PLAN_COPY_FILES: tuple[str, ...] = tuple(
+    dict.fromkeys(
+        (
+            *PROSE_FILES,
+            "docs/hosted.md",
+            "docs/github-action.md",
+            "src/evalshift_cli/cli/commands/_scaffold.py",
+        )
+    )
+)
+
+
+def _whitespace_normalized(name: str) -> str:
+    """The file's text with every whitespace run collapsed to one space, so a retired
+    phrase hard-wrapped across lines (as Markdown and code comments are) still matches."""
+    return " ".join((REPO_ROOT / name).read_text(encoding="utf-8").split())
+
+
+@pytest.mark.parametrize("term", RETIRED_PLAN_TERMS)
+def test_copy_does_not_describe_the_retired_hosted_plans(term: str) -> None:
+    offenders = [name for name in _PLAN_COPY_FILES if term in _whitespace_normalized(name)]
+    assert offenders == [], f"{term!r} still appears in {offenders}"