Add budgeted live ECNU validation and harden the skill

.github/workflows/validate.yml

@@ -13,6 +13,17 @@ jobs:
     steps:
       - name: Check out repository
         uses: actions/checkout@v4
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: "3.11"
+      - name: Install compatibility-test dependencies
+        run: >-
+          python -m pip install
+          anthropic==0.125.0
+          httpx==0.28.1
+          langchain-openai==0.3.35
+          openai==2.48.0
       - name: Run repository validation
         run: python scripts/validate_skill.py
       - name: Run unit tests

.gitignore

@@ -7,3 +7,4 @@ __pycache__/
 *.py[cod]
 smoke-results*.json
 smoke-results/
+.live-artifacts/

AGENTS.md

@@ -1,85 +1,50 @@
 # ecnu-api repository guide
 
+## Purpose
+
 This repository is an Agent Skills package for the ECNU LLM Open Platform API.
 The repository content is the skill; there is no production application.
 
-## Scope
-
-- `SKILL.md` — concise activation and execution instructions
-- `references/api_reference.md` — documented endpoint contracts
-- `references/models.md` — models, aliases, credits, quotas, and deployment
-- `references/examples.md` — minimal, safe examples
-- `references/workflows.md` — implementation, debugging, retry, and test flows
-- `references/known_deviations.md` — dated live observations only
-- `scripts/smoke_test.py` — opt-in live structural checks
-- `scripts/validate_skill.py` — deterministic repository validation
-- `tests/` — offline tests for helper behavior
+## File routing
 
-Do not write outside the repository unless the user explicitly asks to install
-or synchronize the skill into a client-specific directory.
+- `SKILL.md` is the concise task entry point.
+- `references/api_reference.md` contains documented endpoint contracts.
+- `references/models.md` contains models, aliases, credits, and quotas.
+- `references/examples.md` contains minimal safe examples.
+- `references/workflows.md` contains executable integration and test flows.
+- `references/known_deviations.md` contains dated live observations only.
+- `scripts/smoke_test.py` performs opt-in live structural checks.
+- `scripts/validate_skill.py` and `tests/` provide offline validation.
 
-## Source precedence
+## Editing rules
 
-When ECNU documentation pages disagree:
+- Treat current official ECNU documentation as the documented contract.
+- Keep documented facts, live observations, application policy, and unverified
+  claims distinct.
+- Do not invent undocumented fields, limits, model capabilities, or prices.
+- Put point-in-time behavior only in `known_deviations.md` with dated evidence.
+- Keep examples sequential, timeout-bounded, and environment-key based.
+- Do not modify files outside this repository unless the user explicitly asks.
 
-1. Use the current model page for model identity, context figures, aliases, and
-   capability labels.
-2. Use the endpoint page for JSON fields, types, and endpoint-specific limits.
-3. Use the quota page for current prices and quota periods.
-4. Use release notes to establish when a change occurred.
-5. Use `GET /models` for runtime visibility, not as the sole source of
-   capability truth.
-6. Keep live probes in `known_deviations.md`; never let a single observation
-   silently override a documented contract.
+## Validation
 
-## Change workflow
-
-1. Work on a branch.
-2. Identify the exact official pages affected by the change.
-3. Update only the relevant focused reference.
-4. If live testing is needed, use `ECNU_API_KEY` from the environment.
-5. Never paste or persist a real key in a file, command example, report, issue,
-   commit, or pull request.
-6. Run:
+Run before committing:
 
 ```bash
-python scripts/validate_skill.py
-python -m unittest discover -s tests -v
+python3 scripts/validate_skill.py
+python3 -m unittest discover -s tests -v
+python3 -m compileall scripts tests
 uvx --from skills-ref agentskills validate .
 ```
 
-7. Review the diff for secrets, machine-specific paths, duplicated guidance,
-   undocumented request fields, and accidental billable calls.
-8. Summarize whether each changed claim is documented, observed, or unverified.
-
-## Live verification rules
-
-- The default smoke test performs model-list checks only.
-- Chat, embedding, and Anthropic probes require `--low-cost` or `--anthropic`.
-- Do not add image generation to an automatic or CI smoke test.
-- Do not blindly retry image, TTS, or any other billable request after an
-  ambiguous network failure.
-- Record SDK or Python version, account type, date, endpoint, status, content
-  type, and structural result.
-- Sanitize reports before sharing or committing them.
-- Update the date in `known_deviations.md` only when the behavior was actually
-  reproduced.
-
-## Content conventions
-
-- Write documentation in English; retain official Chinese UI labels where
-  needed.
-- Prefer imperative, stepwise instructions over broad prose.
-- Keep `SKILL.md` below the Agent Skills recommended size and route detail to
-  focused references.
-- Never describe `skills-ref` validation as an API correctness test.
-- Never claim undocumented limits.
-- Never describe an output dimension as a request parameter unless ECNU
-  documents it.
-- Use environment variables in every credential example.
+Review `git diff --check` and scan tracked content for secrets and personal paths.
 
-## Current state
+## Safety
 
-The reference content is aligned with the ECNU documentation and repository
-observations available on 2026-08-22. The live deviations remain dated
-2026-08-21 until a new authenticated run reproduces or supersedes them.
+- Read live credentials only from `ECNU_API_KEY`; never accept a CLI key.
+- Never commit keys, Authorization values, private inputs, raw responses,
+  generated media, one-time URLs, or full reasoning content.
+- Keep API calls serial and enforce the declared credit ceiling.
+- Do not automatically retry POST requests after ambiguous transport failures.
+- Record only sanitized response structure and allowlisted diagnostic headers.
+- Do not update an observation date unless the behavior was reproduced.

README.md

@@ -46,6 +46,7 @@ ecnu-api/
 │   ├── smoke_test.py
 │   └── validate_skill.py
 ├── tests/
+│   ├── test_repository_contracts.py
 │   └── test_smoke_test.py
 └── .github/workflows/validate.yml
 ```
@@ -74,41 +75,64 @@ export ECNU_API_KEY="your-api-key"
 A key pasted into a chat or public location should be revoked or rotated after
 testing.
 
-## Reproducible smoke tests
+## Reproducible live validation
 
-The default profile performs model-list checks and does not send chat,
-embedding, Anthropic, image, or TTS POST requests:
+The runner reads only `ECNU_API_KEY`, sends requests serially, and does not
+retry POST requests. Select the smallest profile that answers the question:
+
+| Profile | Scope |
+|---|---|
+| `auth` | Service status plus valid, invalid, and missing-token model discovery; no billable POST requests. This is the default. |
+| `core` | Low-cost Chat Completions, Responses, embeddings, rerank, vision, structured output, error-shape, OpenAI SDK, and LangChain probes. |
+| `compatibility` | Responses vision and Anthropic-compatible models, aliases, effort controls, long-context suffix behavior, vision, and SDK probes. |
+| `billable` | Fixed-price TTS and one documented image-generation probe, subject to the credit ceiling. |
+| `all` | The union of all four profiles; later billable cases are skipped when the ceiling is reached. |
+
+Examples:
 
 ```bash
-python scripts/smoke_test.py
+python3 scripts/smoke_test.py --profile auth --max-credits 0 --output .live-artifacts/auth.json
+python3 scripts/smoke_test.py --profile core --max-credits 50 --output .live-artifacts/core.json
+python3 scripts/smoke_test.py --profile compatibility --max-credits 50 --output .live-artifacts/compatibility.json
+python3 scripts/smoke_test.py --profile billable --max-credits 50 --output .live-artifacts/billable.json
+python3 scripts/smoke_test.py --profile all --max-credits 50 --output .live-artifacts/all.json
 ```
 
-Low-cost POST probes are explicit:
+`--max-credits` is a conservative planned-cost gate, defaulting to 50. The
+runner reserves each case's estimate before sending it and skips a case that
+would exceed the ceiling. The estimate is not proof of the service's actual
+debit. Recheck the official quota and pricing page before a live run.
+
+Use `--case` to rerun only named cases within the selected profile; repeat the
+flag to select more than one:
 
 ```bash
-python scripts/smoke_test.py --low-cost --anthropic \
-  --account-type personal-token \
-  --output smoke-results.json
+python3 scripts/smoke_test.py --profile core --case openai_sdk_chat \
+  --max-credits 1 --output .live-artifacts/openai-sdk-chat.json
 ```
 
-The report contains statuses and structural summaries. It does not print the
-API key or successful model content. Image generation is intentionally absent
-from the automated smoke test because it is comparatively expensive and a
-retry after an ambiguous failure could duplicate charges.
+Keep reports under `.live-artifacts/`, which is Git-ignored. Reports contain
+statuses and structural summaries, not the API key, generated content,
+reasoning text, media, or one-time URLs.
+
+Selected SDK probes on 2026-08-23 passed with OpenAI Python SDK 2.48.0,
+Anthropic Python SDK 0.125.0, `langchain-openai` 0.3.35, and `httpx` 0.28.1.
+This is dated, point-in-time evidence, not a blanket compatibility guarantee;
+see `references/known_deviations.md` for the observed scope and divergences.
 
 ## Validate the skill
 
 Run deterministic repository checks and unit tests:
 
 ```bash
-python scripts/validate_skill.py
-python -m unittest discover -s tests -v
+python3 scripts/validate_skill.py
+python3 -m unittest discover -s tests -v
 ```
 
 Run the Agent Skills reference validator separately:
 
 ```bash
-uvx --from skills-ref agentskills validate .
+uvx --from skills-ref agentskills validate "$PWD"
 ```
 
 The reference validator checks format and naming conventions; it does not
@@ -125,6 +149,20 @@ contract matches live behavior.
 - Do not add local absolute paths or machine-specific deployment instructions.
 - Run repository validation before opening a pull request.
 
+## Revalidate after ECNU platform updates
+
+An ECNU release, model rollout, endpoint change, quota change, or announced fix
+is a reason to consider a new targeted validation; it is not evidence that an
+active deviation has been resolved. Review the updated official contract,
+recalculate the credit allowance, and run only the affected `--case` probes
+serially with fresh sanitized evidence. Billable TTS or image probes require
+new account-owner authorization and must never run automatically.
+
+Update an observation date or mark a deviation `resolved` only after the same
+behavior has been exercised again with the current runner. Preserve the prior
+entry when the new run is inconclusive, and record both the changed contract
+and the new observed result when the platform update changes expectations.
+
 ## Official documentation
 
 API details can change. Verify production-critical behavior against the current

SKILL.md

@@ -15,7 +15,7 @@ description: >
 Use this skill to turn ECNU API documentation into a safe, verifiable
 integration. The current official ECNU developer documentation is the
 authority for documented contracts. Keep documented facts, live observations,
-and application policy separate.
+application policy, and unverified claims separate.
 
 ## Core rules
 
@@ -131,55 +131,89 @@ names as compatibility aliases.
 
 Before a real request:
 
-- use an environment variable such as `ECNU_API_KEY`;
+- recheck the current official quota and pricing page, then calculate a
+  conservative planned cost from those documented prices;
+- use 50 credits as the default ceiling and do not run a larger plan without
+  separate user authorization;
+- use only `ECNU_API_KEY` from the environment; never accept a key through a
+  command-line argument;
 - remove secrets and unnecessary personal or confidential data;
 - confirm the user intended to send the supplied content to ECNU;
-- state when image generation, TTS, or other calls may consume credits;
-- never blindly retry a billable request after an ambiguous timeout.
+- execute requests serially; and
+- never retry a POST after an ambiguous timeout or connection failure.
+
+If the full plan exceeds 50 credits, preserve the core dialog, embedding,
+rerank, compatibility, and error checks; prefer one TTS PCM check; run at most
+one documented image-generation case; and skip expanded voices and
+undocumented model probes.
 
 If a key has already been pasted into a chat or public location, recommend
 revoking or rotating it after testing.
 
 ### 7. Execute and verify
 
-For reproducible checks, run:
-
-```bash
-python scripts/smoke_test.py
-```
-
-This default profile performs model-list checks only. Low-cost POST probes are
-opt-in:
+Use the smallest profile that answers the question. Sanitized reports belong
+under the ignored `.live-artifacts/` directory:
 
 ```bash
-python scripts/smoke_test.py --low-cost --anthropic
+python3 scripts/smoke_test.py --profile auth --max-credits 0 --output .live-artifacts/auth.json
+python3 scripts/smoke_test.py --profile core --max-credits 50 --output .live-artifacts/core.json
+python3 scripts/smoke_test.py --profile compatibility --max-credits 50 --output .live-artifacts/compatibility.json
+python3 scripts/smoke_test.py --profile billable --max-credits 50 --output .live-artifacts/billable.json
+python3 scripts/smoke_test.py --profile all --max-credits 50 --output .live-artifacts/all.json
 ```
 
-The script reads `ECNU_API_KEY`, redacts key-shaped strings, and emits a
-structural JSON report rather than model output.
+The default profile is `auth`. The runner reads `ECNU_API_KEY`, executes
+serially with POST retries disabled, reserves estimated credits before each
+request, skips cases that would exceed the ceiling, and emits response
+structure rather than generated content. A credit estimate is not proof of the
+service's actual debit.
 
 ### 8. Report provenance
 
 Label important conclusions as one of:
 
-- **Documented** — supported by the current official ECNU documentation.
-- **Observed** — reproduced against the live service at a stated date.
-- **Unverified** — inferred, historical, or not reproducible in the current
+- **`documented`** — supported by the current official ECNU documentation.
+- **`observed`** — reproduced against the live service at a stated date.
+- **`application-policy`** — a local safety, cost, or reliability constraint;
+  not an ECNU platform guarantee.
+- **`unverified`** — inferred, historical, or not reproducible in the current
   environment.
 
 Do not silently promote an observed deviation into a documented guarantee.
 
+### 9. Finish repository work
+
+When repository files changed, finish with offline, format, and secret checks:
+
+```bash
+python3 scripts/validate_skill.py
+python3 -m unittest discover -s tests -v
+python3 -m compileall scripts tests
+uvx --from skills-ref agentskills validate "$PWD"
+git grep -nE 'sk-[A-Za-z0-9_-]{16,}'
+git grep -nE 'Authorization:[[:space:]]*Bearer[[:space:]]+[^<"$]'
+git diff --check
+```
+
+Review every secret-scan match; no tracked literal credential may remain.
+Variable-based test fixtures may match the coarse Bearer expression. If `uvx`
+is not available, report that validator as not run rather than treating it as
+live API evidence.
+
 ## High-value gotchas
 
 - `GET /models` is runtime discovery, not a reliable authentication test.
 - A model appearing in `/models` does not prove that a capability is usable.
 - Use `ecnu-max[1m]` only when an Anthropic tool requires the suffix to
-  advertise long context. Fall back to plain `ecnu-max` if the suffix returns
-  an authentication or metadata error.
+  advertise long context. Consider plain `ecnu-max` only when the same
+  credential already succeeds with that model, the suffixed request returns
+  the observed suffix-specific `401` metadata error, and the caller accepts
+  the shorter advertised context.
 - TTS errors may not match the documented JSON shape; preserve the HTTP status,
   content type, and a bounded redacted body sample.
-- Do not depend on optional PCM metadata headers without checking them at
-  runtime.
+- Do not assume the documented PCM metadata headers are present; check them at
+  runtime and configure the format explicitly when they are absent.
 - `422` means request validation failed; inspect `detail`.
 - `429` may represent quota exhaustion, rate control, or short-term service
   protection. Stop parallel retries and inspect credits first.