Skip to content

feat(sdk): add typed UI contracts - #600

Open
solvemproblr wants to merge 1 commit into
mainfrom
asa/sdk-ui-contracts
Open

feat(sdk): add typed UI contracts#600
solvemproblr wants to merge 1 commit into
mainfrom
asa/sdk-ui-contracts

Conversation

@solvemproblr

@solvemproblr solvemproblr commented Aug 19, 2026

Copy link
Copy Markdown
Contributor

Idea

Make the SDK the source of truth for platform-specific task argument editors and grader result renderers.

Issue

Environment authors currently hand-write x-hud-hint JSON Schema metadata for supported argument editors. Grader rendering is inferred from BashGrader / LLMJudgeGrader names or private info._hud metadata, so custom graders and name overrides lose the intended presentation.

Solution

  • Add typed argument contracts: PromptArg, DataFileArg, DataFilesArg, and GradingArg.
  • Add extensible DataFileRef and GradingCriterion Pydantic models and validate wire values into declared models.
  • Add GraderKind to SubScore and EvaluationResult; stamp it from Grader.grade() independently of display names.
  • Mark BashGrader as bash and LLMJudgeGrader as rubric.
  • Document and test the public contracts.

Impact

Environment code can import stable SDK types instead of duplicating UI metadata. Custom graders can opt into a supported renderer without adopting a built-in class name. A separate frontend change can consume the new kind field and remove the name/metadata fallback.

Validation

  • ruff format --check hud
  • ruff check hud
  • targeted ty check on changed runtime modules
  • 1048 passed, 17 deselected in the full available non-robot test suite
  • focused environment/grader suite: 84 passed

The three robot test modules could not be collected in this worktree because the optional openpi_client dependency is not installed.


Note

Medium Risk
Changes task arg coercion on the server (behavior shift for some wire shapes) and extends the public grading wire format with kind; both affect platform clients and environment authors but are backward-compatible defaults for existing graders.

Overview
Introduces SDK types so task manifests and UIs can rely on stable contracts instead of hand-written x-hud-hint metadata and grader name heuristics.

Task arguments: New PromptArg, DataFileArg, DataFilesArg, and GradingArg annotations publish editor hints on the published JSON Schema; DataFileRef is the base model for file references. These types are re-exported from hud.environment. The environment server now coerces wire args with Pydantic (validate_python first, then validate_json for legacy string payloads), so tasks receive validated models at runtime.

Grading: GraderKind (generic, bash, rubric) is added on SubScore and stamped from Grader.grade() independently of display name. Built-in graders set bash and rubric respectively. Grade frames on the wire include kind on each subscore node.

Docs and tests cover manifest hints, end-to-end typed args, and kind serialization.

Reviewed by Cursor Bugbot for commit 2c8dd0d. Bugbot is set up for automated code reviews on this repo. Configure here.

@solvemproblr
solvemproblr marked this pull request as ready for review August 19, 2026 22:22

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit f783e68. Configure here.

Comment thread hud/environment/server.py
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant