Skip to content

feat: structured-data template gallery evidence and cited comparison dimensions #371

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
abrichr merged 2 commits into from
Aug 22, 2026
+2,474 −37
Merged

feat: structured-data template gallery evidence and cited comparison dimensions #371

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
0134cb2
feat: add structured-data template gallery evidence and cited compari…
abrichr Aug 22, 2026
9d5bfde
Merge branch 'main' into growth/template-compare-generators
abrichr Aug 22, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
163 changes: 163 additions & 0 deletions data/compare/browser-agents.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,163 @@
{
"slug": "browser-agents",
"competitor": "browser-use agents (Workflow Use and Browser Use)",
"positioningSummary":
"Workflow Use and Browser Use are the fastest-moving open-source stack for web tasks, and their deterministic-rerun direction validates demonstration-based authoring. OpenAdapt targets repeated consequential workflows that cross beyond the browser and must end in an independently verified business result.",
"dimensions": [
{
"id": "determinism-on-drift",
"label": "Determinism when interfaces drift",
"openadapt":
"Compiled replay resolves its target from retained evidence at run time; drift it was never shown halts instead of guessing, and repair is governed rather than improvised.",
"them":
"Workflow Use records browser interactions into deterministic workflows with variables; when a step fails it falls back to the Browser Use agent - a model re-reasons the step. The project describes itself as in very early development and not recommended for production.",
"sources": [
{
"label": "Workflow Use README: deterministic workflows with agent fallback",
"url": "https://github.com/browser-use/workflow-use/blob/main/README.md"
}
]
},
{
"id": "cost-per-run",
"label": "Cost per run",
"openadapt":
"Healthy runs are local and make zero model calls at $0 model cost on the MIT runtime; model spend is reserved for compilation and reviewable repair.",
"them":
"Browser Use Cloud advertises deterministic rerun of a cached script with no LLM at up to 99% lower cost, agent runs measured at $0.17 per solved task on its internal benchmark, hosted model tokens from $0.24/1M input and $1.44/1M output, plus $0.02 per browser-hour.",
"sources": [
{
"label": "Browser Use developer index: pricing and benchmarks",
"url": "https://browser-use.com/"
},
{
"label": "Browser Use docs: deterministic rerun",
"url": "https://docs.browser-use.com/cloud/agent/cache-script"
},
{
"label": "Frappe Lending benchmark: $0 model cost",
"url": "https://github.com/OpenAdaptAI/openadapt-flow/tree/main/benchmark/frappe_lending"
}
]
},
{
"id": "verification-of-effects",
"label": "Verification of business effects",
"openadapt":
"Success is judged out of band against the system of record (SQL, REST readback, table-delta audit, file arrival); the acting session cannot certify its own result.",
"them":
"Both stacks judge success in band: the session that acted reports completion. Workflow Use offers progress events, recordings, and observability; no independent read-back of a business system of record is part of the published loop.",
"sources": [
{
"label": "Browser Use developer index: recordings and observability",
"url": "https://browser-use.com/"
}
]
},
{
"id": "halting-behavior",
"label": "Halting behavior",
"openadapt":
"Ambiguity or wrong identity halts before acting; a refuted effect ends the run HALTED with evidence preserved for reconciliation - never a silent third state.",
"them":
"Failure handling is retry-or-fallback oriented: Workflow Use's advertised roadmap item is improving LLM fallback when a step fails, which hands control back to the model rather than stopping for review.",
"sources": [
{
"label": "Workflow Use README: roadmap and fallback",
"url": "https://github.com/browser-use/workflow-use/blob/main/README.md"
}
]
},
{
"id": "data-locality",
"label": "Data locality",
"openadapt":
"The runtime is MIT licensed and local-first; screenshots-may-leave-box is an explicit per-run flag observed false in the published demo evidence pack.",
"them":
"The open-source libraries can run locally against your own Chromium, but the managed path routes tasks, tokens, and browser time through Browser Use Cloud infrastructure.",
"sources": [
{
"label": "Browser Use developer index: cloud vs open source",
"url": "https://browser-use.com/"
},
{
"label": "mockmed-triage-v3 pack: egress observed false",
"url": "https://github.com/OpenAdaptAI/openadapt-flow/blob/main/public-demo/evidence-packs/mockmed-triage-v3/manifest.json"
}
]
},
{
"id": "scope",
"label": "Scope: browser, desktop, RDP/Citrix",
"openadapt":
"Browser in production today plus native desktop and external-lane remote execution for managed RDP/Citrix estates, each surface carrying published acceptance evidence.",
"them":
"Web only: Workflow Use records browser sessions and Browser Use drives managed or local Chromium; desktop applications and remote desktop surfaces are outside the published scope.",
"sources": [
{
"label": "Workflow Use README: browser recording focus",
"url": "https://github.com/browser-use/workflow-use/blob/main/README.md"
},
{
"label": "OpenAdapt qualification evidence per surface",
"url": "https://docs.openadapt.ai/get-started/what-works-today/"
}
]
}
],
"strengths": [
{
"text":
"Fastest setup for web tasks: point a Browser Use agent at a URL and useful behavior often emerges in minutes; Workflow Use converts one recording into a reusable workflow file.",
"source": null
},
{
"text":
"Deterministic rerun without an LLM: Browser Use Cloud advertises cached-script reruns at up to 99% lower cost than an agent run.",
"source": {
"label": "Browser Use docs: deterministic rerun",
"url": "https://docs.browser-use.com/cloud/agent/cache-script"
}
},
{
"text":
"Published benchmark results: 82% of 106 internal Bench Hard tasks solved at $0.17 per solved task and 98% across Online-Mind2Web.",
"source": {
"label": "Browser Use developer index: benchmarks",
"url": "https://browser-use.com/"
}
},
{
"text":
"Active open-source ecosystem iterating quickly, with managed cloud-browser infrastructure ($0.02/browser-hour) if you do not want to run browsers yourself.",
"source": {
"label": "Browser Use developer index: browser infrastructure",
"url": "https://browser-use.com/"
}
}
],
"faq": [
{
"question": "Is Workflow Use production ready?",
"answer":
"Not by its own description: the README states the project is in very early development and recommends against production use. It is a strong signal of where demonstration-based authoring is heading.",
"sources": [
{
"label": "Workflow Use README",
"url": "https://github.com/browser-use/workflow-use/blob/main/README.md"
}
]
},
{
"question": "What does OpenAdapt add over deterministic browser replay?",
"answer":
"Out-of-band verification against the system of record, explicit VERIFIED-or-HALTED outcomes with preserved evidence, identity checks before consequential actions, and execution beyond the browser - native desktop and zero-install remote lanes - with published qualification evidence per surface.",
"sources": [
{
"label": "OpenAdapt qualification evidence",
"url": "https://docs.openadapt.ai/get-started/what-works-today/"
}
]
}
]
}
156 changes: 156 additions & 0 deletions data/compare/computer-use-agents.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,156 @@
{
"slug": "computer-use-agents",
"competitor": "computer-use agents (OpenAI and Anthropic)",
"positioningSummary":
"Computer-use models from OpenAI and Anthropic point a frontier model at the screen and reason their way through a task; that flexibility is real and improving fast. OpenAdapt compiles a demonstration once and replays it deterministically, reserving models for compilation and reviewable repair.",
"dimensions": [
{
"id": "determinism-on-drift",
"label": "Determinism when interfaces drift",
"openadapt":
"Healthy runs replay the same compiled steps deterministically with zero model calls; drift outside what was demonstrated halts for a governed decision.",
"them":
"Every run re-reasons from screenshots: OpenAI's loop sends a screenshot back after each action batch so the model can plan the next step, which adapts to novelty but makes each run non-deterministic by construction.",
"sources": [
{
"label": "OpenAI computer use guide: screenshot-action loop",
"url": "https://platform.openai.com/docs/guides/tools-computer-use"
}
]
},
{
"id": "cost-per-run",
"label": "Cost per run",
"openadapt":
"$0 model cost per healthy run on the MIT runtime - published trials across MockMed triage (18 trials), Frappe Lending (6/6), and openIMIS eligibility (6 runs) all recorded zero model calls.",
"them":
"Metered per model turn: screenshots are billed as image inputs every step. Claude's computer-use toolset definition alone adds about 4,500 input tokens to a request before any screenshot; OpenAI recommends original-detail screenshots each turn for click accuracy.",
"sources": [
{
"label": "Claude computer use tool: pricing",
"url": "https://platform.claude.com/docs/en/agents-and-tools/tool-use/computer-use-tool"
},
{
"label": "OpenAI computer use guide: screenshot detail guidance",
"url": "https://platform.openai.com/docs/guides/tools-computer-use"
},
{
"label": "OpenAdapt published trials with 0 model calls",
"url": "https://github.com/OpenAdaptAI/openadapt-flow/blob/main/public-demo/evidence-packs/mockmed-triage-v3/manifest.json"
}
]
},
{
"id": "verification-of-effects",
"label": "Verification of business effects",
"openadapt":
"Every consequential run ends VERIFIED or HALTED based on an independent check of the system of record, with silent incorrect success counted as a tracked failure metric in published evidence.",
"them":
"The acting model reports task completion from what it sees; providers direct developers to keep a human in the loop for high-impact actions rather than providing an out-of-band business-effect oracle.",
"sources": [
{
"label": "OpenAI computer use guide: human-in-the-loop safeguards",
"url": "https://platform.openai.com/docs/guides/tools-computer-use"
}
]
},
{
"id": "halting-behavior",
"label": "Halting behavior",
"openadapt":
"Halting is a designed outcome: ambiguity, wrong identity, or a refuted effect stops the run with preserved evidence instead of proceeding or retrying blindly.",
"them":
"The model keeps attempting until it believes the task is done or its budget is exhausted; stopping criteria are prompt- and harness-level, not contract-level.",
"sources": [
{
"label": "OpenAI computer use guide: agent loop semantics",
"url": "https://platform.openai.com/docs/guides/tools-computer-use"
}
]
},
{
"id": "data-locality",
"label": "Data locality",
"openadapt":
"Runs execute locally inside your boundary; egress is an explicit observed flag (false in the published demo pack), and Claude computer use is documented as ZDR eligible when your application controls storage.",
"them":
"Screen content leaves the machine to the model provider on every step: OpenAI's reference loop ships screenshots to the API each action batch, and Anthropic bills those screenshots as image input under standard API data handling.",
"sources": [
{
"label": "Claude computer use tool: data retention and ZDR",
"url": "https://platform.claude.com/docs/en/agents-and-tools/tool-use/computer-use-tool"
},
{
"label": "mockmed-triage-v3 pack: off-box egress observed false",
"url": "https://github.com/OpenAdaptAI/openadapt-flow/blob/main/public-demo/evidence-packs/mockmed-triage-v3/artifacts/cases/representative/trial-01/outcome.json"
}
]
},
{
"id": "scope",
"label": "Scope: browser, desktop, RDP/Citrix",
"openadapt":
"Browser in production today plus native desktop and external-lane remote execution into managed RDP/Citrix estates, with published acceptance evidence per surface.",
"them":
"Any UI a screenshot can describe, including desktops inside VMs or containers the harness controls; consumer agents such as Operator extend the same approach to end users.",
"sources": [
{
"label": "OpenAI computer use guide: browser and VM harnesses",
"url": "https://platform.openai.com/docs/guides/tools-computer-use"
}
]
}
],
"strengths": [
{
"text":
"Genuine flexibility on novel, one-off, or loosely specified tasks with no authoring step at all.",
"source": null
},
{
"text":
"They generalize across unfamiliar interfaces, recovering from situations nobody anticipated in advance, and improve with every model generation without changes to your workflow definitions.",
"source": null
},
{
"text":
"A plain-language instruction is the whole interface; OpenAI documents harness shapes from built-in loops to code-execution environments mixing visual and programmatic interaction.",
"source": {
"label": "OpenAI computer use guide: integration paths",
"url": "https://platform.openai.com/docs/guides/tools-computer-use"
}
},
{
"text":
"Anthropic provides a reference implementation with a web UI, Docker container, example tools, and an agent loop to get started quickly.",
"source": {
"label": "Claude computer use tool: quick start",
"url": "https://platform.claude.com/docs/en/agents-and-tools/tool-use/computer-use-tool"
}
}
],
"faq": [
{
"question": "Are computer-use agents and OpenAdapt rivals?",
"answer":
"They are complementary. Agent providers themselves recommend isolated environments and human oversight for high-impact actions; OpenAdapt uses models too - at compile and repair time - while healthy runs replay deterministically with zero model calls.",
"sources": [
{
"label": "OpenAI computer use guide: safeguards",
"url": "https://platform.openai.com/docs/guides/tools-computer-use"
}
]
},
{
"question": "Which should run the same workflow a thousand times?",
"answer":
"When correctness matters, replay a verified program: OpenAdapt's published evidence counts verified outcomes, expected halts, and silent incorrect successes per workflow - for example 18 MockMed triage trials with 0 model calls and 0 silent incorrect successes - instead of re-reasoning the task on every run.",
"sources": [
{
"label": "mockmed-triage-v3 evidence pack",
"url": "https://github.com/OpenAdaptAI/openadapt-flow/blob/main/public-demo/evidence-packs/mockmed-triage-v3/manifest.json"
}
]
}
]
}
Loading
Loading