feat(desktop-context): add approved desktop context capsules

[hiqiancheng]

First external contributors may need to complete the CLA Assistant check before merge.
For code changes, this PR must link the related issue or RFC in the section below.

Summary

Adds the approved read-only desktop context path for TouchAI invocation context:

• captures a safe invocation capsule before the search surface replaces the user's desktop context
• captures selected text at invocation and injects only a redacted selected-text summary into the default model context
• keeps selected-text full text, clipboard contents, and screenshots behind builtin__get_desktop_context approval
• supports extensible include: string[] fields for safe metadata, selected text, clipboard, screenshot metadata/image, capabilities, and redactions
• renders desktop context details inside the builtin__get_desktop_context tool-call expansion instead of the user message bubble, with screenshot preview first and raw complete tool JSON after it
• captures approved screenshots with xcap, persists them under the desktop-context artifact store, attaches image results to the model, and updates turn artifacts for history replay
• avoids persistent Tauri content protection for screenshots; approved screenshots now temporarily hide TouchAI transient windows (main, popup-*, tray-menu) during capture and restore them immediately, preserving remote desktop and screen sharing compatibility
• improves selected-text extraction on Windows with UI Automation TextPattern, active-window descendant scanning, focused-element process checks, and a read-only native Edit/RichEdit HWND fallback
• persists context metadata and approved screenshot artifacts across session history without storing clipboard full text or selected-text full text as history artifacts

Related issue or RFC

• Related to [RFC]: add read-only desktop context capsules #281

AI assistance disclosure

• Tool(s) used: Codex
• Scope of assistance: implementation, local verification, commit/PR preparation
• Human review or rewrite performed: reviewed the capsule capture/binding, approval, AgentService, built-in tool, native runtime, persistence, history hydration, screenshot capture, and conversation rendering paths before submission
• Architecture or boundary impact: touches AgentService, tool execution, desktop runtime, search surface window behavior, conversation rendering, and session persistence under the [RFC]: add read-only desktop context capsules #281 RFC scope

Testing evidence

Commands run locally:

$env:TEMP='E:\TouchAI\tmp\issue-281-desktop-context-capsules'; $env:TMP='E:\TouchAI\tmp\issue-281-desktop-context-capsules'; pnpm exec vitest run --configLoader runner tests/services/DesktopContextService/tool-payload.test.ts tests/services/AgentService/prompt/language-context.test.ts tests/services/BuiltInToolService/tools/desktopContext/index.test.ts tests/components/DesktopContextToolCallItem.test.ts
# pass: 4 test files, 22 tests

pnpm -C apps/desktop exec vitest run --configLoader runner --passWithNoTests tests/services/DesktopContextService/tool-payload.test.ts tests/services/BuiltInToolService/tools/desktopContext/index.test.ts tests/services/AgentService/prompt/language-context.test.ts tests/services/AgentService/session-history-i18n.test.ts tests/composables/SearchView/useSearchRequest.test.ts tests/i18n/sourceCoverage.test.ts
# pass: 6 test files, 30 tests

$env:TEMP='E:\TouchAI\tmp\issue-281-desktop-context-capsules'; $env:TMP='E:\TouchAI\tmp\issue-281-desktop-context-capsules'; pnpm --filter @touchai/desktop type:check
# pass

$env:TEMP='E:\TouchAI\tmp\issue-281-desktop-context-capsules'; $env:TMP='E:\TouchAI\tmp\issue-281-desktop-context-capsules'; pnpm --filter @touchai/desktop test:typecheck
# pass

$env:TEMP='E:\TouchAI\tmp\issue-281-desktop-context-capsules'; $env:TMP='E:\TouchAI\tmp\issue-281-desktop-context-capsules'; pnpm --filter @touchai/desktop lint:check
# pass with 20 existing warnings in unrelated Vue/test files; 0 errors

$env:TEMP='E:\TouchAI\tmp\issue-281-desktop-context-capsules'; $env:TMP='E:\TouchAI\tmp\issue-281-desktop-context-capsules'; pnpm --filter @touchai/desktop format:check
# pass

$env:CARGO_TARGET_DIR='E:\TouchAI\target\issue-281-desktop-context-capsules'; $env:TEMP='E:\TouchAI\tmp\issue-281-desktop-context-capsules'; $env:TMP='E:\TouchAI\tmp\issue-281-desktop-context-capsules'; cargo fmt --manifest-path apps/desktop/src-tauri/Cargo.toml --all -- --check
# pass

$env:CARGO_TARGET_DIR='E:\TouchAI\target\issue-281-desktop-context-capsules'; $env:TEMP='E:\TouchAI\tmp\issue-281-desktop-context-capsules'; $env:TMP='E:\TouchAI\tmp\issue-281-desktop-context-capsules'; cargo check --manifest-path apps/desktop/src-tauri/Cargo.toml --lib --profile ci-check -j 1
# pass with existing warnings: unused import/function try_lock_mutex in src/core/search/manager.rs

$env:CARGO_TARGET_DIR='E:\TouchAI\target\issue-281-desktop-context-capsules'; $env:TEMP='E:\TouchAI\tmp\issue-281-desktop-context-capsules'; $env:TMP='E:\TouchAI\tmp\issue-281-desktop-context-capsules'; cargo test --manifest-path apps/desktop/src-tauri/Cargo.toml core::system::desktop_context::tests --lib --profile ci-check -j 1
# pass: 4 tests

$env:TEMP='E:\TouchAI\tmp\issue-281-desktop-context-capsules'; $env:TMP='E:\TouchAI\tmp\issue-281-desktop-context-capsules'; $env:CARGO_TARGET_DIR='E:\TouchAI\target\issue-281-desktop-context-capsules'; cargo test --manifest-path apps/desktop/src-tauri/Cargo.toml --lib core::window::search::surface::tests --profile ci-check -j 1
# pass: 5 tests

git diff --check
# pass

git diff --staged --check
# pass before commit

rg -n "set_content_protected|content_protected|protect_search_window_from_screen_capture" apps/desktop/src-tauri/src -g "*.rs"
# no matches

pnpm --filter @touchai/desktop test:e2e built the desktop app successfully, then failed before running specs because the local machine does not have msedgedriver.exe installed/configured:

Error: msedgedriver.exe was not found. Install a matching Edge driver via the official Tauri WebDriver docs or set TOUCHAI_MSEDGEDRIVER_PATH.

TDD: the latest follow-up added focused regression tests for default selected-text redaction, prompt injection, approval boundaries, user-message rendering removal, and desktop-context tool-call expansion before the final implementation was completed. Earlier feature work was not fully test-first.

Risk notes

• AgentService, runtime, MCP, or schema impact: AgentService prompt/runtime/tool execution now carries an optional bound desktop context capsule for the current turn; built-in tool execution can persist an updated approved screenshot artifact after tool approval.
• database baseline or migration impact: uses the existing session_turn_context_artifacts table and updates artifact upsert behavior so approved screenshot metadata can replace the pending metadata for the bound capsule.
• release or packaging impact: adds native dependency xcap for screenshot/window capture; Windows selected-text support uses UI Automation plus read-only Win32 text-control fallback; macOS/Linux selected-text extraction returns explicit unsupported/permission reasons until a verified platform backend is added.
• privacy impact: selected text is captured at invocation to preserve the original desktop state, but default model exposure is limited to a redacted summary; raw selected-text full text still requires tool approval.
• windowing impact: removes persistent set_content_protected(true) from the search surface; screenshot capture briefly hides and restores TouchAI transient windows, with blur suppression during the capture window to avoid treating the temporary hide as a user dismiss.

Screenshots or recordings

Not included. This change affects tool-call rendering and approval/runtime behavior; the new desktop-context tool expansion is covered by tests/components/DesktopContextToolCallItem.test.ts.

Checklist

• The PR title follows Conventional Commits and is valid for squash merge.
• This PR is either ready for review or explicitly marked as a Draft PR.
• I did not use [WIP] or similar title prefixes.
• If AI materially assisted this PR, I disclosed the tools and scope and I personally reviewed every affected change.
• I can explain the why, what, and how of this change without relying on an AI tool.
• If this touches AgentService, runtime, MCP, or schema boundaries, there is an accepted RFC.
• If this changes architecture or adds a new cross-boundary abstraction, there is an accepted RFC.
• I ran pnpm test:pr for this code PR, or this is a docs-only change.
• If I changed Rust behavior or tests, I reviewed pnpm test:coverage:rust or relied on CI coverage evidence.
• If I changed desktop startup/window/search/popup/settings/E2E paths, I ran pnpm test:e2e locally or documented why CI is the first valid proof.
• I added tests or explained why tests are not appropriate.
• I updated docs when behavior changed.

[coderabbitai]

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: baed79a5-98de-47f5-b70b-c31dc49faf30

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch codex/issue-281-desktop-context-capsules

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}