better separation on desktop to backend boundary

.github/workflows/desktop-build.yml

@@ -1,10 +1,10 @@
 name: Desktop native builds
 on:
   pull_request:
-    paths: ['umbod/api/**', 'packages/umbod-sdk/**', 'desktop/**', 'website/**', '.github/workflows/desktop-*.yml', '.dockerignore']
+    paths: ['umbod/api/**', 'packages/umbod-sdk/**', 'desktop/**', 'website/**', '.github/workflows/desktop-*.yml', '.dockerignore', 'flake.nix', 'flake.lock']
   push:
     branches: [main, feat/desktop-monorepo-releases]
-    paths: ['umbod/api/**', 'packages/umbod-sdk/**', 'desktop/**', 'website/**', '.github/workflows/desktop-*.yml', '.dockerignore']
+    paths: ['umbod/api/**', 'packages/umbod-sdk/**', 'desktop/**', 'website/**', '.github/workflows/desktop-*.yml', '.dockerignore', 'flake.nix', 'flake.lock']
   workflow_dispatch:
   workflow_call:
 permissions:
@@ -48,10 +48,8 @@ jobs:
       - uses: actions/checkout@v4
         with:
           persist-credentials: false
-      - name: Verify native Apple Silicon runner and Swift 6
-        run: |
-          test "$(uname -m)" = arm64
-          swift --version | grep -E "Swift version 6\."
+      - name: Verify full Xcode and SwiftUI toolchain
+        run: bash scripts/check-macos-toolchain.sh
       - uses: actions/setup-python@v5
         with:
           python-version: '3.12'

desktop/README.md

@@ -6,7 +6,7 @@ Verified desktop releases are available from https://umbod.ai/download/. Local b
 
 ## Build and test
 
-Run from the monorepo root unless a command explicitly changes directory. Build prerequisites: uv, Python 3.11+, Swift 6 and Apple developer tools on macOS. Packaging downloads a pinned CPython 3.14.3 runtime and verifies its SHA-256. No command below installs or launches the production app.
+Run from the monorepo root unless a command explicitly changes directory. Build prerequisites: uv, Python 3.11+, and full Xcode with Swift 6 or newer on Apple Silicon macOS; Command Line Tools alone are insufficient. Install Xcode and open it to finish first-launch setup and accept its license. Select the installation with `DEVELOPER_DIR` or `sudo xcode-select --switch /Applications/Xcode.app/Contents/Developer`. Run `bash desktop/scripts/check-macos-toolchain.sh` to verify the selection and SwiftUI macros before building. On Apple Silicon macOS, `nix develop .#desktop` supplies the normal repository tools and pinned Python/uv environment and checks the external Apple compiler and SDK; it does not install Swift, Xcode, or an SDK. The general `nix develop` shell has no Xcode requirement. Packaging downloads a pinned CPython 3.14.3 runtime and verifies its SHA-256. No command below installs or launches the production app.
 
 ```sh
 cd desktop
@@ -55,11 +55,15 @@ OAuth supports native public clients with loopback callbacks, discovery, PKCE an
 
 Older installed versions that require a Windows asset in update metadata cannot read new macOS-only manifests. Download the first macOS-only release manually from the website; its updater accepts both historical and macOS-only manifests. Existing published releases are not changed.
 
+## Shared composition
+
+The Python host uses the backend-owned [`umbod.hosting`](../umbod/api/src/umbod/hosting/README.md) API also used by the separate Kubernetes REST/MCP services. Native lifecycle and credential ownership remain desktop-specific. Desktop migration and tool-name repair live exclusively in the removable [`backward_compatibility`](python/umbod_desktop/backward_compatibility/README.md) package, alongside old-host retirement, historical profile fallback, OAuth conversion and migrated token matching. Compatibility orchestration and permanent usage recording depend on desktop-owned typed ports; concrete shared HTTP, hosting, transport and OAuth adapters live in `python/umbod_desktop/backend_integration/`. The Python index documents the exact remaining naming, configuration and OAuth persistence coupling; the compatibility index owns historical formats, lifecycle entrypoints and concrete removal steps.
+
 ## Code and release index
 
 - [`python/`](python/README.md): macOS host, transport adapters, Keychain, migration and tests; reuses `umbod/api` and `packages/umbod-sdk`.
 - `native/`: SwiftUI app and Swift contract/lifecycle tests; `UmbodCore/AdvancedSettings.swift` owns backend-reported pending/effective settings, confirmation and verified restart/recovery, while `Umbod/AdvancedSettingsView.swift` presents Settings → Advanced; Settings provides compact update controls and generic stdio/HTTP connection details, with the bearer token masked and available only through an explicit copy action. Connector authentication stays optional and credentials remain in Keychain; `UmbodCore/ConnectorEditor.swift` owns the actual editor submission and retry state; `UmbodCore/ToolPermissionQueue.swift` owns optimistic permission edits, ordered save state, and stale-refresh protection; `UmbodCore/DesktopUpdate.swift` validates update metadata and `Umbod/UpdatesView.swift` owns update preferences and the install guidance.
-- `scripts/`: local builds, packaging, isolated fixtures, and release policy/metadata.
+- `scripts/`: local builds, packaging, isolated fixtures, and release policy/metadata; `check-macos-toolchain.sh` owns the shared full-Xcode, Swift 6, SDK and SwiftUI macro preflight used by builds, verification, acceptance preparation and the desktop Nix shell.
 - `VERSION`: independent desktop version; the single source for bundle and package versions.
 - [Release procedure](docs/RELEASING.md), [manifest application](docs/APPLY-MANIFEST.md), and [import provenance and audit](docs/PROVENANCE.md).
 - [0.2.4 release candidate](docs/releases/0.2.4.md): announcement draft and release-owner acceptance checklist; not a publication record.

desktop/python/README.md

@@ -5,21 +5,27 @@ The SwiftUI app keeps its native screens. `AdministrationClient.swift` maps the
 ## Entry points and ownership
 
 - `main.py`: packaged `serve` and `bridge` commands, loopback socket, stable port, process lock, readiness pipe, owner shutdown and dev/release isolation. Its `credential` command lets Swift reattach through a private pipe using the same installed backend identity; Swift does not access Keychain independently. Legacy positional bridge identities remain accepted. `build.sh` bundles a relocatable interpreter and locked dependencies, then signs a fresh bundle.
-- `runtime.py`, `configuration.py`: shared application composition and authenticated `/desktop` routes. SQL event polling synchronizes API writes with MCP. Native single-tool and bulk enablement publish the connector before applying activation and batched permissions; rename-only saves preserve the live transport. Native removal revokes the selected connector’s desktop-owner grants, including stale tool grants, before deleting its registry entry and transport. Dedicated metrics listeners are disabled for desktop. MCP defaults to gateway exposure (`search_tools` and `execute_tool`); saved backend settings select gateway, flat or codemode before constructing the application. Settings never hot-swap registered tools.
+- `runtime.py`, `backend_integration/configuration.py`: shared application composition and authenticated `/desktop` routes. SQL event polling synchronizes API writes with MCP. Native single-tool and bulk enablement publish the connector before applying activation and batched permissions; rename-only saves preserve the live transport. Native removal revokes the selected connector’s desktop-owner grants, including stale tool grants, before deleting its registry entry and transport. Dedicated metrics listeners are disabled for desktop. MCP defaults to gateway exposure (`search_tools` and `execute_tool`); saved backend settings select gateway, flat or codemode before constructing the application. Settings never hot-swap registered tools.
+- `contracts.py`, `ports.py`: immutable desktop DTOs and semantic administration, naming, invocation recording, metadata and upstream preparation protocols. Use cases consume these ports; `DesktopState` satisfies the shared metadata protocol without a persistence rewrite. `CredentialVault` and `DESKTOP_GROUP` remain available through their established public security imports.
+- [`backend_integration/`](umbod_desktop/backend_integration/README.md): concrete HTTP administration, hosting authentication/configuration, shared naming, Keychain OAuth, transport and usage mapping adapters. Permanent backend adapters live here; the compatibility OAuth adapter has a narrow historical conversion import, and runtime additionally uses supported hosting root composition exports.
 - `security.py`: separate random admin, agent, upstream and encryption credentials in macOS Keychain. No development auth or plaintext fallback. Existing agent token hashes are retained for migration; they never grant REST owner access. Agent identities receive `desktop-owner`, so opting into administrator MCP tools grants all of these clients MCP administration access.
-- `upstreams.py`, `models.py`: validated stdio and Streamable HTTP transports exposed through private authenticated loopback proxies to the shared downstream connector API. A proxy owns its lifespan in one task. Offline drafts remain editable and cannot retain an obsolete live transport after an invalid edit. Upstream HTTP challenges and runtime OAuth reauthentication persist a sign-in-required status; successful requests clear it. The private loopback bearer is consumed before proxying, so only connector-owned credentials reach upstream servers.
-- `oauth.py`: macOS Keychain storage and the `DesktopOAuthHost` adapter for shared OAuth connection storage, stable loopback callback addresses, and local transport reconnection. `callback_transport` forwards callback parameters to shared protocol validation. Discovery, public-client registration, scopes, status transitions, safe errors, diagnostics, cancellation, token exchange/refresh, and authorization-versus-discovery outcomes live in `umbod.core.connectors.downstream_mcp.oauth`. Native clients use the shared `/api/admin/connectors/mcp/oauth` API; private `oauth_*` host actions are no longer supported. Existing callback paths and Keychain records remain compatible.
-- `migration.py`, `legacy.py`: authenticated retirement of the old process, resumable settings import, OAuth credential conversion and stable endpoint/client identity preservation. Original settings and Keychain entries are preserved. No PID from disk is signalled.
-- `tool_names.py`: startup repair of desktop-generated UUID/hash tool prefixes through the shared metadata API. Readable prefixes use the shared core naming policy; duplicate imported/repaired names receive numeric suffixes. Only managed connections whose prefix matches their generated public path are repaired. Explicit prefixes, IDs, credentials, paths, activation and grants are preserved; rejected repairs keep the working prefix, log a warning, and retry on restart. New native connections omit the prefix so the shared Python backend derives it from the display name. Renaming a display label does not implicitly rename published tools.
+- `backend_integration/transports.py`, `backend_integration/models.py`: validated stdio and Streamable HTTP transports exposed through private authenticated loopback proxies to the shared downstream connector API. A proxy owns its lifespan in one task. Offline drafts remain editable and cannot retain an obsolete live transport after an invalid edit. Upstream HTTP challenges and runtime OAuth reauthentication persist a sign-in-required status; successful requests clear it. The private loopback bearer is consumed before proxying, so only connector-owned credentials reach upstream servers.
+- `backend_integration/oauth.py`: macOS Keychain storage and the `DesktopOAuthHost` adapter for shared OAuth connection storage, stable loopback callback addresses, and local transport reconnection. `callback_transport` forwards callback parameters to shared protocol validation. Discovery, public-client registration, scopes, status transitions, safe errors, diagnostics, cancellation, token exchange/refresh, and authorization-versus-discovery outcomes live in `umbod.core.connectors.downstream_mcp.oauth`. Native clients use the shared `/api/admin/connectors/mcp/oauth` API; private `oauth_*` host actions are no longer supported. Existing callback paths and Keychain records remain compatible.
+- [`backward_compatibility/`](umbod_desktop/backward_compatibility/README.md): exclusively historical settings import, generated tool-prefix repair, authenticated old-host retirement, legacy OAuth conversion, saved-port fallback and migrated agent hash matching. Migration-only contracts and adapters live inside this removable package. Original settings and Keychain entries are preserved; no PID from disk is signalled. Its index describes lifecycle entrypoints and the deletion checklist. New connections use permanent backend naming rather than compatibility repair.
+- `endpoint.py`: validates the current `python-endpoint.json` port independently of historical formats; `main.py` explicitly supplies the compatibility fallback.
 - `settings.py`: strict exposure-mode and boolean administrator settings, defaulting to gateway/off for existing profiles. `GET /desktop/settings` returns `pending`, immutable process `effective`, and `restart_required`; `PUT /desktop/settings` validates and atomically saves pending values without changing the running app. Both require the private owner/admin bearer; the agent token cannot access this API even when administrator MCP is enabled. Swift verifies health and reads back effective settings after restarting. Unknown modes, malformed booleans and unexpected fields are rejected. Invalid persisted settings fail closed without rewriting or resetting the profile; repair only the `settings` object (or restore its valid backup), leaving connectors, credentials, migration journals and grants intact. Operator `.env` settings remain ignored.
-- `storage.py`, `usage.py`: owner-only atomic host metadata and aggregate call counts, without arguments or results. Shared connectors and encrypted configuration live in `backend.sqlite3`.
+- `persistence.py`, `storage.py`, `usage.py`: owner-only atomic file helpers, host metadata and typed aggregate completion recording, without arguments or results. `backend_integration/usage.py` filters and maps hosting events before recording. Shared connectors and encrypted configuration live in `backend.sqlite3`.
 - `client_setup.py`: consented configuration previews, conflict detection and byte-preserving backup restoration.
 - `testing.py`: explicitly selected fixture credentials, restricted to owned `umbod-test-*` directories under the system temporary directory. This is never a fallback for unavailable Keychain access; normal profiles always use Keychain.
 - `tests/`: real HTTP/stdio MCP calls, default-deny and live revocation, resumable import, authentication isolation, OAuth refresh/callback races and safe client-file editing. Native tests additionally exercise the packaged interpreter, an explicit disposable fixture vault and persistent-process reattachment. Routine tests never open the user’s Keychain. Automated native UI and window-lifecycle tests are currently removed.
 
 ## Shared backend extension points
 
-Shared backend host extension points include OAuth storage/callback composition, an explicitly supplied REST identity verifier, optional REST/MCP metrics listeners (enabled by default), SQL-backed group permission reloads, and an injectable MCP invocation sink. The host injects the shared `OAuthService` through `create_app(oauth_service=...)` and closes it with the bundled process lifecycle. Enterprise verifier selection and domain behavior retain their defaults. Desktop authentication is fixed to one local owner and one permission group.
+`runtime.py` is the outer composition root. Desktop use cases depend on `ports.py` and `contracts.py`; `backend_integration` implements those desktop-owned ports. Supported backend assembly, verifiers, configuration and invocation events come from [`umbod.hosting`](../../umbod/api/src/umbod/hosting/README.md). Integration adapters retain exact naming, secret derivation, EndpointUrl and OAuth storage/connection/provider imports, detailed in their index; the compatibility OAuth adapter separately owns historical authorization conversion. The shared administration HTTP JSON contract remains authoritative. This is dependency inversion for desktop use cases, not a replacement backend or a claim that all desktop-local file I/O is abstracted.
+
+Runtime loads DesktopState before mapping its immutable settings snapshot. Administration owns the authenticated ASGI client, and runtime closes it in the former migration-client callback position. Naming repair is composed independently of migration. Existing startup/shutdown order, credentials, journal dictionaries and public JSON remain unchanged. Changes to these interfaces require explicit compatibility review and updated contract tests. `tests/test_architecture.py` and API hosting boundary tests walk nested imports, resolve relative imports, check exact residual symbols by adapter file and test their scanners against violating snippets; desktop tests additionally verify transitive use-case independence.
+
+Shared backend host extension points include OAuth storage/callback composition, an explicitly supplied REST identity verifier, optional REST/MCP metrics listeners (enabled by default), SQL-backed group permission reloads, and an injectable MCP invocation sink. The host injects the shared `OAuthService` through the `oauth_service` hook on `create_rest_app` and closes it with the bundled process lifecycle. Enterprise verifier selection and domain behavior retain their defaults. Desktop authentication is fixed to one local owner and one permission group.
 
 The desktop host provides authenticated stdio/HTTP transport adapters instead of adding desktop subprocess, Keychain or browser lifecycle behavior to the shared core. The REST API remains the authority for connector discovery, tool activation, publication and permissions.