From 0d490edb677254e1aa5b2ebf719be5e64fae330f Mon Sep 17 00:00:00 2001 From: Marvin-Cypher Date: Wed, 24 Jun 2026 11:11:46 -0700 Subject: [PATCH] docs: sharpen README positioning --- README.md | 296 +++++++++++++++++++++++++++++------------------------- 1 file changed, 158 insertions(+), 138 deletions(-) diff --git a/README.md b/README.md index b0bb0dfe..f5e34847 100644 --- a/README.md +++ b/README.md @@ -2,7 +2,7 @@ ![dstack](./dstack-logo.svg) -### The open framework for confidential AI. +### Run private AI services with verifiable app identity. [![GitHub Stars](https://img.shields.io/github/stars/dstack-tee/dstack?style=flat-square&logo=github)](https://github.com/Dstack-TEE/dstack/stargazers) [![License](https://img.shields.io/github/license/dstack-tee/dstack?style=flat-square)](https://github.com/Dstack-TEE/dstack/blob/master/LICENSE) @@ -10,8 +10,6 @@ [![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/Dstack-TEE/dstack) [![Telegram](https://img.shields.io/badge/Telegram-2CA5E0?style=flat-square&logo=telegram&logoColor=white)](https://t.me/+UO4bS4jflr45YmUx) -Original Contributors: Hang Yin, Kevin Wang, Andrew Miller - [Documentation](https://docs.phala.com/dstack) · [Examples](https://github.com/Dstack-TEE/dstack-examples) · [Community](https://t.me/+UO4bS4jflr45YmUx) @@ -20,43 +18,48 @@ Original Contributors: Hang Yin, Kevin Wang, Andrew Miller ## What is dstack? -dstack is the open framework for confidential AI — deploy AI applications with cryptographic privacy guarantees. - -AI providers ask users to trust them with sensitive data. But trust doesn't scale, and trust can't be verified. With dstack, your containers run inside confidential VMs (Intel TDX) with native support for NVIDIA Confidential Computing (H100, Blackwell). Users can cryptographically verify exactly what's running: private AI with your existing Docker workflow. +dstack turns a Docker Compose app into a verifiable Confidential VM deployment. -## Supported Platforms +It is useful when a service must prove more than "it runs in a TEE." dstack +binds the application config, guest OS image, KMS key release, and hardware +evidence into one identity that users or auditors can verify. -| Platform | Status | Attestation | -|----------|--------|-------------| -| **Bare metal TDX** | Available | TDX | -| **[Phala Cloud](https://cloud.phala.network)** | Available | TDX | -| **GCP Confidential VMs** | Available | TDX + TPM | -| **AWS Nitro Enclaves** | Available | NSM | +```text +docker-compose.yaml + -> app-compose.json + -> compose-hash + -> RTMR3 measurement + -> attestation quote + -> KMS key release / verifier decision +``` -## Features +The main use case is private AI inference: run an OpenAI-compatible endpoint in +a TEE, keep prompts and credentials protected from operators, and give customers +proof of the workload that handled their request. -**Zero friction onboarding** -- **Docker Compose native**: Bring your docker-compose.yaml as-is. No SDK, no code changes. -- **Encrypted by default**: Network traffic and disk storage encrypted out of the box. +## Why dstack? -**Hardware-rooted security** -- **Private by hardware**: Data encrypted in memory, inaccessible even to the host. -- **Reproducible OS**: Deterministic builds mean anyone can verify the OS image hash. -- **Workload identity**: Every app gets an attested identity users can verify cryptographically. -- **Confidential GPUs**: Native support for NVIDIA Confidential Computing (H100, Blackwell). +Most confidential-computing tools solve one layer: VM isolation, attestation, +secret release, or a fixed inference appliance. dstack connects those layers +into an application stack. -**Trustless operations** -- **Isolated keys**: Per-app keys derived in TEE. Survives hardware failure. Never exposed to operators. -- **Code governance**: Updates follow predefined rules (e.g., multi-party approval). Operators can't swap code or access secrets. +| Need | dstack provides | +| --- | --- | +| Run custom services | Docker Compose deployment into Confidential VMs | +| Prove the app, not only the machine | `compose-hash`, `app-id`, `instance-id`, and KMS binding in RTMR3 | +| Release secrets to the right workload | KMS verifies evidence before deriving app-bound keys | +| Serve production traffic | Gateway routing, TLS, and attested service channels | +| Use TEE features from code | SDK access to keys, quotes, certs, and signing over `/var/run/dstack.sock` | +| Verify AI hardware boundaries | CPU TEE evidence plus NVIDIA confidential GPU evidence where supported | -## Getting Started +Best fit: private inference endpoints, confidential AI agents, and verifiable +backends where the proof must cover app config, keys, endpoint, and hardware. -**Try it now:** Chat with LLMs running in TEE at [chat.redpill.ai](https://chat.redpill.ai). Click the shield icon to verify attestations from Intel TDX and NVIDIA GPUs. +## Best-practice use case: private inference -**Deploy your own:** +Start with an OpenAI-compatible model server: ```yaml -# docker-compose.yaml services: vllm: image: vllm/vllm-openai:latest @@ -66,139 +69,162 @@ services: - "8000:8000" ``` -Deploy to any Intel TDX host using a guest OS image from [meta-dstack releases](https://github.com/Dstack-TEE/meta-dstack/releases), or use [Phala Cloud](https://cloud.phala.network) for managed infrastructure. - -Setting up dstack on your own hardware? See the [full deployment guide →](./docs/deployment.md) - -## Architecture - -![Architecture](./docs/assets/arch.png) - -Your container runs inside a Confidential VM (Intel TDX) with optional GPU isolation via NVIDIA Confidential Computing. The CPU TEE protects application logic; the GPU TEE protects model weights and inference data. - -**Core components:** - -- **Guest Agent**: Runs inside each CVM. Generates TDX attestation quotes so users can verify exactly what's running. Provisions per-app cryptographic keys from KMS. Encrypts local storage. Apps interact via `/var/run/dstack.sock`. - -- **KMS**: Runs in its own TEE. Verifies TDX quotes before releasing keys. Enforces authorization policies defined in on-chain smart contracts — operators cannot bypass these checks. Derives deterministic keys bound to each app's attested identity. - -- **Gateway**: Terminates TLS at the edge and provisions ACME certificates automatically. Routes traffic to CVMs. All internal communication uses RA-TLS for mutual attestation. +Production shape: + +1. Pin container images by digest. +2. Generate `app-compose.json`; its `compose-hash` becomes the workload identity. +3. Put model credentials, API keys, TLS material, and signing keys behind + KMS-gated release. +4. Run the service in a TDX CVM, with NVIDIA confidential GPU evidence when GPU + inference is used. +5. Expose the endpoint through the gateway or your own TLS path. +6. Let customers verify quote signature, TCB status, OS image hash, RTMR3 replay, + `compose-hash`, KMS binding, and GPU evidence. + +This is the difference between a private endpoint and a response tied to a +measured workload with app-bound keys. + +## How it works + +```mermaid +flowchart TB + Dev["Developer
docker-compose.yaml"] --> Compose["app-compose.json
compose-hash"] + Compose --> VMM["dstack-vmm
CVM lifecycle + resources"] + + subgraph Host["TDX host / cloud TEE node"] + VMM + subgraph CVM["App CVM"] + Agent["dstack-guest-agent
RTMR3 events + Docker startup"] + App["App containers
vLLM / agent / backend"] + Socket["/var/run/dstack.sock"] + Agent <--> Socket + App <--> Socket + end + end + + VMM --> Agent + Agent -->|"quote + app identity"| KMS["dstack-kms
TEE key release"] + KMS --> Auth["auth-simple / on-chain policy"] + KMS -->|"app-bound keys"| Agent + + User["User / client"] --> Gateway["dstack-gateway
TLS + routing"] + Gateway -->|"RA-TLS / WireGuard"| Agent + + Agent --> Evidence["Evidence
TDX quote + RTMR3 log
OS image hash + GPU evidence"] + Evidence --> Verifier["Verifier
policy decision"] + Verifier --> User +``` -- **VMM**: Runs on bare-metal TDX hosts. Parses docker-compose files directly — no app changes needed. Boots CVMs from a reproducible OS image. Allocates CPU, memory, and confidential GPU resources. +| Component | Role | +| --- | --- | +| `dstack-vmm` | Boots and manages CVMs with QEMU, resources, disks, networking, and optional GPU passthrough. | +| `dstack-guest-agent` | Runs inside the CVM, extends runtime measurements, requests keys, starts containers, and exposes the app API. | +| `dstack-kms` | Runs in a TEE and releases app-bound keys only after evidence and policy checks. | +| `dstack-gateway` | Routes public traffic and connects to CVMs through attested channels. | +| SDKs | Let app code request keys, quotes, RA-TLS certs, and signatures. | +| Verifier | Checks quote signatures, OS measurements, RTMR3 replay, app identity, and TCB status. | + +Source areas: `vmm/`, `guest-agent/`, `kms/`, `gateway/`, `dstack-attest/`, +`ra-tls/`, `verifier/`, and `sdk/`. + +## Key technical primitives + +- **`app-compose.json`**: normalized deployment config derived from Docker Compose. +- **`compose-hash`**: SHA256 of `app-compose.json`, measured into RTMR3. +- **RTMR3 event log replay**: verifier recomputes runtime measurements and + compares them with the hardware-signed quote. +- **KMS-gated key release**: app-bound keys are derived only after quote and + policy verification. +- **RA-TLS**: attestation material is embedded in X.509 extensions under the + `1.3.6.1.4.1.62397.1.*` OID arc. +- **GPU confidential computing**: supported NVIDIA H100, H200, and Blackwell + deployments can include confidential GPU evidence. + +## Supported platforms -[Full security model →](./docs/security/security-model.md) +| Platform | Status | Attestation | +| --- | --- | --- | +| Bare metal Intel TDX | Available | TDX | +| [Phala Cloud](https://cloud.phala.network) | Available | TDX | +| GCP Confidential VMs | Available | TDX + TPM | +| AWS Nitro Enclaves | Available | NSM | + +## Start here + +| Goal | Link | +| --- | --- | +| Deploy on GCP | [Quickstart](./docs/quickstart.md) | +| Self-host on TDX hardware | [Deployment Guide](./docs/deployment.md) | +| Build a confidential AI app | [Confidential AI Guide](./docs/confidential-ai.md) | +| Verify a deployment | [Verification Guide](./docs/verification.md) | +| Read the security model | [Security Model](./docs/security/security-model.md) | +| See examples | [dstack-examples](https://github.com/Dstack-TEE/dstack-examples) | + +Try a live private AI deployment at [chat.redpill.ai](https://chat.redpill.ai) +and open the shield icon to inspect attestation evidence. ## SDKs -Apps communicate with the guest agent via HTTP over `/var/run/dstack.sock`. Use the [HTTP API](./sdk/curl/api.md) directly with curl, or use a language SDK: +Applications talk to the guest agent over HTTP on `/var/run/dstack.sock`. | Language | Install | Docs | -|----------|---------|------| +| --- | --- | --- | | Python | `pip install dstack-sdk` | [README](./sdk/python/README.md) | | TypeScript | `npm install @phala/dstack-sdk` | [README](./sdk/js/README.md) | | Rust | `cargo add dstack-sdk` | [README](./sdk/rust/README.md) | | Go | `go get github.com/Dstack-TEE/dstack/sdk/go` | [README](./sdk/go/README.md) | +| HTTP | Any Unix-socket-capable HTTP client | [API](./sdk/curl/api.md) | -## Documentation - -**For Developers** -- [Confidential AI](./docs/confidential-ai.md) - Inference, agents, and training with hardware privacy -- [Usage Guide](./docs/usage.md) - Deploying and managing apps -- [Verification](./docs/verification.md) - How to verify TEE attestation - -**For Operators** -- [Deployment](./docs/deployment.md) - Self-hosting on TDX hardware -- [On-Chain Governance](./docs/onchain-governance.md) - Smart contract authorization -- [Gateway](./docs/dstack-gateway.md) - Gateway configuration - -**Reference** -- [App Compose Format](./docs/normalized-app-compose.md) - Compose file specification -- [VMM CLI Guide](./docs/vmm-cli-user-guide.md) - Command-line reference -- [Design Decisions](./docs/design-and-hardening-decisions.md) - Architecture rationale -- [FAQ](./docs/faq.md) - Frequently asked questions - -## Security - -- [Security Overview](./docs/security/) - Security documentation and responsible disclosure -- [Security Model](./docs/security/security-model.md) - Threat model and trust boundaries -- [Security Best Practices](./docs/security/security-best-practices.md) - Production hardening -- [Security Audit](./docs/security/dstack-audit.pdf) - Third-party audit by zkSecurity -- [CVM Boundaries](./docs/security/cvm-boundaries.md) - Information exchange and isolation - -## FAQ - -
-Why not use AWS Nitro / Azure Confidential VMs / GCP directly? - -You can — but you'll build everything yourself: attestation verification, key management, Docker orchestration, certificate provisioning, and governance. dstack provides all of this out of the box. - -| Approach | Docker native | GPU TEE | Key management | Attestation tooling | Open source | -|----------|:-------------:|:-------:|:--------------:|:-------------------:|:-----------:| -| **dstack** | ✓ | ✓ | ✓ | ✓ | ✓ | -| AWS Nitro Enclaves | - | - | Manual | Manual | - | -| Azure Confidential VMs | - | Preview | Manual | Manual | - | -| GCP Confidential Computing | - | - | Manual | Manual | - | - -Cloud providers give you the hardware primitive. dstack gives you the full stack: reproducible OS images, automatic attestation, per-app key derivation, TLS certificates, and smart contract governance. No vendor lock-in. - -
- -
-How is this different from SGX/Gramine? - -SGX requires porting applications to enclaves. dstack uses full-VM isolation (Intel TDX) — bring your Docker containers as-is. Plus GPU TEE support that SGX doesn't offer. - -
+## Verification -
-What's the performance overhead? +A relying party should be able to check: -Minimal. Intel TDX adds ~2-5% overhead for CPU workloads. NVIDIA Confidential Computing has negligible impact on GPU inference. The main cost is memory encryption, which is hardware-accelerated on supported CPUs. +- quote signature and TCB status; +- approved OS image hash and expected MRTD / RTMR0-2 measurements; +- RTMR3 replay, including `compose-hash`, `app-id`, `instance-id`, and + key-provider binding; +- KMS identity and policy decision; +- NVIDIA confidential GPU evidence for GPU workloads. -
+Tools and docs: -
-Is this production-ready? +- [dstack verifier](./verifier/) +- [Verification Guide](./docs/verification.md) +- [Attestation Verification Tutorial](./docs/tutorials/attestation-verification.md) -Yes. dstack powers production AI infrastructure at [OpenRouter](https://openrouter.ai/provider/phala) and [NEAR AI](https://x.com/ilblackdragon/status/1962920246148268235). The framework has been [audited by zkSecurity](./docs/security/dstack-audit.pdf) and is a Linux Foundation Confidential Computing Consortium project. - -
- -
-Can I run this on my own hardware? - -Yes. dstack runs on any Intel TDX-capable server. See the [deployment guide](./docs/deployment.md) for self-hosting instructions. You can also use [Phala Cloud](https://cloud.phala.network) for managed infrastructure. - -
- -
-What TEE hardware is supported? +## Security -Currently: Intel TDX (4th/5th Gen Xeon) and NVIDIA Confidential Computing (H100, Blackwell). AMD SEV-SNP support is planned. +dstack has been audited by [zkSecurity](./docs/security/dstack-audit.pdf). -
+Attestation proves which hardware and measured software produced the evidence. +It does not prove application code is bug-free. For production, pin container +images by digest, audit code that handles secrets, and define a policy for +accepted OS images, compose hashes, KMS instances, and GPU evidence. -
-How do users verify my deployment? +Security docs: -Your app exposes attestation quotes via the SDK. Users verify these quotes using [dstack-verifier](https://github.com/Dstack-TEE/dstack/tree/master/verifier), [dcap-qvl](https://github.com/Phala-Network/dcap-qvl), or the [Trust Center](https://trust.phala.com). See the [verification guide](./docs/verification.md) for details. +- [Security Overview](./docs/security/) +- [Security Model](./docs/security/security-model.md) +- [Security Best Practices](./docs/security/security-best-practices.md) +- [CVM Boundaries](./docs/security/cvm-boundaries.md) -
+## Production use -## Trusted by +dstack powers confidential AI infrastructure for: -- [OpenRouter](https://openrouter.ai/provider/phala) - Confidential AI inference providers powered by dstack -- [NEAR AI](https://x.com/ilblackdragon/status/1962920246148268235) - Private AI infrastructure powered by dstack +- [OpenRouter](https://openrouter.ai/provider/phala) +- [NEAR AI](https://x.com/ilblackdragon/status/1962920246148268235) -dstack is a Linux Foundation [Confidential Computing Consortium](https://confidentialcomputing.io/2025/10/02/welcoming-phala-to-the-confidential-computing-consortium/) open source project. +dstack is a Linux Foundation +[Confidential Computing Consortium](https://confidentialcomputing.io/2025/10/02/welcoming-phala-to-the-confidential-computing-consortium/) +open source project. ## Community [Telegram](https://t.me/+UO4bS4jflr45YmUx) · [GitHub Discussions](https://github.com/Dstack-TEE/dstack/discussions) · [Examples](https://github.com/Dstack-TEE/dstack-examples) -For enterprise support and licensing, [book a call](https://cal.com/team/phala/founders) or email us at support@phala.network. - -[![Repobeats](https://repobeats.axiom.co/api/embed/0a001cc3c1f387fae08172a9e116b0ec367b8971.svg)](https://github.com/Dstack-TEE/dstack/pulse) +For enterprise support, [book a call](https://cal.com/team/phala/founders) or +email support@phala.network. ## Cite @@ -213,12 +239,6 @@ If you use dstack in your research, please cite: } ``` -## Media Kit - -Logo and branding assets: [dstack-logo-kit](./docs/assets/dstack-logo-kit/) - ## License Apache 2.0 - -