Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@
| How the system is built (packages, data flow, frontend/backend) | [architecture/](architecture/) |
| What happens at runtime (startup, chat, live reload, sharing) | [sequence-flows/](sequence-flows/) |
| Frontend build details (templates vs. web, export design) | [dev/](dev/) |
| Design system (themes, CSS tokens, custom themes) | [design/design-system.md](design/design-system.md) |
| Worker metrics dashboard | [dev/metrics-dashboard.md](dev/metrics-dashboard.md) |
| End-to-end browser testing (Playwright) | [dev/e2e-testing.md](dev/e2e-testing.md) |
| Keyboard shortcuts | [../user-docs/en/keyboard-shortcuts.md](../user-docs/en/keyboard-shortcuts.md) |
| Why this exists | [../user-docs/en/why.md](../user-docs/en/why.md) |
Expand Down
14 changes: 6 additions & 8 deletions docs/architecture/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ This directory contains the architecture documentation for **pi-web**, a local w
|----------|-------------|
| [system-overview.md](./system-overview.md) | High-level system architecture, component diagram, and tech stack |
| [backend.md](./backend.md) | Go backend: packages, responsibilities, and key types |
| [frontend.md](./frontend.md) | Frontend architecture: embedded templates, Vite build, and vanilla JS |
| [frontend.md](./frontend.md) | Frontend architecture: Svelte SPA, Vite build, embedded shell, and static export |
| [data-flow.md](./data-flow.md) | Session file format, data model, and storage layout |

## Architecture at a Glance
Expand All @@ -17,9 +17,9 @@ This directory contains the architecture documentation for **pi-web**, a local w
┌─────────────────────────────────────────────────────────────────────┐
│ Browser │
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────────────────────┐ │
│ │ / (index) │ │ /session?id │ │ SSE /events │ │
│ │ vanilla JS │ │ Embedded │ │ Live reload + status │ │
│ │ (Vite) HTML/CSS │ │ updates │ │
│ │ / /session │ │ /settings │ │ SSE /events │ │
│ │ /schedules │ │ Svelte SPA │ │ Live reload + status │ │
│ │ Svelte SPA │ │ │ updates │ │
│ └─────────────┘ └─────────────┘ └─────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────┘
Expand All @@ -46,14 +46,12 @@ This directory contains the architecture documentation for **pi-web**, a local w

## Key Design Decisions

1. **Append-only session metadata**: pi-web reads from `~/.pi/agent/sessions/` and avoids rewriting session history. New sessions can be created via the web UI, and browser rename appends a `session_info` metadata line to the existing JSONL file.
1. **Append-only session metadata**: pi-web reads from `~/.pi/agent/sessions/` and avoids rewriting session history. New sessions can be created via the web UI; rename and auto-title append a `session_info` metadata line, and entry labels append a `label` line, to the existing JSONL file.

2. **Live updates via SSE**: The browser opens an EventSource connection. The server watches session files via `fsnotify` (with polling fallback) and pushes `reload` events; session pages fetch `/api/session` to reconcile canonical JSONL entries. Browser chat can also receive best-effort `chat-preview` SSE events before JSONL reconciliation.

3. **Chat via RPC workers**: Each session gets a dedicated `pi --mode rpc` subprocess. Workers are cached and reaped after 10 minutes of idle time.

4. **Dual frontend strategy**:
- **Index page** (`/`): Built with Vite + vanilla JS, served from embedded `web/dist`
- **Session page** (`/session`): Server-rendered HTML shell with Vite-built session JS
4. **Single Svelte SPA + static export**: All live browser routes (`/`, `/session`, `/settings`, `/schedules`) are Svelte 5 components built by Vite and served by one embedded shell (`internal/ui/embedded/app.html`). Sharing/export renders a separate self-contained snapshot (`internal/ui/export.go`). The pre-auth token prompt is the only other HTML page, rendered by the Go auth middleware.

5. **Security**: Token-based auth (`PI_WEB_TOKEN`) is required when binding to non-loopback addresses (e.g., Tailscale).
72 changes: 43 additions & 29 deletions docs/architecture/backend.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ pi-web/
│ │ ├── spa_page.go # Live SPA shell renderer (RenderAppShell)
│ │ ├── app_script.go # SPA Vite module URL path + script tag
│ │ ├── session_page.go # Session page data prep (bootstrap base64 + CSS)
│ │ ├── live_page.go # Live document shell + theme/font providers
│ │ ├── live_page.go # Shared live document head + theme/font providers
│ │ ├── export.go # Static export renderer
│ │ ├── auth_page.go # Auth/token entry page
│ │ ├── pwa.go # PWA routes: manifest, sw.js, icons, css, cat.webm
Expand All @@ -35,6 +35,8 @@ pi-web/
│ │ └── auth.go # Token-based HTTP middleware
│ ├── chat/
│ │ └── request.go # Multipart chat request parser (text + images)
│ ├── chatqueue/
│ │ └── chatqueue.go # SQLite-backed per-session chat queue (items + paused state)
│ ├── files/
│ │ └── files.go # Bounded read-only dir listing for @mention autocomplete
│ ├── render/
Expand All @@ -53,7 +55,10 @@ pi-web/
│ ├── server/
│ │ ├── server.go # Server type, deps, SSE registry, route registration, SQLite open
│ │ ├── handlers.go # index, session, api/session(s), new, fork/clone, rename, locations, models, custom-themes
│ │ ├── request.go # Shared JSON body decoding + request-size caps
│ │ ├── chat.go # Chat, set-model, set-thinking, worker-status, commands handlers
│ │ ├── chat_queue.go # /api/chat/queue handler (list/add/delete/pause)
│ │ ├── chat_queue_drainer.go # Autonomous queue dispatcher (running → idle, 5s tick, mutation kick)
│ │ ├── new_session.go # New-session creation logic
│ │ ├── git.go # /api/git/info, /api/git/rename-branch handlers
│ │ ├── diff.go # /api/git/diff, /api/diff/reviews handlers
Expand Down Expand Up @@ -119,28 +124,32 @@ type Server struct {
lastKnown map[string]struct{} // sessions currently broadcast as running
lastKnownMu sync.Mutex
push *PushManager // web-push subscriptions + done notifications
db *sql.DB // SQLite (~/.pi/agent/pi-web.sqlite)
stopCh chan struct{}
stopOnce sync.Once
wg sync.WaitGroup

taskMu sync.Mutex // shuts out new tasks and owns taskCtx
taskCtx context.Context // server lifecycle context for background work
taskCancel context.CancelFunc
stopping bool

db *sql.DB // SQLite (~/.pi/agent/pi-web.sqlite)
schedules *schedules.Store // cron definitions + run history
chatQueue *chatqueue.Store // per-session queue items + paused state
queueDrainer *queueDrainer // autonomous queue dispatcher
updater *updater.Checker // optional; nil disables /api/version etc.
runInstall func(ctx context.Context) error // optional self-update install
runRestart func() error // optional self-update restart
updateMu sync.Mutex // serializes install/restart
stopCh chan struct{}
stopOnce sync.Once
wg sync.WaitGroup
disableBackgroundJobs bool // development server: no scheduler/drainer/auto-title/push

fileWalk *fileWalkCache // bounded dir-listing cache for @mention autocomplete
fileWalkOnce sync.Once

startedAt time.Time // process uptime for the metrics dashboard
metricsSampler processSampler // swappable in tests
metricsCPUMu sync.Mutex
metricsCPULast map[int]cpuMark // per-PID CPU baselines for delta %CPU

titleMu sync.Mutex // auto-title bookkeeping (see auto_title.go)
titleInFlight map[string]bool
titledName map[string]string // sessID -> title pi-web last set
titledCount map[string]int // sessID -> user-msg count at last titling
titleUserOwned map[string]bool // sessID -> user named it; never auto-title
// Metrics dashboard (see metrics.go) and auto-title bookkeeping (see
// auto_title.go), grouped so each subsystem owns its fields + lock.
metrics metricsState
autoTitle autoTitleState
}
```

Expand All @@ -154,12 +163,16 @@ and serves the shared data, but does not run scheduling, queue draining,
auto-titling, or push-delivery side effects.

On `New`, the server opens (and migrates) a SQLite database at
`~/.pi/agent/pi-web.sqlite` with six tables: `scratchpads` (per project path),
`~/.pi/agent/pi-web.sqlite` with eleven tables: `scratchpads` (per project path),
`settings` (server-backed user settings key/value), `project_prefs` (which
projects are enabled), `app_settings` (the project-filter master switch, default
off), `btw_sessions` (the btw scratch-chat registry), and `annotations`
(per-session review notes keyed by session id; see `annotations.go`). See
`projects.go`, `settings.go`, and `btw.go`. The pool is capped to a single
off), `btw_sessions` (the btw scratch-chat registry), `annotations` (per-session
review notes keyed by session id; see `annotations.go`), `schedules` +
`schedule_runs` (definitions and firing history; see `internal/schedules`),
`review_comments` (per-session diff review notes; see `diff.go`), and
`chat_queue_items` + `chat_queue_state` (queued messages and per-session pause
state; see `internal/chatqueue`). See `projects.go`, `settings.go`, `btw.go`,
`schedules.go`, and `chat_queue.go`. The pool is capped to a single
connection (`SetMaxOpenConns(1)`) so concurrent writers queue instead of failing
with "database is locked". A `PushManager` (when configured) persists web-push
subscriptions and VAPID keys under the agent dir.
Expand Down Expand Up @@ -200,13 +213,14 @@ Manages `pi --mode rpc` subprocesses per session.

```go
type Manager struct {
mu sync.Mutex
workers map[string]ChatWorker // sessionID → worker
creating map[string]*createCall // single-flight: coalesce concurrent creates per session
factory Factory // (sessionID, sessionPath) → ChatWorker
idleTTL time.Duration // default 10m
reaperStop chan struct{}
reaperDone chan struct{}
mu sync.Mutex
workers map[string]ChatWorker // sessionID → worker
creating map[string]*createCall // single-flight: coalesce concurrent creates per session
factory Factory // (sessionID, sessionPath) → ChatWorker
pendingSends map[string]int // accepted Sends not yet acked; keeps Status from dipping to idle
idleTTL time.Duration // default 10m
reaperStop chan struct{}
reaperDone chan struct{}
}
```

Expand Down Expand Up @@ -250,11 +264,11 @@ type piRPCWorker struct {
| `/` | GET | `handleIndex` | Render SPA shell for the sessions route |
| `/session` | GET | `handleSession` | Render SPA shell for the session route |
| `/settings` | GET | `handleSettingsPage` | Render SPA shell for the settings route |
| `/login` | GET | `handleAppShell` | Render SPA shell for the login route |
| `/api/session` | GET | `handleApiSession` | JSON session data |
| `/api/sessions` | GET | `handleApiSessions` | JSON list of session summaries |
| `/api/chat` | POST | `handleChat` | Send chat message (multipart) |
| `/api/chat/cancel` | POST | `handleCancelChat` | Abort running chat worker |
| `/api/chat/queue` | GET/POST/DELETE/PATCH | `handleChatQueue` | Per-session queued messages + pause state (SQLite) |
| `/api/set-model` | POST | `handleSetModel` | Change model for session |
| `/api/set-thinking-level` | POST | `handleSetThinkingLevel` | Change thinking level |
| `/api/models` | GET | `handleAvailableModels` | List available AI models |
Expand Down Expand Up @@ -301,7 +315,7 @@ PWA / static asset routes (registered outside `Server.Register`):

| Route | Source |
|-------|--------|
| `/manifest.webmanifest`, `/sw.js`, `/icon.svg`, `/icon-maskable.svg`, `/pi-logo.svg`, `/cat.webm`, `/theme.css`, `/index.css`, `/menu.css`, `/palette.css` | `internal/ui/pwa.go` (embedded assets) |
| `/manifest.webmanifest`, `/sw.js`, `/icon.svg`, `/icon-maskable.svg`, `/pi-logo.svg`, `/cat.webm` | `internal/ui/pwa.go` (embedded assets) |
| `/static/assets/app-*.js`, `/static/assets/...` | Embedded Vite SPA bundle and chunks (`internal/app/app.go` + `internal/frontend`) |

## Auth Flow
Expand Down Expand Up @@ -386,6 +400,6 @@ Three signals are OR'd together to determine if a session is "running":

1. **session-status file** (`~/.pi/agent/session-status/<id>`): written by the terminal pi process
2. **In-process chat worker**: `chatSender.Status(id).State == running`
3. **Recent file activity**: modtime within 3 seconds
3. **Recent file activity**: JSONL file modtime within 800 ms

Status changes are broadcast as SSE `status-delta` events to `__all__` subscribers. A 1-second sweeper periodically revalidates all known running sessions to clean up stale states.
8 changes: 6 additions & 2 deletions docs/architecture/data-flow.md
Original file line number Diff line number Diff line change
Expand Up @@ -140,7 +140,7 @@ Browser POST /api/chat?id=<id>
│ ├──▶ Await response on pending channel
│ └──▶ Update status → running
└──▶ Return {"ok": true, "status": "accepted"}
└──▶ Return {"ok": true, "status": "queued"}
```

## Data Flow: Rename Session
Expand All @@ -160,7 +160,11 @@ Browser POST /api/rename-session?id=<id>
└──▶ Return {"ok": true, "name": "New Name"}
```

Rename is the only intentional pi-web write to an existing session JSONL file. It appends metadata history; it does not rewrite existing entries. Creating a new session is the other direct write path, but it only creates a fresh JSONL file.
pi-web only ever **appends** metadata to an existing session JSONL file — it never
rewrites existing entries. There are three append paths: browser rename
(`session_info`), auto-titling (`session_info`, marked so a user rename always
wins), and entry labels (`label`). Creating new sessions (including btw, fork,
clone, and schedule runs) writes fresh JSONL files instead.

## Data Flow: Live Reload

Expand Down
Loading
Loading