diff --git a/.env.example b/.env.example index e0ec46b2..6da63013 100644 --- a/.env.example +++ b/.env.example @@ -107,6 +107,8 @@ TZ=UTC # Byte budget for images returned by vault_read_asset, in binary bytes before # base64 encoding (default: 49152 = 48 KiB, sized for Claude Code's response cap). # MAX_IMAGE_OUTPUT_BYTES=49152 +# Max PDF pages to render as images for raw: true (default: 5). +# MAX_PDF_RENDER_PAGES=5 # Enable or disable the memory layer (default: true). # Set to false to hide memory tools, skip auto-initialization, and omit # memory references from server metadata. MEMORY_DIR is ignored when false. diff --git a/.github/workflows/trivy.yml b/.github/workflows/trivy.yml index e92920c8..7b7d4730 100644 --- a/.github/workflows/trivy.yml +++ b/.github/workflows/trivy.yml @@ -193,7 +193,7 @@ jobs: uses: github/codeql-action/upload-sarif@7188fc363630916deb702c7fdcf4e481b751f97a # v4.37.1 with: sarif_file: trivy-results.sarif - category: trivy-pr-image + category: trivy-published-image # hashFiles guard: the remote SARIF doesn't exist when the local scan # gates the job first (exit-code 1) or the remote build fails. @@ -202,4 +202,4 @@ jobs: uses: github/codeql-action/upload-sarif@7188fc363630916deb702c7fdcf4e481b751f97a # v4.37.1 with: sarif_file: trivy-results-remote.sarif - category: trivy-pr-image-remote + category: trivy-published-image-remote diff --git a/AGENTS.md b/AGENTS.md index b80a8676..b547defb 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -858,7 +858,7 @@ pattern: 6. **Root compose files** (`docker-compose.yml`, `docker-compose.local.yml`) — maintainer/contributor surfaces (if applicable) -CI drift tests in `templates.test.ts` catch omissions across steps 2–4, +CI drift tests in `cli/src/__tests__/templates.test.ts` catch omissions across steps 2–4, but the checklist prevents them. **Regenerating `social-preview.png`:** Run `npm run render:social-preview`. diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index ead42118..e97eda5b 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -296,7 +296,7 @@ Link queries use a `links` table populated during indexing: 1. **Images** (`.png`/`.jpg`/`.jpeg`/`.gif`/`.webp`) return an MCP `image` content block plus a one-line metadata text block. A shared fit-to-byte-budget pipeline (`utils/fit-image-to-byte-budget.ts`, built on sharp) makes oversized images deliverable: EXIF auto-orient → resize long edge to ≤1568px → walk a fixed quality ladder (JPEG via mozjpeg for opaque images, WebP for alpha — PNG has no quality knob) → shrink dimensions by √(budget/actual) if the ladder floor still exceeds the budget. Deterministic and terminating (bounded attempts, 64px floor); sharp's default `limitInputPixels` stays active as the decompression-bomb guard. The budget (`MAX_IMAGE_OUTPUT_BYTES`, default 48 KiB binary) is sized for the tightest mainstream client cap. 2. **Canvas** (`.canvas`) linearizes to markdown via the pure `obsidian-markdown/canvas.ts` parser ([JSON Canvas 1.0](https://jsoncanvas.org)): group membership by spatial rect containment (innermost group wins; equal rects tiebreak deterministically by id), nodes in reading order (y, then x), and an edge list with node ids resolved to display names. Lenient parsing — unknown properties ignored, malformed entries skipped. `raw: true` skips the linearizer and returns the JSON source verbatim for full structural fidelity. 3. **Text formats** (`.svg`/`.json`/`.txt`/`.csv`/`.xml`/`.log`/`.base`) pass through verbatim as text, capped at a fixed 100 KiB output size (explicit error over silent truncation). -4. **PDFs** (`.pdf`) return structured markdown reconstructed from layout-aware extraction (`unpdf`, based on Mozilla's PDF.js). A document metadata header (title, page count, link count), heading hierarchy inferred from relative font sizes, fenced code blocks detected via monospace fonts, page separators, and a deduplicated links footer. Scanned or image-only PDFs with no extractable text return an error stating the file's existence, size, and page count. +4. **PDFs** (`.pdf`) return structured markdown reconstructed from layout-aware extraction (`unpdf`, based on Mozilla's PDF.js). A document metadata header (title, page count, link count), heading hierarchy inferred from relative font sizes, fenced code blocks detected via monospace fonts, page separators, and a deduplicated links footer. `raw: true` switches to page-image mode: each page is rendered at 2× scale via `unpdf`'s `renderPageAsImage` with `@napi-rs/canvas` (prebuilt Skia, no system deps), then fitted through the same byte-budget pipeline as regular images. The total image budget is divided evenly across rendered pages (capped at `MAX_PDF_RENDER_PAGES`, default 5). Scanned or image-only PDFs with no extractable text work in raw mode — the model's own vision handles recognition. 5. **Unknown types** return an error naming the readable set. The extension-to-representation routing above is implemented by the `vault-operations/asset-operations.ts` use-case. Beneath it, every read goes through `vaultFs.readAsset`, which applies the same `resolveSafePath` traversal guard as notes, rejects `.md` paths (notes belong to `vault_read_note`), and enforces a stat-before-read size cap (`MAX_ASSET_BYTES`, default 50 MiB). diff --git a/DOCKERHUB.md b/DOCKERHUB.md index 8979836e..24e6c0e9 100644 --- a/DOCKERHUB.md +++ b/DOCKERHUB.md @@ -65,7 +65,7 @@ Your notes embed screenshots, reference architecture diagrams, and link out to c - **Images** — the image itself, not the filename. Screenshots and diagrams are downscaled and recompressed server-side when they exceed what MCP clients accept, so even a phone session can look at a 5MB architecture diagram - **Canvases** — a [Canvas](https://help.obsidian.md/canvas) board arrives as a readable outline: its groups, each card's content in reading order, and the connections between them. The exact JSON source is one flag away when full fidelity matters -- **PDFs** — text is extracted from the document's content streams; scanned or image-only PDFs return an error stating the page count +- **PDFs** — text is extracted with heading hierarchy, code blocks, and hyperlinks preserved; set `raw: true` to render pages as images instead, showing layout, diagrams, and tables that text extraction can't preserve — scanned and image-only PDFs work in this mode - **Text and data files** — SVG, JSON, CSV, logs, and [Bases](https://help.obsidian.md/bases) files return exactly as written - **Browse** — list any folder's assets with per-extension counts and file sizes; assets a note links to report their size in the link graph too @@ -136,25 +136,26 @@ Vault Cortex indexes every [property](https://help.obsidian.md/Editing+and+forma All settings are environment variables with sensible defaults. -| Variable | Required? | Default | Description | -| --------------------------- | ----------- | ------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `MCP_AUTH_TOKEN` | Yes | — | Bearer token for authentication (also the JWT signing key) | -| `VAULT_PATH` | Local only | — | Host path to your vault (bind mount source; remote uses a named volume) | -| `PUBLIC_URL` | Remote only | — | Public URL for OAuth discovery metadata | -| `EMBEDDING_ENABLED` | — | `true` | Set `false` to disable the embedding pipeline — skips model download, vector tables, embedding passes, and hybrid search. Search falls back to FTS5 keyword matching. | -| `RERANK_MODE` | — | `blended` | Cross-encoder reranking mode: `blended` applies position-aware score blending after RRF fusion (~200ms added latency), `none` skips reranking. Only takes effect when `EMBEDDING_ENABLED` is true. | -| `MEMORY_ENABLED` | — | `true` | Set `false` to fully disable the memory layer — hides memory tools, skips bootstrap, omits memory from server metadata. `MEMORY_DIR` is ignored when `false`. | -| `MEMORY_DIR` | — | `About Me` | Vault folder for structured memory files | -| `PROTECTED_PATHS` | — | `MEMORY_DIR, Daily Notes` | Folders that `vault_delete_note` refuses to touch | -| `ORPHAN_EXCLUDE_FOLDERS` | — | `Daily Notes, Templates, MEMORY_DIR` | Folders excluded from orphan detection | -| `TZ` | — | `UTC` | IANA timezone for timestamps and daily note resolution | -| `SERVICE_DOCUMENTATION_URL` | — | GitHub repo URL | URL returned in OAuth discovery metadata | -| `LOG_LEVEL` | — | `info` | Logging verbosity: `debug`, `info`, `warn`, `error` | -| `LOG_DIR` | — | `/data/logs` (remote), unset (local) | Directory for persistent log files. When set, logs are written to date-stamped files there alongside stdout. Unset means stdout only. | -| `LOG_RETENTION_DAYS` | — | `30` | Days to keep log files before automatic cleanup on startup | -| `WINDOWS_MODE` | — | `false` | On Windows? Set `true`. Switches the file watcher to polling and note moves to rename-based writes so a vault on a `C:` drive works through Docker Desktop. Safe to leave on for any Windows setup; unneeded on macOS/Linux/WSL2. | -| `MAX_ASSET_BYTES` | — | `52428800` (50 MiB) | Maximum file size `vault_read_asset` will read (in bytes). Files exceeding this are rejected before reading. Raise for vaults with very large individual files. | +| Variable | Required? | Default | Description | +| --------------------------- | ----------- | ------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `MCP_AUTH_TOKEN` | Yes | — | Bearer token for authentication (also the JWT signing key) | +| `VAULT_PATH` | Local only | — | Host path to your vault (bind mount source; remote uses a named volume) | +| `PUBLIC_URL` | Remote only | — | Public URL for OAuth discovery metadata | +| `EMBEDDING_ENABLED` | — | `true` | Set `false` to disable the embedding pipeline — skips model download, vector tables, embedding passes, and hybrid search. Search falls back to FTS5 keyword matching. | +| `RERANK_MODE` | — | `blended` | Cross-encoder reranking mode: `blended` applies position-aware score blending after RRF fusion (~200ms added latency), `none` skips reranking. Only takes effect when `EMBEDDING_ENABLED` is true. | +| `MEMORY_ENABLED` | — | `true` | Set `false` to fully disable the memory layer — hides memory tools, skips bootstrap, omits memory from server metadata. `MEMORY_DIR` is ignored when `false`. | +| `MEMORY_DIR` | — | `About Me` | Vault folder for structured memory files | +| `PROTECTED_PATHS` | — | `MEMORY_DIR, Daily Notes` | Folders that `vault_delete_note` refuses to touch | +| `ORPHAN_EXCLUDE_FOLDERS` | — | `Daily Notes, Templates, MEMORY_DIR` | Folders excluded from orphan detection | +| `TZ` | — | `UTC` | IANA timezone for timestamps and daily note resolution | +| `SERVICE_DOCUMENTATION_URL` | — | GitHub repo URL | URL returned in OAuth discovery metadata | +| `LOG_LEVEL` | — | `info` | Logging verbosity: `debug`, `info`, `warn`, `error` | +| `LOG_DIR` | — | `/data/logs` (remote), unset (local) | Directory for persistent log files. When set, logs are written to date-stamped files there alongside stdout. Unset means stdout only. | +| `LOG_RETENTION_DAYS` | — | `30` | Days to keep log files before automatic cleanup on startup | +| `WINDOWS_MODE` | — | `false` | On Windows? Set `true`. Switches the file watcher to polling and note moves to rename-based writes so a vault on a `C:` drive works through Docker Desktop. Safe to leave on for any Windows setup; unneeded on macOS/Linux/WSL2. | +| `MAX_ASSET_BYTES` | — | `52428800` (50 MiB) | Maximum file size `vault_read_asset` will read (in bytes). Files exceeding this are rejected before reading. Raise for vaults with very large individual files. | | `MAX_IMAGE_OUTPUT_BYTES` | — | `49152` (48 KiB) | Byte budget for images delivered by `vault_read_asset`, in binary bytes before base64 encoding. Images exceeding this are downscaled and recompressed to fit. Sized for the tightest mainstream MCP client cap; raise for clients that accept larger responses. | +| `MAX_PDF_RENDER_PAGES` | — | `5` | Maximum PDF pages to render as images when `raw: true` is set on `vault_read_asset`. The per-page byte budget is `MAX_IMAGE_OUTPUT_BYTES` divided evenly across the rendered pages — fewer pages means higher quality each. | ## Deployment Options diff --git a/README.md b/README.md index 2c13eb58..083368b8 100644 --- a/README.md +++ b/README.md @@ -219,7 +219,7 @@ Your notes embed screenshots, reference architecture diagrams, and link out to c - **Images** — the image itself, not the filename. Screenshots and diagrams are downscaled and recompressed server-side when they exceed what MCP clients accept, so even a phone session can look at a 5MB architecture diagram - **Canvases** — a [Canvas](https://help.obsidian.md/canvas) board arrives as a readable outline: its groups, each card's content in reading order, and the connections between them. The exact JSON source is one flag away when full fidelity matters -- **PDFs** — text is extracted from the document's content streams; scanned or image-only PDFs return an error stating the page count +- **PDFs** — text is extracted with heading hierarchy, code blocks, and hyperlinks preserved; set `raw: true` to render pages as images instead, showing layout, diagrams, and tables that text extraction can't preserve — scanned and image-only PDFs work in this mode - **Text and data files** — SVG, JSON, CSV, logs, and [Bases](https://help.obsidian.md/bases) files return exactly as written - **Browse** — list any folder's assets with per-extension counts and file sizes; assets a note links to report their size in the link graph too @@ -315,6 +315,7 @@ All settings are environment variables with sensible defaults. | `WINDOWS_MODE` | — | `false` | On Windows? Set `true`. Switches the file watcher to polling and note moves to rename-based writes so a vault on a `C:` drive works through Docker Desktop. Safe to leave on for any Windows setup; unneeded on macOS/Linux/WSL2. | | `MAX_ASSET_BYTES` | — | `52428800` (50 MiB) | Maximum file size `vault_read_asset` will read (in bytes). Files exceeding this are rejected before reading. Raise for vaults with very large individual files. | | `MAX_IMAGE_OUTPUT_BYTES` | — | `49152` (48 KiB) | Byte budget for images delivered by `vault_read_asset`, in binary bytes before base64 encoding. Images exceeding this are downscaled and recompressed to fit. Sized for the tightest mainstream MCP client cap; raise for clients that accept larger responses. | +| `MAX_PDF_RENDER_PAGES` | — | `5` | Maximum PDF pages to render as images when `raw: true` is set on `vault_read_asset`. The per-page byte budget is `MAX_IMAGE_OUTPUT_BYTES` divided evenly across the rendered pages — fewer pages means higher quality each. | **Smart defaults:** Setting `MEMORY_DIR` automatically updates the defaults for `PROTECTED_PATHS` and `ORPHAN_EXCLUDE_FOLDERS`. You only set those explicitly for a fully custom list. When `MEMORY_ENABLED` is `false`, the memory layer is fully disabled — memory tools are hidden and the memory folder is not auto-created. diff --git a/cli/src/env.ts b/cli/src/env.ts index bf52788f..97c35e88 100644 --- a/cli/src/env.ts +++ b/cli/src/env.ts @@ -44,6 +44,11 @@ MAX_ASSET_BYTES=52428800 # that accept larger tool responses. MAX_IMAGE_OUTPUT_BYTES=49152 +# Maximum number of PDF pages to render as images when raw: true is set on +# vault_read_asset (default: 5). The per-page byte budget is MAX_IMAGE_OUTPUT_BYTES +# divided evenly across the rendered pages. Fewer pages = higher quality each. +MAX_PDF_RENDER_PAGES=5 + # Your IANA timezone — affects daily note resolution and memory timestamps. # TZ=America/New_York @@ -148,6 +153,11 @@ MAX_ASSET_BYTES=52428800 # that accept larger tool responses. MAX_IMAGE_OUTPUT_BYTES=49152 +# Maximum number of PDF pages to render as images when raw: true is set on +# vault_read_asset (default: 5). The per-page byte budget is MAX_IMAGE_OUTPUT_BYTES +# divided evenly across the rendered pages. Fewer pages = higher quality each. +MAX_PDF_RENDER_PAGES=5 + # Enable or disable the memory layer (default: true). # Set to false to hide memory tools and skip About Me/ creation. MEMORY_ENABLED=true diff --git a/deploy/local/.env.example b/deploy/local/.env.example index 500fe76c..ff0e8fb7 100644 --- a/deploy/local/.env.example +++ b/deploy/local/.env.example @@ -27,6 +27,11 @@ MAX_ASSET_BYTES=52428800 # that accept larger tool responses. MAX_IMAGE_OUTPUT_BYTES=49152 +# Maximum number of PDF pages to render as images when raw: true is set on +# vault_read_asset (default: 5). The per-page byte budget is MAX_IMAGE_OUTPUT_BYTES +# divided evenly across the rendered pages. Fewer pages = higher quality each. +MAX_PDF_RENDER_PAGES=5 + # Your IANA timezone — affects daily note resolution and memory timestamps. # TZ=America/New_York diff --git a/deploy/local/docker-compose.yml b/deploy/local/docker-compose.yml index bcfb9d95..6f772138 100644 --- a/deploy/local/docker-compose.yml +++ b/deploy/local/docker-compose.yml @@ -38,6 +38,7 @@ services: WINDOWS_MODE: ${WINDOWS_MODE:-false} MAX_ASSET_BYTES: ${MAX_ASSET_BYTES:-52428800} MAX_IMAGE_OUTPUT_BYTES: ${MAX_IMAGE_OUTPUT_BYTES:-49152} + MAX_PDF_RENDER_PAGES: ${MAX_PDF_RENDER_PAGES:-5} # Optional overrides. When unset, the server applies smart defaults # ( below is the resolved MEMORY_DIR value, default "About Me"): # PROTECTED_PATHS default: ", Daily Notes" (blocked from vault_delete_note) diff --git a/deploy/remote/.env.example b/deploy/remote/.env.example index a16c75f5..98bd91ae 100644 --- a/deploy/remote/.env.example +++ b/deploy/remote/.env.example @@ -59,6 +59,11 @@ MAX_ASSET_BYTES=52428800 # that accept larger tool responses. MAX_IMAGE_OUTPUT_BYTES=49152 +# Maximum number of PDF pages to render as images when raw: true is set on +# vault_read_asset (default: 5). The per-page byte budget is MAX_IMAGE_OUTPUT_BYTES +# divided evenly across the rendered pages. Fewer pages = higher quality each. +MAX_PDF_RENDER_PAGES=5 + # Enable or disable the memory layer (default: true). # Set to false to hide memory tools and skip About Me/ creation. MEMORY_ENABLED=true diff --git a/deploy/remote/docker-compose.yml b/deploy/remote/docker-compose.yml index 08a5fadb..951c2470 100644 --- a/deploy/remote/docker-compose.yml +++ b/deploy/remote/docker-compose.yml @@ -49,6 +49,7 @@ services: WINDOWS_MODE: ${WINDOWS_MODE:-false} MAX_ASSET_BYTES: ${MAX_ASSET_BYTES:-52428800} MAX_IMAGE_OUTPUT_BYTES: ${MAX_IMAGE_OUTPUT_BYTES:-49152} + MAX_PDF_RENDER_PAGES: ${MAX_PDF_RENDER_PAGES:-5} # Optional overrides. When unset, the server applies smart defaults # ( below is the resolved MEMORY_DIR value, default "About Me"): # PROTECTED_PATHS default: ", Daily Notes" (blocked from vault_delete_note) diff --git a/docker-compose.local.yml b/docker-compose.local.yml index 8821df8a..de757ae9 100644 --- a/docker-compose.local.yml +++ b/docker-compose.local.yml @@ -34,6 +34,7 @@ services: WINDOWS_MODE: ${WINDOWS_MODE:-false} MAX_ASSET_BYTES: ${MAX_ASSET_BYTES:-52428800} MAX_IMAGE_OUTPUT_BYTES: ${MAX_IMAGE_OUTPUT_BYTES:-49152} + MAX_PDF_RENDER_PAGES: ${MAX_PDF_RENDER_PAGES:-5} # Left empty = the server applies smart defaults # ( below is the resolved MEMORY_DIR value, default "About Me"): # PROTECTED_PATHS default: ", Daily Notes" diff --git a/docker-compose.yml b/docker-compose.yml index f65e1dc2..7e6166f4 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -47,6 +47,7 @@ services: WINDOWS_MODE: ${WINDOWS_MODE:-false} MAX_ASSET_BYTES: ${MAX_ASSET_BYTES:-52428800} MAX_IMAGE_OUTPUT_BYTES: ${MAX_IMAGE_OUTPUT_BYTES:-49152} + MAX_PDF_RENDER_PAGES: ${MAX_PDF_RENDER_PAGES:-5} MEMORY_DIR: ${MEMORY_DIR:-About Me} # Left empty = the server applies smart defaults # ( below is the resolved MEMORY_DIR value, default "About Me"): diff --git a/package-lock.json b/package-lock.json index 9983c90a..837b5217 100644 --- a/package-lock.json +++ b/package-lock.json @@ -11,6 +11,7 @@ "dependencies": { "@huggingface/transformers": "4.2.0", "@modelcontextprotocol/sdk": "1.29.0", + "@napi-rs/canvas": "0.1.100", "better-sqlite3": "12.11.1", "chokidar": "5.0.0", "env-var": "7.5.0", @@ -2076,6 +2077,270 @@ } } }, + "node_modules/@napi-rs/canvas": { + "version": "0.1.100", + "resolved": "https://registry.npmjs.org/@napi-rs/canvas/-/canvas-0.1.100.tgz", + "integrity": "sha512-xglYA6q3XO5P3BNJYxVZ1IV7DLVjp1Py6nwag88YntrS+3vKHyYcMqXVS4ZztJmwz2uGvz1FWhI/4LgbR5uQDA==", + "license": "MIT", + "workspaces": [ + "e2e/*" + ], + "engines": { + "node": ">= 10" + }, + "funding": { + "type": "github", + "url": "https://github.com/sponsors/Brooooooklyn" + }, + "optionalDependencies": { + "@napi-rs/canvas-android-arm64": "0.1.100", + "@napi-rs/canvas-darwin-arm64": "0.1.100", + "@napi-rs/canvas-darwin-x64": "0.1.100", + "@napi-rs/canvas-linux-arm-gnueabihf": "0.1.100", + "@napi-rs/canvas-linux-arm64-gnu": "0.1.100", + "@napi-rs/canvas-linux-arm64-musl": "0.1.100", + "@napi-rs/canvas-linux-riscv64-gnu": "0.1.100", + "@napi-rs/canvas-linux-x64-gnu": "0.1.100", + "@napi-rs/canvas-linux-x64-musl": "0.1.100", + "@napi-rs/canvas-win32-arm64-msvc": "0.1.100", + "@napi-rs/canvas-win32-x64-msvc": "0.1.100" + } + }, + "node_modules/@napi-rs/canvas-android-arm64": { + "version": "0.1.100", + "resolved": "https://registry.npmjs.org/@napi-rs/canvas-android-arm64/-/canvas-android-arm64-0.1.100.tgz", + "integrity": "sha512-hjhCKhntPv9+t4ckHymdx0phYNcVW+GKQR6Lzw2zE+pOVjOplSmtx9nNNknTjbEDLcuLZqA1y8ufKg1XfgftzQ==", + "cpu": [ + "arm64" + ], + "license": "MIT", + "optional": true, + "os": [ + "android" + ], + "engines": { + "node": ">= 10" + }, + "funding": { + "type": "github", + "url": "https://github.com/sponsors/Brooooooklyn" + } + }, + "node_modules/@napi-rs/canvas-darwin-arm64": { + "version": "0.1.100", + "resolved": "https://registry.npmjs.org/@napi-rs/canvas-darwin-arm64/-/canvas-darwin-arm64-0.1.100.tgz", + "integrity": "sha512-2PcswRaC7Ly645DGt88///zuFDhJxJYdKAs1uU3mfk1atYkXufgcgLfBpk6Tm12nCQBaNt1wpybuPZ4qOhTo8A==", + "cpu": [ + "arm64" + ], + "license": "MIT", + "optional": true, + "os": [ + "darwin" + ], + "engines": { + "node": ">= 10" + }, + "funding": { + "type": "github", + "url": "https://github.com/sponsors/Brooooooklyn" + } + }, + "node_modules/@napi-rs/canvas-darwin-x64": { + "version": "0.1.100", + "resolved": "https://registry.npmjs.org/@napi-rs/canvas-darwin-x64/-/canvas-darwin-x64-0.1.100.tgz", + "integrity": "sha512-ePNZtj7pNIva/siZMg+HmbeozkIjqUIYdoymH8HaA3qK7LfzFN4WMBM8G6HQ9ZC+H3+Dnn5pqtiXpgLykaPOhw==", + "cpu": [ + "x64" + ], + "license": "MIT", + "optional": true, + "os": [ + "darwin" + ], + "engines": { + "node": ">= 10" + }, + "funding": { + "type": "github", + "url": "https://github.com/sponsors/Brooooooklyn" + } + }, + "node_modules/@napi-rs/canvas-linux-arm-gnueabihf": { + "version": "0.1.100", + "resolved": "https://registry.npmjs.org/@napi-rs/canvas-linux-arm-gnueabihf/-/canvas-linux-arm-gnueabihf-0.1.100.tgz", + "integrity": "sha512-d5cDB48oWFGU8/XPhUOFAlySgb/VAu7D+s8fi55K1Pcfg8aPplHWqMgibhVLU8ky7Pyg/fuiVLz4Nf3JrSTuUA==", + "cpu": [ + "arm" + ], + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">= 10" + }, + "funding": { + "type": "github", + "url": "https://github.com/sponsors/Brooooooklyn" + } + }, + "node_modules/@napi-rs/canvas-linux-arm64-gnu": { + "version": "0.1.100", + "resolved": "https://registry.npmjs.org/@napi-rs/canvas-linux-arm64-gnu/-/canvas-linux-arm64-gnu-0.1.100.tgz", + "integrity": "sha512-rDxgxRu69RvDlX/bh9o22DxLsGr8EqsNgotL9+RwQE1S0b0cqeatqsw6aW45mukm0B42DIAaAacKaYQ8cqS1nw==", + "cpu": [ + "arm64" + ], + "libc": [ + "glibc" + ], + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">= 10" + }, + "funding": { + "type": "github", + "url": "https://github.com/sponsors/Brooooooklyn" + } + }, + "node_modules/@napi-rs/canvas-linux-arm64-musl": { + "version": "0.1.100", + "resolved": "https://registry.npmjs.org/@napi-rs/canvas-linux-arm64-musl/-/canvas-linux-arm64-musl-0.1.100.tgz", + "integrity": "sha512-K3mDW66N+xT2/V439u1alFANiBUjdEx2gLiNYnCmUsva5jZMxWTjafBYwTzYK+EMFMHrUoabuU+T1BIP5CgbYQ==", + "cpu": [ + "arm64" + ], + "libc": [ + "musl" + ], + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">= 10" + }, + "funding": { + "type": "github", + "url": "https://github.com/sponsors/Brooooooklyn" + } + }, + "node_modules/@napi-rs/canvas-linux-riscv64-gnu": { + "version": "0.1.100", + "resolved": "https://registry.npmjs.org/@napi-rs/canvas-linux-riscv64-gnu/-/canvas-linux-riscv64-gnu-0.1.100.tgz", + "integrity": "sha512-mooqUBTIsccZpnoQC4NgrC1v6C1vof39etLNMnBwCY+p0gajWJvAHLGQ6g/gGyS5YrpDW+GefSN4+Cvcr08UWw==", + "cpu": [ + "riscv64" + ], + "libc": [ + "glibc" + ], + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">= 10" + }, + "funding": { + "type": "github", + "url": "https://github.com/sponsors/Brooooooklyn" + } + }, + "node_modules/@napi-rs/canvas-linux-x64-gnu": { + "version": "0.1.100", + "resolved": "https://registry.npmjs.org/@napi-rs/canvas-linux-x64-gnu/-/canvas-linux-x64-gnu-0.1.100.tgz", + "integrity": "sha512-1eCvkDCazm7FFhsT7DfGOdSaHgZVK3bt/dSBl5EWHOWmnz+I7j8tPseJqqD81NF+MH21jKUK4wQSDjN0mdhnTg==", + "cpu": [ + "x64" + ], + "libc": [ + "glibc" + ], + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">= 10" + }, + "funding": { + "type": "github", + "url": "https://github.com/sponsors/Brooooooklyn" + } + }, + "node_modules/@napi-rs/canvas-linux-x64-musl": { + "version": "0.1.100", + "resolved": "https://registry.npmjs.org/@napi-rs/canvas-linux-x64-musl/-/canvas-linux-x64-musl-0.1.100.tgz", + "integrity": "sha512-20arT6lnI19S68qNlii73TSEDbECNgzMz2EpldC1V3mZFuRkeujXkcebRk0LRJe9SEUAooYiLokfMViY8IX7yA==", + "cpu": [ + "x64" + ], + "libc": [ + "musl" + ], + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">= 10" + }, + "funding": { + "type": "github", + "url": "https://github.com/sponsors/Brooooooklyn" + } + }, + "node_modules/@napi-rs/canvas-win32-arm64-msvc": { + "version": "0.1.100", + "resolved": "https://registry.npmjs.org/@napi-rs/canvas-win32-arm64-msvc/-/canvas-win32-arm64-msvc-0.1.100.tgz", + "integrity": "sha512-DZFFT1wIAg37LJw37yhMRFfjATd3vTQzjZ1Yki8u2vhO6Hi5VE6BVaGQ1aaDu7xb4iMErz+9EOwjpS7xcxFeBw==", + "cpu": [ + "arm64" + ], + "license": "MIT", + "optional": true, + "os": [ + "win32" + ], + "engines": { + "node": ">= 10" + }, + "funding": { + "type": "github", + "url": "https://github.com/sponsors/Brooooooklyn" + } + }, + "node_modules/@napi-rs/canvas-win32-x64-msvc": { + "version": "0.1.100", + "resolved": "https://registry.npmjs.org/@napi-rs/canvas-win32-x64-msvc/-/canvas-win32-x64-msvc-0.1.100.tgz", + "integrity": "sha512-MyT1j3mHC2+Lu4pBi9mKyMJhtP6U7k7EldY7sj/uS5gJA65gTXt8MefJQXLJo5d/vZbuWmfxzkEUNc/urV3pHA==", + "cpu": [ + "x64" + ], + "license": "MIT", + "optional": true, + "os": [ + "win32" + ], + "engines": { + "node": ">= 10" + }, + "funding": { + "type": "github", + "url": "https://github.com/sponsors/Brooooooklyn" + } + }, "node_modules/@napi-rs/wasm-runtime": { "version": "1.1.6", "resolved": "https://registry.npmjs.org/@napi-rs/wasm-runtime/-/wasm-runtime-1.1.6.tgz", @@ -3844,20 +4109,20 @@ } }, "node_modules/body-parser": { - "version": "2.2.2", - "resolved": "https://registry.npmjs.org/body-parser/-/body-parser-2.2.2.tgz", - "integrity": "sha512-oP5VkATKlNwcgvxi0vM0p/D3n2C3EReYVX+DNYs5TjZFn/oQt2j+4sVJtSMr18pdRr8wjTcBl6LoV+FUwzPmNA==", + "version": "2.3.0", + "resolved": "https://registry.npmjs.org/body-parser/-/body-parser-2.3.0.tgz", + "integrity": "sha512-2cGmJupaNgg+QUwVLAucDuWuoMZ6EX9iHDRswZ5lsNYEmwPaRknMPCLZz07yTzVq/83p4o/wzbDZbBrTvGGTIw==", "license": "MIT", "dependencies": { "bytes": "^3.1.2", - "content-type": "^1.0.5", + "content-type": "^2.0.0", "debug": "^4.4.3", - "http-errors": "^2.0.0", - "iconv-lite": "^0.7.0", + "http-errors": "^2.0.1", + "iconv-lite": "^0.7.2", "on-finished": "^2.4.1", - "qs": "^6.14.1", - "raw-body": "^3.0.1", - "type-is": "^2.0.1" + "qs": "^6.15.2", + "raw-body": "^3.0.2", + "type-is": "^2.1.0" }, "engines": { "node": ">=18" @@ -3867,6 +4132,19 @@ "url": "https://opencollective.com/express" } }, + "node_modules/body-parser/node_modules/content-type": { + "version": "2.0.0", + "resolved": "https://registry.npmjs.org/content-type/-/content-type-2.0.0.tgz", + "integrity": "sha512-j/O/d7GcZCyNl7/hwZAb606rzqkyvaDctLmckbxLzHvFBzTJHuGEdodATcP3yIRoDrLHkIATJuvzbFlp/ki2cQ==", + "license": "MIT", + "engines": { + "node": ">=18" + }, + "funding": { + "type": "opencollective", + "url": "https://opencollective.com/express" + } + }, "node_modules/boolean": { "version": "3.2.0", "resolved": "https://registry.npmjs.org/boolean/-/boolean-3.2.0.tgz", @@ -7655,17 +7933,34 @@ } }, "node_modules/type-is": { - "version": "2.0.1", - "resolved": "https://registry.npmjs.org/type-is/-/type-is-2.0.1.tgz", - "integrity": "sha512-OZs6gsjF4vMp32qrCbiVSkrFmXtG/AZhY3t0iAMrMBiAZyV9oALtXO8hsrHbMXF9x6L3grlFuwW2oAz7cav+Gw==", + "version": "2.1.0", + "resolved": "https://registry.npmjs.org/type-is/-/type-is-2.1.0.tgz", + "integrity": "sha512-faYHw0anBbc/kWF3zFTEnxSFOAGUX9GFbOBthvDdLsIlEoWOFOtS0zgCiQYwIskL9iGXZL3kAXD8OoZ4GmMATA==", "license": "MIT", "dependencies": { - "content-type": "^1.0.5", + "content-type": "^2.0.0", "media-typer": "^1.1.0", "mime-types": "^3.0.0" }, "engines": { - "node": ">= 0.6" + "node": ">= 18" + }, + "funding": { + "type": "opencollective", + "url": "https://opencollective.com/express" + } + }, + "node_modules/type-is/node_modules/content-type": { + "version": "2.0.0", + "resolved": "https://registry.npmjs.org/content-type/-/content-type-2.0.0.tgz", + "integrity": "sha512-j/O/d7GcZCyNl7/hwZAb606rzqkyvaDctLmckbxLzHvFBzTJHuGEdodATcP3yIRoDrLHkIATJuvzbFlp/ki2cQ==", + "license": "MIT", + "engines": { + "node": ">=18" + }, + "funding": { + "type": "opencollective", + "url": "https://opencollective.com/express" } }, "node_modules/typed-query-selector": { diff --git a/package.json b/package.json index 28d7848f..d477db4a 100644 --- a/package.json +++ b/package.json @@ -73,6 +73,7 @@ "picomatch": "4.0.5", "sharp": "0.35.3", "sqlite-vec": "0.1.9", + "@napi-rs/canvas": "0.1.100", "unpdf": "1.6.2", "yaml": "2.9.0", "zod": "4.4.3" diff --git a/server.json b/server.json index 9e1660cb..8edaf41f 100644 --- a/server.json +++ b/server.json @@ -147,6 +147,12 @@ "description": "Byte budget for images returned by vault_read_asset, in binary bytes before base64 encoding. Images exceeding the budget are downscaled/recompressed server-side to fit; raise for clients that accept larger tool responses.", "default": "49152", "format": "number" + }, + { + "name": "MAX_PDF_RENDER_PAGES", + "description": "Maximum PDF pages to render as images when raw: true is set on vault_read_asset. The per-page byte budget is MAX_IMAGE_OUTPUT_BYTES divided evenly across the rendered pages.", + "default": "5", + "format": "number" } ] } diff --git a/src/vault-mcp/__tests__/config.test.ts b/src/vault-mcp/__tests__/config.test.ts index 3a4968fd..b4a69823 100644 --- a/src/vault-mcp/__tests__/config.test.ts +++ b/src/vault-mcp/__tests__/config.test.ts @@ -348,4 +348,28 @@ describe("loadConfig", () => { ) }) }) + + describe("MAX_PDF_RENDER_PAGES", () => { + it("defaults to 5 when unset", () => { + const config = loadConfig(EMPTY_ENV) + expect(config.maxPdfRenderPages).toBe(5) + }) + + it("accepts a custom positive integer", () => { + const config = loadConfig({ MAX_PDF_RENDER_PAGES: "10" }) + expect(config.maxPdfRenderPages).toBe(10) + }) + + it("rejects a non-integer value", () => { + expect(() => loadConfig({ MAX_PDF_RENDER_PAGES: "abc" })).toThrow( + /MAX_PDF_RENDER_PAGES/, + ) + }) + + it.each(["0", "-1", "1.5"])("rejects non-positive-integer %s", (value) => { + expect(() => loadConfig({ MAX_PDF_RENDER_PAGES: value })).toThrow( + /MAX_PDF_RENDER_PAGES/, + ) + }) + }) }) diff --git a/src/vault-mcp/config.ts b/src/vault-mcp/config.ts index e0cab0fa..43d1b35c 100644 --- a/src/vault-mcp/config.ts +++ b/src/vault-mcp/config.ts @@ -65,6 +65,10 @@ export type VaultConfig = Readonly<{ * cap (base64 expands ~4/3, then tokenizes at roughly 3 chars/token). * Set via MAX_IMAGE_OUTPUT_BYTES; raise for clients with looser caps. */ maxImageOutputBytes: number + /** Maximum number of PDF pages to render as images when raw: true is set on + * vault_read_asset. The per-page byte budget is maxImageOutputBytes divided + * evenly across the rendered pages. Set via MAX_PDF_RENDER_PAGES. */ + maxPdfRenderPages: number }> // ── Loader ───────────────────────────────────────────────────── @@ -121,7 +125,7 @@ export const loadConfig = ( // env-var's asIntPositive admits 0, but a zero byte cap would make every // asset read fail at runtime — reject it at startup instead. - const requireNonZeroBytes = (name: string, value: number): number => { + const requireNonZero = (name: string, value: number): number => { if (value === 0) { throw new Error(`env-var: "${name}" must be greater than 0`) } @@ -129,14 +133,14 @@ export const loadConfig = ( } // 50 MiB — matches the most permissive prior art for MCP asset reads. - const maxAssetBytes = requireNonZeroBytes( + const maxAssetBytes = requireNonZero( "MAX_ASSET_BYTES", envVar.from(env).get("MAX_ASSET_BYTES").default("52428800").asIntPositive(), ) // 48 KiB binary ≈ 64 KiB base64 ≈ ~21k tokens — under Claude Code's 25k-token // MCP output cap with headroom for the metadata text block. - const maxImageOutputBytes = requireNonZeroBytes( + const maxImageOutputBytes = requireNonZero( "MAX_IMAGE_OUTPUT_BYTES", envVar .from(env) @@ -145,6 +149,11 @@ export const loadConfig = ( .asIntPositive(), ) + const maxPdfRenderPages = requireNonZero( + "MAX_PDF_RENDER_PAGES", + envVar.from(env).get("MAX_PDF_RENDER_PAGES").default("5").asIntPositive(), + ) + return Object.freeze({ memoryEnabled, memoryDir, @@ -156,5 +165,6 @@ export const loadConfig = ( windowsBindMount, maxAssetBytes, maxImageOutputBytes, + maxPdfRenderPages, }) } diff --git a/src/vault-mcp/mcp-core/tools/asset-tools.ts b/src/vault-mcp/mcp-core/tools/asset-tools.ts index 5a12760d..af6de087 100644 --- a/src/vault-mcp/mcp-core/tools/asset-tools.ts +++ b/src/vault-mcp/mcp-core/tools/asset-tools.ts @@ -2,6 +2,7 @@ import { z } from "zod" import { assetOperations } from "../../vault-operations/asset-operations.js" +import type { AssetReadResult } from "../../vault-operations/asset-operations.js" import type { FittedImage } from "../../../utils/fit-image-to-byte-budget.js" import type { ToolRegistrationContext } from "./tool-helpers.js" import { safeHandler, safeHandlerContent } from "./tool-helpers.js" @@ -13,6 +14,10 @@ const TOOL_NAMES = { export { TOOL_NAMES as ASSET_TOOL_NAMES } +type ContentBlock = + | { type: "text"; text: string } + | { type: "image"; data: string; mimeType: string } + /** One-line, model-facing summary accompanying an image block: what file it * is, what was delivered, and whether/how it was shrunk to fit. */ const describeDeliveredImage = (result: { @@ -27,6 +32,45 @@ const describeDeliveredImage = (result: { return `${delivered} (recompressed from ${fitted.originalWidth}×${fitted.originalHeight}, ${originalBytes} bytes)` } +/** Formats an asset read result into MCP content blocks — the image, pages, + * or text representation the model sees. */ +const formatAssetReadResult = (result: AssetReadResult): ContentBlock[] => { + if (result.kind === "image") { + return [ + { + type: "image", + data: result.fitted.data.toString("base64"), + mimeType: result.fitted.mimeType, + }, + { type: "text", text: describeDeliveredImage(result) }, + ] + } + if (result.kind === "pages") { + const titleSegment = result.title ? `, "${result.title}"` : "" + const metadataLine = + `${result.path} — PDF, ${result.totalPages} pages` + + titleSegment + + ` — rendered ${result.pagesRendered} ` + + `page${result.pagesRendered === 1 ? "" : "s"} as images` + const pageBlocks = result.pages.flatMap((page) => [ + { + type: "image" as const, + data: page.fitted.data.toString("base64"), + mimeType: page.fitted.mimeType, + }, + { + type: "text" as const, + text: + `Page ${page.pageNumber} — ${page.fitted.mimeType}, ` + + `${page.fitted.width}×${page.fitted.height}, ` + + `${page.fitted.data.length} bytes`, + }, + ]) + return [{ type: "text", text: metadataLine }, ...pageBlocks] + } + return [{ type: "text", text: result.text }] +} + export const registerAssetTools = ({ server, vaultPath, @@ -44,11 +88,12 @@ Example: vault_read_asset({ path: "Boards/Roadmap.canvas" }) — a readable outl Example: vault_read_asset({ path: "Boards/Roadmap.canvas", raw: true }) — the canvas's exact JSON source Example: vault_read_asset({ path: "exports/data.json" }) — the file content as text Example: vault_read_asset({ path: "papers/research.pdf" }) — structured text with title, headings, and links +Example: vault_read_asset({ path: "papers/research.pdf", raw: true }) — each page rendered as an image block What each type returns: - Images (.png/.jpg/.jpeg/.gif/.webp): the image as a viewable image block — downscaled and recompressed server-side when it exceeds client response limits, delivered untouched otherwise — plus a text line stating the path, delivered format/dimensions/bytes, and the original dimensions when shrunk. Animated GIFs are reduced to their first frame when recompressed to fit the budget. - Canvas (.canvas): a readable markdown outline per JSON Canvas 1.0 — groups (by visual containment), node content in reading order, and a connections list with edge labels. Set raw: true for the exact JSON source instead (geometry, ids, colors — full fidelity). -- PDFs (.pdf): structured text with document metadata — title, page count, heading hierarchy (from font sizes), fenced code blocks (from monospace fonts), page separators, and a deduplicated links footer. Richer than flat text extraction: headings, code, and hyperlinks that flat extraction loses are preserved. Scanned or image-only PDFs with no embedded text return an error stating the page count. +- PDFs (.pdf): structured text with document metadata — title, page count, heading hierarchy (from font sizes), fenced code blocks (from monospace fonts), page separators, and a deduplicated links footer. Richer than flat text extraction: headings, code, and hyperlinks that flat extraction loses are preserved. Set raw: true for page images instead — each page rendered and returned as an image block, showing layout, diagrams, tables, and formatting that text extraction cannot preserve. Image-only and scanned PDFs work in raw mode. Up to ${config.maxPdfRenderPages} pages are rendered. - Text formats (.svg/.json/.txt/.csv/.xml/.log/.base): the file content verbatim as text. .svg is returned as its XML source; .base as its YAML source. When to use: whenever a note references an asset you need to actually see or read — an embedded diagram, a linked canvas, data file, or PDF. Find the assets a note links to (with byte sizes) via vault_get_outgoing_links; browse a folder's assets via vault_list_assets. For .md notes use vault_read_note — this tool rejects them. @@ -59,12 +104,13 @@ Errors: - "asset too large" — the file exceeds the server's read cap (MAX_ASSET_BYTES, default 50 MiB) - "text output too large" — a text asset or PDF renders past the output cap; only smaller files can be returned whole - "not valid UTF-8" — the file's bytes aren't UTF-8 text; returning them would silently corrupt the content -- "PDF has no extractable text" — the PDF exists but contains no text content (scanned or image-only); states the page count +- "PDF has no extractable text" — the PDF exists but contains no text content (scanned or image-only); states the page count. Set raw: true to render pages as images instead +- "PDF page rendering failed" — raw: true was set but no pages could be rendered; the PDF may be corrupt - "image cannot be fitted" — the image could not be compressed under the output budget (MAX_IMAGE_OUTPUT_BYTES) - "raw source is not available for images" — raw applies to text-representable files; an image's delivered form is its image block - unsupported types (audio, archives, …) return an error naming the readable types plus the file's existence and size -Returns: for images, an image content block plus a one-line metadata text block; for every other supported type, a single text content block. +Returns: for images, an image content block plus a one-line metadata text block; for PDFs with raw: true, a metadata text block followed by alternating image and text blocks (one pair per page); for every other supported type, a single text content block. Search coverage: vault_search indexes markdown notes; find assets by browsing (vault_list_assets) or through a note's links (vault_get_outgoing_links).`, inputSchema: { @@ -78,7 +124,7 @@ Search coverage: vault_search indexes markdown notes; find assets by browsing (v .boolean() .optional() .describe( - "Return the file's exact text source instead of any rendered form. For .canvas this is the JSON Canvas source (geometry, ids, colors); text formats already return their source, so raw changes nothing there. Images have no text source — raw returns an error.", + "Return an alternative representation of the file. For .canvas this is the JSON Canvas source (geometry, ids, colors); for .pdf this renders pages as images instead of extracting text — useful for scanned documents, diagrams, and layout-sensitive content. Text formats already return their source, so raw changes nothing there. Images have no text source — raw returns an error.", ), }, annotations: { @@ -104,6 +150,7 @@ Search coverage: vault_search indexes markdown notes; find assets by browsing (v raw, maxAssetBytes: config.maxAssetBytes, maxImageOutputBytes: config.maxImageOutputBytes, + maxPdfRenderPages: config.maxPdfRenderPages, }, reqLogger, ), @@ -120,20 +167,19 @@ Search coverage: vault_search indexes markdown notes; find assets by browsing (v originalHeight: result.fitted.originalHeight, recompressed: result.fitted.recompressed, }) - return [ - { - type: "image" as const, - data: result.fitted.data.toString("base64"), - mimeType: result.fitted.mimeType, - }, - { type: "text" as const, text: describeDeliveredImage(result) }, - ] + } else if (result.kind === "pages") { + reqLogger.info("tool_result", { + path, + totalPages: result.totalPages, + pagesRendered: result.pagesRendered, + }) + } else { + reqLogger.info("tool_result", { + path, + textBytes: Buffer.byteLength(result.text, "utf8"), + }) } - reqLogger.info("tool_result", { - path, - textBytes: Buffer.byteLength(result.text, "utf8"), - }) - return [{ type: "text" as const, text: result.text }] + return formatAssetReadResult(result) }, ) }, diff --git a/src/vault-mcp/vault-operations/__tests__/asset-operations.test.ts b/src/vault-mcp/vault-operations/__tests__/asset-operations.test.ts index 790f7b94..e6ce1605 100644 --- a/src/vault-mcp/vault-operations/__tests__/asset-operations.test.ts +++ b/src/vault-mcp/vault-operations/__tests__/asset-operations.test.ts @@ -1,4 +1,4 @@ -import { describe, it, expect, vi } from "vitest" +import { describe, it, expect, vi, beforeEach, onTestFinished } from "vitest" import { assetOperations } from "../asset-operations.js" import { logger } from "../../../logger.js" @@ -14,14 +14,16 @@ const { mockGetMeta, mockExtractTextItems, mockExtractLinks, + mockRenderPageAsImage, } = vi.hoisted(() => { const mockCleanup = vi.fn() return { mockCleanup, - mockGetDocumentProxy: vi.fn(() => ({ cleanup: mockCleanup })), + mockGetDocumentProxy: vi.fn(() => ({ cleanup: mockCleanup, numPages: 1 })), mockGetMeta: vi.fn(), mockExtractTextItems: vi.fn(), mockExtractLinks: vi.fn(), + mockRenderPageAsImage: vi.fn(), } }) @@ -30,6 +32,7 @@ vi.mock("unpdf", () => ({ getMeta: mockGetMeta, extractTextItems: mockExtractTextItems, extractLinks: mockExtractLinks, + renderPageAsImage: mockRenderPageAsImage, })) vi.mock("../../../utils/fit-image-to-byte-budget.js", () => ({ @@ -37,13 +40,16 @@ vi.mock("../../../utils/fit-image-to-byte-budget.js", () => ({ })) import { vaultFs } from "../vault-filesystem.js" +import { fitImageToByteBudget } from "../../../utils/fit-image-to-byte-budget.js" const mockedReadAsset = vi.mocked(vaultFs.readAsset) +const mockedFitImage = vi.mocked(fitImageToByteBudget) const defaultParams = { vaultPath: "/vault", maxAssetBytes: 52_428_800, maxImageOutputBytes: 49_152, + maxPdfRenderPages: 5, } /** Builds a single-page StructuredTextItem array from lines of text. Items @@ -362,6 +368,14 @@ describe("readAssetContent — PDF extraction", () => { extension: ".pdf", }) mockGetDocumentProxy.mockRejectedValue(new Error("Invalid PDF structure")) + // Restore the default mock regardless of assertion outcome — without + // this, a failing assertion leaves subsequent tests with a rejecting mock. + onTestFinished(() => { + mockGetDocumentProxy.mockResolvedValue({ + cleanup: mockCleanup, + numPages: 1, + }) + }) await expect( assetOperations.readAssetContent( @@ -369,9 +383,6 @@ describe("readAssetContent — PDF extraction", () => { logger, ), ).rejects.toThrow("Invalid PDF structure") - - // Restore the default mock for other tests - mockGetDocumentProxy.mockResolvedValue({ cleanup: mockCleanup }) }) it("cleans up the document proxy after successful extraction", async () => { @@ -503,3 +514,266 @@ describe("readAssetContent — PDF extraction", () => { }) }) }) + +// ── PDF page rendering (raw: true) ──────────────────────────── + +/** Builds a fake FittedImage result for page rendering tests. */ +const buildFittedImage = (overrides?: { + width?: number + height?: number + dataLength?: number +}) => ({ + data: Buffer.alloc(overrides?.dataLength ?? 9_600), + mimeType: "image/jpeg", + width: overrides?.width ?? 800, + height: overrides?.height ?? 1036, + originalWidth: 1224, + originalHeight: 1584, + recompressed: true, +}) + +/** Standard PDF mock setup: readAsset returns a .pdf buffer, getMeta + * returns the given title, the proxy reports numPages, and + * extractTextItems is pre-configured (only called in non-raw mode). */ +const setupPdfMocks = (params: { numPages: number; title?: string }) => { + mockedReadAsset.mockResolvedValue({ + buffer: Buffer.from("fake-pdf-bytes"), + bytes: 50_000, + extension: ".pdf", + }) + mockGetDocumentProxy.mockResolvedValue({ + cleanup: mockCleanup, + numPages: params.numPages, + }) + mockGetMeta.mockResolvedValue({ + info: params.title ? { Title: params.title } : {}, + }) + mockExtractTextItems.mockResolvedValue({ + totalPages: params.numPages, + items: Array.from({ length: params.numPages }, () => []), + }) +} + +describe("readAssetContent — PDF page rendering (raw: true)", () => { + beforeEach(() => { + mockRenderPageAsImage.mockReset() + mockedFitImage.mockReset() + mockCleanup.mockClear() + mockGetDocumentProxy.mockReset() + mockGetDocumentProxy.mockResolvedValue({ + cleanup: mockCleanup, + numPages: 1, + }) + mockGetMeta.mockReset() + mockExtractTextItems.mockReset() + }) + + it("returns kind pages with rendered images", async () => { + setupPdfMocks({ numPages: 2, title: "Visual Doc" }) + const fakePng = new ArrayBuffer(10_000) + mockRenderPageAsImage.mockResolvedValue(fakePng) + const fittedResult = buildFittedImage() + mockedFitImage.mockResolvedValue(fittedResult) + + const result = await assetOperations.readAssetContent( + { ...defaultParams, path: "doc.pdf", raw: true }, + logger, + ) + + expect(result).toEqual({ + kind: "pages", + pages: [ + { pageNumber: 1, fitted: fittedResult, originalBytes: 10_000 }, + { pageNumber: 2, fitted: fittedResult, originalBytes: 10_000 }, + ], + title: "Visual Doc", + totalPages: 2, + pagesRendered: 2, + path: "doc.pdf", + }) + }) + + it("respects maxPdfRenderPages cap", async () => { + setupPdfMocks({ numPages: 10, title: "Long PDF" }) + mockRenderPageAsImage.mockResolvedValue(new ArrayBuffer(5_000)) + const fittedResult = buildFittedImage() + mockedFitImage.mockResolvedValue(fittedResult) + + const result = await assetOperations.readAssetContent( + { ...defaultParams, path: "long.pdf", raw: true, maxPdfRenderPages: 3 }, + logger, + ) + + expect(result).toMatchObject({ + kind: "pages", + pagesRendered: 3, + totalPages: 10, + pages: [ + { pageNumber: 1, fitted: fittedResult }, + { pageNumber: 2, fitted: fittedResult }, + { pageNumber: 3, fitted: fittedResult }, + ], + }) + expect(mockRenderPageAsImage).toHaveBeenCalledTimes(3) + }) + + it("divides per-page budget evenly across rendered pages", async () => { + setupPdfMocks({ numPages: 4 }) + mockRenderPageAsImage.mockResolvedValue(new ArrayBuffer(1_000)) + mockedFitImage.mockResolvedValue(buildFittedImage()) + + await assetOperations.readAssetContent( + { + ...defaultParams, + path: "budget.pdf", + raw: true, + maxPdfRenderPages: 4, + maxImageOutputBytes: 40_000, + }, + logger, + ) + + // 40,000 / 4 pages = 10,000 per page + for (const call of mockedFitImage.mock.calls) { + expect(call[0].budgetBytes).toBe(10_000) + } + }) + + it("skips failed pages and returns the rest", async () => { + setupPdfMocks({ numPages: 3, title: "Partial" }) + mockRenderPageAsImage + .mockResolvedValueOnce(new ArrayBuffer(5_000)) + .mockRejectedValueOnce(new Error("render failed")) + .mockResolvedValueOnce(new ArrayBuffer(5_000)) + const fittedResult = buildFittedImage() + mockedFitImage.mockResolvedValue(fittedResult) + + const result = await assetOperations.readAssetContent( + { + ...defaultParams, + path: "partial.pdf", + raw: true, + maxPdfRenderPages: 3, + }, + logger, + ) + + expect(result).toEqual({ + kind: "pages", + pages: [ + { pageNumber: 1, fitted: fittedResult, originalBytes: 5_000 }, + { pageNumber: 3, fitted: fittedResult, originalBytes: 5_000 }, + ], + title: "Partial", + totalPages: 3, + pagesRendered: 2, + path: "partial.pdf", + }) + }) + + it("throws when all pages fail to render", async () => { + setupPdfMocks({ numPages: 2 }) + mockRenderPageAsImage.mockRejectedValue(new Error("render failed")) + + await expect( + assetOperations.readAssetContent( + { + ...defaultParams, + path: "broken.pdf", + raw: true, + maxPdfRenderPages: 2, + }, + logger, + ), + ).rejects.toThrow( + 'PDF page rendering failed: "broken.pdf" exists ' + + "(50000 bytes, 2 pages) but no pages could be rendered", + ) + }) + + it("cleans up the proxy after successful page rendering", async () => { + setupPdfMocks({ numPages: 1 }) + mockRenderPageAsImage.mockResolvedValue(new ArrayBuffer(1_000)) + mockedFitImage.mockResolvedValue(buildFittedImage()) + + await assetOperations.readAssetContent( + { ...defaultParams, path: "cleanup.pdf", raw: true }, + logger, + ) + + expect(mockCleanup).toHaveBeenCalledOnce() + }) + + it("cleans up the proxy even when all pages fail", async () => { + setupPdfMocks({ numPages: 1 }) + mockRenderPageAsImage.mockRejectedValue(new Error("render failed")) + + await expect( + assetOperations.readAssetContent( + { ...defaultParams, path: "fail.pdf", raw: true, maxPdfRenderPages: 1 }, + logger, + ), + ).rejects.toThrow("PDF page rendering failed") + + expect(mockCleanup).toHaveBeenCalledOnce() + }) + + it("passes canvasImport and scale to renderPageAsImage", async () => { + setupPdfMocks({ numPages: 1 }) + mockRenderPageAsImage.mockResolvedValue(new ArrayBuffer(1_000)) + mockedFitImage.mockResolvedValue(buildFittedImage()) + + await assetOperations.readAssetContent( + { ...defaultParams, path: "opts.pdf", raw: true }, + logger, + ) + + expect(mockRenderPageAsImage).toHaveBeenCalledOnce() + expect(mockRenderPageAsImage).toHaveBeenCalledWith( + expect.any(Uint8Array), + 1, + expect.objectContaining({ + canvasImport: expect.any(Function), + scale: 2.0, + }), + ) + }) + + it("does not change text extraction when raw is false", async () => { + setupPdfMocks({ numPages: 1, title: "Text Mode" }) + mockExtractTextItems.mockResolvedValue({ + totalPages: 1, + items: [[...buildPageItems(["Body text"])]], + }) + mockExtractLinks.mockResolvedValue({ links: [], totalPages: 1 }) + + const result = await assetOperations.readAssetContent( + { ...defaultParams, path: "text.pdf", raw: false }, + logger, + ) + + expect(result.kind).toBe("text") + expect(mockRenderPageAsImage).not.toHaveBeenCalled() + }) + + it("omits title from pages result when PDF has no title metadata", async () => { + setupPdfMocks({ numPages: 1 }) + mockRenderPageAsImage.mockResolvedValue(new ArrayBuffer(1_000)) + const fittedResult = buildFittedImage() + mockedFitImage.mockResolvedValue(fittedResult) + + const result = await assetOperations.readAssetContent( + { ...defaultParams, path: "notitle.pdf", raw: true }, + logger, + ) + + expect(result).toEqual({ + kind: "pages", + pages: [{ pageNumber: 1, fitted: fittedResult, originalBytes: 1_000 }], + title: undefined, + totalPages: 1, + pagesRendered: 1, + path: "notitle.pdf", + }) + }) +}) diff --git a/src/vault-mcp/vault-operations/asset-operations.ts b/src/vault-mcp/vault-operations/asset-operations.ts index ec2f7f3a..b1b109b9 100644 --- a/src/vault-mcp/vault-operations/asset-operations.ts +++ b/src/vault-mcp/vault-operations/asset-operations.ts @@ -3,6 +3,7 @@ import { getMeta, extractTextItems, extractLinks, + renderPageAsImage, } from "unpdf" import type { StructuredTextItem } from "unpdf" import { vaultFs } from "./vault-filesystem.js" @@ -53,7 +54,8 @@ const TEXT_PASSTHROUGH_EXTENSIONS = new Set([ const MAX_TEXT_OUTPUT_BYTES = 102_400 /** The computed result of one asset read, before content-block formatting: - * an image (fitted to the byte budget) or a text rendition. */ + * an image (fitted to the byte budget), a text rendition, or multiple + * sequential page images (e.g. rendered PDF pages). */ export type AssetReadResult = | Readonly<{ kind: "image" @@ -62,6 +64,21 @@ export type AssetReadResult = path: string }> | Readonly<{ kind: "text"; text: string }> + /** Multiple sequential page images — e.g. rendered PDF pages. */ + | Readonly<{ + kind: "pages" + pages: ReadonlyArray< + Readonly<{ + fitted: FittedImage + pageNumber: number + originalBytes: number + }> + > + title: string | undefined + totalPages: number + pagesRendered: number + path: string + }> /** Rejects text output past the fixed cap — an explicit error beats silent * truncation, and states the actual size so the caller knows what exists. */ @@ -88,6 +105,63 @@ const decodeUtf8Strict = (params: { buffer: Buffer; path: string }): string => { } } +// ── PDF page rendering ──────────────────────────────────────── + +/** Render scale for PDF page images — 2.0 produces 1224×1584px for US Letter + * (close to MAX_LONG_EDGE_PX 1568), giving sharp text after JPEG compression + * without wasting pixels that fitImageToByteBudget would discard anyway. */ +const PDF_RENDER_SCALE = 2.0 + +/** Renders `pagesToRender` pages of a PDF as fitted images, sequentially. + * Individual page failures are logged and skipped — the caller checks whether + * any pages succeeded. Sequential because the unpdf worker can't handle + * concurrent calls on the same proxy (structuredClone error on Node 24). + * Takes raw PDF bytes, not a proxy — renderPageAsImage must create its own + * proxy so pdfjs-dist's internal CanvasFactory is wired to @napi-rs/canvas + * (a proxy created without CanvasFactory falls back to a stub that throws + * on pages needing intermediate canvases for transparency/patterns/masks). */ +const renderPdfPages = async ( + params: { + pdfData: Uint8Array + pagesToRender: number + perPageBudget: number + }, + logger: Logger, +): Promise< + Array<{ pageNumber: number; fitted: FittedImage; originalBytes: number }> +> => { + const results: Array<{ + pageNumber: number + fitted: FittedImage + originalBytes: number + }> = [] + + for (let pageNumber = 1; pageNumber <= params.pagesToRender; pageNumber++) { + try { + const pngArrayBuffer = await renderPageAsImage( + params.pdfData, + pageNumber, + { + canvasImport: () => import("@napi-rs/canvas"), + scale: PDF_RENDER_SCALE, + }, + ) + const pngBuffer = Buffer.from(pngArrayBuffer) + const fitted = await fitImageToByteBudget({ + buffer: pngBuffer, + budgetBytes: params.perPageBudget, + }) + results.push({ pageNumber, fitted, originalBytes: pngBuffer.length }) + } catch (error) { + logger.warn("pdf_page_render_failed", { + page: pageNumber, + error: String(error), + }) + } + } + return results +} + // ── PDF reconstruction ───────────────────────────────────────── /** Rounds a font size to one decimal place — used as the bucketing key for @@ -238,6 +312,7 @@ const readAssetContent = async ( raw?: boolean | undefined maxAssetBytes: number maxImageOutputBytes: number + maxPdfRenderPages: number }, logger: Logger, ): Promise => { @@ -284,6 +359,43 @@ const readAssetContent = async ( // Sequential — the unpdf worker can't handle concurrent calls // on the same proxy (structuredClone error on Node 24). const meta = await getMeta(proxy) + + if (raw) { + // Page rendering mode — proxy.numPages is a direct getter on + // PDFDocumentProxy; avoids extractTextItems which walks every + // page to extract text we don't need in raw mode. + const totalPages = proxy.numPages + const pagesToRender = Math.min(totalPages, params.maxPdfRenderPages) + if (pagesToRender === 0) { + throw new Error( + `PDF page rendering failed: "${path}" exists ` + + `(${asset.bytes} bytes) but has 0 pages`, + ) + } + const perPageBudget = Math.floor( + params.maxImageOutputBytes / pagesToRender, + ) + const pages = await renderPdfPages( + { pdfData, pagesToRender, perPageBudget }, + logger, + ) + if (pages.length === 0) { + throw new Error( + `PDF page rendering failed: "${path}" exists ` + + `(${asset.bytes} bytes, ${totalPages} pages) but no ` + + `pages could be rendered`, + ) + } + return { + kind: "pages", + pages, + title: meta.info?.Title ?? undefined, + totalPages, + pagesRendered: pages.length, + path, + } + } + const { totalPages, items } = await extractTextItems(proxy) const linkResult = await extractLinks(proxy)