Skip to content

feat(assets): PDF page rendering via raw: true #356

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
aliasunder merged 14 commits into from
Jul 21, 2026
Merged

feat(assets): PDF page rendering via raw: true #356

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
a31d63a
feat(assets): PDF page rendering via raw: true (#356)
aliasunder Jul 21, 2026
e61ada7
docs(agents): add full path to templates.test.ts reference
aliasunder Jul 21, 2026
50f528f
fix(assets): state page limit directly in tool description
aliasunder Jul 21, 2026
d8ed3ab
fix(assets): interpolate page limit from config in tool description
aliasunder Jul 21, 2026
3549368
fix(registry): add MAX_PDF_RENDER_PAGES to server.json
aliasunder Jul 21, 2026
4e8560b
fix(deps): bump body-parser via npm audit fix (CVE-2026-12590)
aliasunder Jul 21, 2026
30bb525
fix(assets): skip extractTextItems in raw PDF mode for performance
aliasunder Jul 21, 2026
9921024
style: code quality — simplify PDF proxy type, extract result formatt…
aliasunder Jul 21, 2026
ab115d3
fix(ci): align trivy-pr SARIF categories with main
aliasunder Jul 21, 2026
3c5d7ae
test: audit fixes — assertion consolidation, config coverage, cleanup…
aliasunder Jul 21, 2026
b895c9a
fix(ci): restore distinct SARIF categories for trivy-pr uploads
aliasunder Jul 21, 2026
d7c484c
fix(ci): re-align trivy-pr SARIF categories with main (triage)
aliasunder Jul 21, 2026
834fb22
fix(assets): zero-page guard + pass pagesToRender directly
aliasunder Jul 21, 2026
6b18810
fix(assets): pass raw PDF data to renderPageAsImage, not the proxy
aliasunder Jul 21, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .env.example
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -107,6 +107,8 @@ TZ=UTC
# Byte budget for images returned by vault_read_asset, in binary bytes before
# base64 encoding (default: 49152 = 48 KiB, sized for Claude Code's response cap).
# MAX_IMAGE_OUTPUT_BYTES=49152
# Max PDF pages to render as images for raw: true (default: 5).
# MAX_PDF_RENDER_PAGES=5
# Enable or disable the memory layer (default: true).
# Set to false to hide memory tools, skip auto-initialization, and omit
# memory references from server metadata. MEMORY_DIR is ignored when false.
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/trivy.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -193,7 +193,7 @@ jobs:
uses: github/codeql-action/upload-sarif@7188fc363630916deb702c7fdcf4e481b751f97a # v4.37.1
with:
sarif_file: trivy-results.sarif
category: trivy-pr-image
category: trivy-published-image

# hashFiles guard: the remote SARIF doesn't exist when the local scan
# gates the job first (exit-code 1) or the remote build fails.
Expand All @@ -202,4 +202,4 @@ jobs:
uses: github/codeql-action/upload-sarif@7188fc363630916deb702c7fdcf4e481b751f97a # v4.37.1
with:
sarif_file: trivy-results-remote.sarif
category: trivy-pr-image-remote
category: trivy-published-image-remote
2 changes: 1 addition & 1 deletion AGENTS.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -858,7 +858,7 @@ pattern:
6. **Root compose files** (`docker-compose.yml`, `docker-compose.local.yml`)
— maintainer/contributor surfaces (if applicable)

CI drift tests in `templates.test.ts` catch omissions across steps 2–4,
CI drift tests in `cli/src/__tests__/templates.test.ts` catch omissions across steps 2–4,
but the checklist prevents them.

**Regenerating `social-preview.png`:** Run `npm run render:social-preview`.
Expand Down
2 changes: 1 addition & 1 deletion ARCHITECTURE.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -296,7 +296,7 @@ Link queries use a `links` table populated during indexing:
1. **Images** (`.png`/`.jpg`/`.jpeg`/`.gif`/`.webp`) return an MCP `image` content block plus a one-line metadata text block. A shared fit-to-byte-budget pipeline (`utils/fit-image-to-byte-budget.ts`, built on sharp) makes oversized images deliverable: EXIF auto-orient → resize long edge to ≤1568px → walk a fixed quality ladder (JPEG via mozjpeg for opaque images, WebP for alpha — PNG has no quality knob) → shrink dimensions by √(budget/actual) if the ladder floor still exceeds the budget. Deterministic and terminating (bounded attempts, 64px floor); sharp's default `limitInputPixels` stays active as the decompression-bomb guard. The budget (`MAX_IMAGE_OUTPUT_BYTES`, default 48 KiB binary) is sized for the tightest mainstream client cap.
2. **Canvas** (`.canvas`) linearizes to markdown via the pure `obsidian-markdown/canvas.ts` parser ([JSON Canvas 1.0](https://jsoncanvas.org)): group membership by spatial rect containment (innermost group wins; equal rects tiebreak deterministically by id), nodes in reading order (y, then x), and an edge list with node ids resolved to display names. Lenient parsing — unknown properties ignored, malformed entries skipped. `raw: true` skips the linearizer and returns the JSON source verbatim for full structural fidelity.
3. **Text formats** (`.svg`/`.json`/`.txt`/`.csv`/`.xml`/`.log`/`.base`) pass through verbatim as text, capped at a fixed 100 KiB output size (explicit error over silent truncation).
4. **PDFs** (`.pdf`) return structured markdown reconstructed from layout-aware extraction (`unpdf`, based on Mozilla's PDF.js). A document metadata header (title, page count, link count), heading hierarchy inferred from relative font sizes, fenced code blocks detected via monospace fonts, page separators, and a deduplicated links footer. Scanned or image-only PDFs with no extractable text return an error stating the file's existence, size, and page count.
4. **PDFs** (`.pdf`) return structured markdown reconstructed from layout-aware extraction (`unpdf`, based on Mozilla's PDF.js). A document metadata header (title, page count, link count), heading hierarchy inferred from relative font sizes, fenced code blocks detected via monospace fonts, page separators, and a deduplicated links footer. `raw: true` switches to page-image mode: each page is rendered at 2× scale via `unpdf`'s `renderPageAsImage` with `@napi-rs/canvas` (prebuilt Skia, no system deps), then fitted through the same byte-budget pipeline as regular images. The total image budget is divided evenly across rendered pages (capped at `MAX_PDF_RENDER_PAGES`, default 5). Scanned or image-only PDFs with no extractable text work in raw mode — the model's own vision handles recognition.
5. **Unknown types** return an error naming the readable set.

The extension-to-representation routing above is implemented by the `vault-operations/asset-operations.ts` use-case. Beneath it, every read goes through `vaultFs.readAsset`, which applies the same `resolveSafePath` traversal guard as notes, rejects `.md` paths (notes belong to `vault_read_note`), and enforces a stat-before-read size cap (`MAX_ASSET_BYTES`, default 50 MiB).
Expand Down
39 changes: 20 additions & 19 deletions DOCKERHUB.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ Your notes embed screenshots, reference architecture diagrams, and link out to c

- **Images** — the image itself, not the filename. Screenshots and diagrams are downscaled and recompressed server-side when they exceed what MCP clients accept, so even a phone session can look at a 5MB architecture diagram
- **Canvases** — a [Canvas](https://help.obsidian.md/canvas) board arrives as a readable outline: its groups, each card's content in reading order, and the connections between them. The exact JSON source is one flag away when full fidelity matters
- **PDFs** — text is extracted from the document's content streams; scanned or image-only PDFs return an error stating the page count
- **PDFs** — text is extracted with heading hierarchy, code blocks, and hyperlinks preserved; set `raw: true` to render pages as images instead, showing layout, diagrams, and tables that text extraction can't preserve — scanned and image-only PDFs work in this mode
- **Text and data files** — SVG, JSON, CSV, logs, and [Bases](https://help.obsidian.md/bases) files return exactly as written
- **Browse** — list any folder's assets with per-extension counts and file sizes; assets a note links to report their size in the link graph too

Expand Down Expand Up @@ -136,25 +136,26 @@ Vault Cortex indexes every [property](https://help.obsidian.md/Editing+and+forma

All settings are environment variables with sensible defaults.

| Variable | Required? | Default | Description |
| --------------------------- | ----------- | ------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `MCP_AUTH_TOKEN` | Yes | — | Bearer token for authentication (also the JWT signing key) |
| `VAULT_PATH` | Local only | — | Host path to your vault (bind mount source; remote uses a named volume) |
| `PUBLIC_URL` | Remote only | — | Public URL for OAuth discovery metadata |
| `EMBEDDING_ENABLED` | — | `true` | Set `false` to disable the embedding pipeline — skips model download, vector tables, embedding passes, and hybrid search. Search falls back to FTS5 keyword matching. |
| `RERANK_MODE` | — | `blended` | Cross-encoder reranking mode: `blended` applies position-aware score blending after RRF fusion (~200ms added latency), `none` skips reranking. Only takes effect when `EMBEDDING_ENABLED` is true. |
| `MEMORY_ENABLED` | — | `true` | Set `false` to fully disable the memory layer — hides memory tools, skips bootstrap, omits memory from server metadata. `MEMORY_DIR` is ignored when `false`. |
| `MEMORY_DIR` | — | `About Me` | Vault folder for structured memory files |
| `PROTECTED_PATHS` | — | `MEMORY_DIR, Daily Notes` | Folders that `vault_delete_note` refuses to touch |
| `ORPHAN_EXCLUDE_FOLDERS` | — | `Daily Notes, Templates, MEMORY_DIR` | Folders excluded from orphan detection |
| `TZ` | — | `UTC` | IANA timezone for timestamps and daily note resolution |
| `SERVICE_DOCUMENTATION_URL` | — | GitHub repo URL | URL returned in OAuth discovery metadata |
| `LOG_LEVEL` | — | `info` | Logging verbosity: `debug`, `info`, `warn`, `error` |
| `LOG_DIR` | — | `/data/logs` (remote), unset (local) | Directory for persistent log files. When set, logs are written to date-stamped files there alongside stdout. Unset means stdout only. |
| `LOG_RETENTION_DAYS` | — | `30` | Days to keep log files before automatic cleanup on startup |
| `WINDOWS_MODE` | — | `false` | On Windows? Set `true`. Switches the file watcher to polling and note moves to rename-based writes so a vault on a `C:` drive works through Docker Desktop. Safe to leave on for any Windows setup; unneeded on macOS/Linux/WSL2. |
| `MAX_ASSET_BYTES` | — | `52428800` (50 MiB) | Maximum file size `vault_read_asset` will read (in bytes). Files exceeding this are rejected before reading. Raise for vaults with very large individual files. |
| Variable | Required? | Default | Description |
| --------------------------- | ----------- | ------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `MCP_AUTH_TOKEN` | Yes | — | Bearer token for authentication (also the JWT signing key) |
| `VAULT_PATH` | Local only | — | Host path to your vault (bind mount source; remote uses a named volume) |
| `PUBLIC_URL` | Remote only | — | Public URL for OAuth discovery metadata |
| `EMBEDDING_ENABLED` | — | `true` | Set `false` to disable the embedding pipeline — skips model download, vector tables, embedding passes, and hybrid search. Search falls back to FTS5 keyword matching. |
| `RERANK_MODE` | — | `blended` | Cross-encoder reranking mode: `blended` applies position-aware score blending after RRF fusion (~200ms added latency), `none` skips reranking. Only takes effect when `EMBEDDING_ENABLED` is true. |
| `MEMORY_ENABLED` | — | `true` | Set `false` to fully disable the memory layer — hides memory tools, skips bootstrap, omits memory from server metadata. `MEMORY_DIR` is ignored when `false`. |
| `MEMORY_DIR` | — | `About Me` | Vault folder for structured memory files |
| `PROTECTED_PATHS` | — | `MEMORY_DIR, Daily Notes` | Folders that `vault_delete_note` refuses to touch |
| `ORPHAN_EXCLUDE_FOLDERS` | — | `Daily Notes, Templates, MEMORY_DIR` | Folders excluded from orphan detection |
| `TZ` | — | `UTC` | IANA timezone for timestamps and daily note resolution |
| `SERVICE_DOCUMENTATION_URL` | — | GitHub repo URL | URL returned in OAuth discovery metadata |
| `LOG_LEVEL` | — | `info` | Logging verbosity: `debug`, `info`, `warn`, `error` |
| `LOG_DIR` | — | `/data/logs` (remote), unset (local) | Directory for persistent log files. When set, logs are written to date-stamped files there alongside stdout. Unset means stdout only. |
| `LOG_RETENTION_DAYS` | — | `30` | Days to keep log files before automatic cleanup on startup |
| `WINDOWS_MODE` | — | `false` | On Windows? Set `true`. Switches the file watcher to polling and note moves to rename-based writes so a vault on a `C:` drive works through Docker Desktop. Safe to leave on for any Windows setup; unneeded on macOS/Linux/WSL2. |
| `MAX_ASSET_BYTES` | — | `52428800` (50 MiB) | Maximum file size `vault_read_asset` will read (in bytes). Files exceeding this are rejected before reading. Raise for vaults with very large individual files. |
| `MAX_IMAGE_OUTPUT_BYTES` | — | `49152` (48 KiB) | Byte budget for images delivered by `vault_read_asset`, in binary bytes before base64 encoding. Images exceeding this are downscaled and recompressed to fit. Sized for the tightest mainstream MCP client cap; raise for clients that accept larger responses. |
| `MAX_PDF_RENDER_PAGES` | — | `5` | Maximum PDF pages to render as images when `raw: true` is set on `vault_read_asset`. The per-page byte budget is `MAX_IMAGE_OUTPUT_BYTES` divided evenly across the rendered pages — fewer pages means higher quality each. |

## Deployment Options

Expand Down
Loading
Loading