Skip to content

fix: serve original PDF uploads (passthrough) and persist artifacts on disk#107

Merged
seonghobae merged 5 commits into
mainfrom
fix/pdf-passthrough-and-persistent-store
Jul 12, 2026
Merged

fix: serve original PDF uploads (passthrough) and persist artifacts on disk#107
seonghobae merged 5 commits into
mainfrom
fix/pdf-passthrough-and-persistent-store

Conversation

@seonghobae

Copy link
Copy Markdown
Collaborator

What / Why

Clearfolio's conversion pipeline was a stub: DefaultConversionWorker always called PdfBoxArtifactGenerator.generatePdf(job), so every upload — including PDFs — was served as a one-page placeholder containing only the filename and hash, and InMemoryArtifactStore lost every artifact on restart. This PR makes uploaded PDFs actually viewable and makes artifacts survive restarts, which unblocks real sheet-music viewing for the BandScope integration.

1. PDF passthrough (originals are served, not placeholders)

  • DefaultDocumentConversionService now detects PDF sources (extension or content type, confirmed by the %PDF- magic header) and seeds the artifact store with the original upload bytes at submit time.
  • DefaultConversionWorker.performDefaultConversion serves seeded bytes as the artifact and only generates the placeholder when nothing was seeded.
  • Non-PDF sources (docx/hwp/…) intentionally keep the placeholder path — real document conversion remains future work and is documented as such in ARCHITECTURE.md and the worker Javadoc. No conversion is faked.

2. Disk-backed artifact store (survives restarts)

  • New FileSystemArtifactStore persists artifact bytes plus a minimal metadata sidecar (docId, sizeBytes, sha256, storedAt) under a configurable root:
    clearfolio:
      artifact-store:
        mode: filesystem   # or in-memory
        root-dir: data/artifacts
  • Reads are served from an in-memory cache after first access, keeping the hot path CPU-only; disk reads only happen on cache misses (e.g. after restart).
  • InMemoryArtifactStore is kept for tests and mode: in-memory; ArtifactStoreConfig selects the implementation.
  • HMAC artifact-token gating and HTTP Range support are unchanged (all existing ArtifactControllerTest / ArtifactLinkServiceTest cases pass untouched).

3. LICENSE

OWNER MUST CONFIRM: the demo UI footer asserts "Copyright (c) 2026 by HYOSUNG. All rights reserved." (ViewerUiController) — if HYOSUNG holds rights, replace this LICENSE choice accordingly; merging this PR is the owner's license decision.

The repo had no LICENSE while docs/security/2026-07-02-license-policy.json itself blocks unlicensed components. This adds an Apache-2.0 LICENSE with copyright holder "ContextualWisdomLab" (separate commit so it can be dropped independently). The HYOSUNG footer text was deliberately not modified.

Tests

  • PDF passthrough serves original bytes, magic-byte verified (DefaultConversionWorkerTest, DefaultDocumentConversionServiceTest)
  • Fake .pdf without %PDF- magic and non-PDF uploads still get the placeholder path
  • Disk store round-trips, survives store re-instantiation, writes the metadata sidecar, wraps IO failures (FileSystemArtifactStoreTest)
  • Store mode selection and property normalization (ArtifactStoreConfigTest, ArtifactStorePropertiesTest)
  • Token gating unchanged: existing controller/link-service suites pass unmodified
  • Plus deterministic coverage for previously unexercised state-store race branches

Gates (all green locally)

  • mvn -q -DskipTests compile (warning/deprecated budget 0, -Werror)
  • mvn -q verify / mvn test
  • JaCoCo com.clearfolio.viewer.*: 0 missed lines / 0 missed branches
  • mvn -q -DskipTests javadoc:javadoc: no warnings
  • markdownlint on changed docs
  • python3 scripts/check_sbom_license_policy.py …: 0 violations (engineering-review mode)
  • E2E smoke on the running app: PDF upload → SUCCEEDED → artifact bytes byte-identical to upload; missing token → 401; Range: bytes=0-4 → 206/5 bytes; data/artifacts/<jobId>.pdf + .meta.properties written to disk; docx upload → placeholder PDF

BandScope integration note

BandScope currently integrates via viewer-contract + pdfjs-dist on its own; once this lands it can swap to clearfolio's /api/v1/convert/jobs + signed /artifacts/{docId}.pdf endpoints for real sheet-music viewing.

🤖 Generated with Claude Code

https://claude.ai/code/session_01RjGVapDZ3k7V7zKYk16P4C

seonghobae and others added 2 commits July 7, 2026 10:50
Serve uploaded PDFs passthrough instead of the placeholder artifact:
DefaultDocumentConversionService now seeds the artifact store with the
original upload bytes when the source declares PDF (extension or content
type) and carries the %PDF- magic header, and DefaultConversionWorker
completes the job without generating a placeholder when seeded bytes
exist. Non-PDF sources keep the placeholder path; real docx/hwp
conversion remains future work and is documented as such.

Add FileSystemArtifactStore, a disk-backed ArtifactStore that persists
artifact bytes plus a minimal metadata sidecar (docId, size, sha256,
storedAt) under clearfolio.artifact-store.root-dir (default
data/artifacts) so artifacts survive restarts, with an in-memory cache
on the read path. ArtifactStoreConfig selects filesystem (default) or
in-memory mode via clearfolio.artifact-store.mode; InMemoryArtifactStore
remains for tests and the in-memory mode. HMAC artifact-token gating and
HTTP Range serving are unchanged.

Tests: PDF passthrough serves original bytes (magic-byte verified),
non-PDF still gets the placeholder, disk store round-trips and survives
re-instantiation, metadata sidecar contents, IO failure paths, store
mode selection, plus coverage for previously unexercised constructor and
state-store branches (JaCoCo line/branch miss = 0).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RjGVapDZ3k7V7zKYk16P4C
OWNER MUST CONFIRM: the demo UI footer asserts HYOSUNG copyright
(ViewerUiController); if HYOSUNG holds rights, replace this LICENSE
choice accordingly. Merging is the owner's license decision.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RjGVapDZ3k7V7zKYk16P4C
@seonghobae seonghobae enabled auto-merge (squash) July 7, 2026 02:17
@seonghobae seonghobae closed this Jul 7, 2026
auto-merge was automatically disabled July 7, 2026 10:43

Pull request was closed

@seonghobae seonghobae reopened this Jul 7, 2026
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RjGVapDZ3k7V7zKYk16P4C
@seonghobae seonghobae enabled auto-merge (squash) July 7, 2026 10:46
@opencode-agent

opencode-agent Bot commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

OpenCode Review Overview

  • Head SHA: 344fbc06bd7583621b3f827a5150a6bcb16313b9
  • Workflow run: 29174205435
  • Workflow attempt: 1
  • Gate result: APPROVE (approval step)

Pull request overview

OpenCode reviewed the current-head bounded evidence and found no blocking issues.

Findings

No blocking findings.

Summary

Approval sufficiency: bounded evidence supplied affirmative approval evidence for changed files, coverage/docstring posture, risk surfaces, and current-head verification; approval is not based merely on the absence of known blockers.
Verification posture: CodeGraph evidence was initialized and bounded current-head evidence reviewed for changed-file evidence including ARCHITECTURE.md, src/main/java/com/clearfolio/viewer/artifact/ArtifactStore.java, src/main/java/com/clearfolio/viewer/artifact/FileSystemArtifactStore.java, src/main/java/com/clearfolio/viewer/artifact/InMemoryArtifactStore.java, src/main/java/com/clearfolio/viewer/config/ArtifactStoreConfig.java, and 11 more.
Linter/static: workflow/static review evidence is bounded by the current-head GitHub Checks gate and changed-file evidence.
TDD/regression: coverage execution evidence and focused changed hunks were reviewed from bounded-review-evidence.md.
Coverage: coverage execution evidence reports test coverage as not applicable because no supported changed source files or package manifests were found.
Docstring coverage: coverage execution evidence reports docstring coverage as not applicable because no supported changed source files or package manifests were found.
DAG: CodeGraph/source-backed behavior map connects ARCHITECTURE.md to the affected review, runtime, or workflow path and required checks.
PoC/execution: coverage-evidence job executed on the current head and reported PASS.
DDD/domain: workflow and repository-governance invariants were reviewed against changed files in bounded evidence.
CDD/context: CodeGraph evidence, changed-file history, and focused hunks were reviewed from bounded-review-evidence.md.
Similar issues: changed-file history evidence was reviewed for comparable local precedents.
Claim/concept check: bounded evidence, repository source, current-head workflow evidence, and, where numeric, scientific, statistical, or literature-backed claims are affected, original-paper/formula evidence and parameter-recovery expectations were used for claims.
Standards search: standards and external-source checks are delegated to configured OpenCode web_search/Context7/DeepWiki sources when applicable; no evidence-backed standards blocker is present in bounded evidence.
Compatibility/convention: changed workflow/script conventions, object naming, and reserved-word safety for schema/API/config/code surfaces were checked in bounded evidence.
Breaking-change/backcompat: deployment evidence and changed-file history were checked for backward-compatibility risk.
Performance: changed surfaces were checked for performance risk in bounded evidence.
Developer experience: changed automation, review, test, setup, and maintenance surfaces were checked for helpful or obstructive DX impact in bounded evidence.
User experience: connected user, operator, API, CLI, documentation, review-comment, status-check, rendering, and workflow-reader behavior was checked for contradictions against code, docs, and tests in bounded evidence.
Visual/DOM: Playwright visual, DOM locator, ARIA snapshot, console, and responsive evidence were checked when a web UI surface was present; for non-web surfaces, API/CLI/log/docs/workflow interaction evidence was reviewed instead.
Accessibility/i18n: accessibility, localization, and human-readable text surfaces were checked where UI, CLI, API message, docs, logs, or review text changed.
Supply-chain/license: dependency, package, model, container, and external-tool changes were checked in bounded evidence.
Packaging: package, build, test, lint, and security contracts were checked in bounded evidence.
Security/privacy: workflow-token, review-gate, and repository-automation security/privacy boundaries were checked in bounded evidence.

  • Result: APPROVE
  • Reason: PR introduces a disk-backed artifact store and PDF passthrough, with complete tests and no blocking issues.
  • Head SHA: 344fbc06bd7583621b3f827a5150a6bcb16313b9
  • Workflow run: 29174205435
  • Workflow attempt: 1

Changed-File Evidence Map

flowchart LR
  PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
  Evidence --> S1["Changed file (17 files)"]
  S1 --> I1["repository behavior"]
  I1 --> R1["Review risk: Changed file (17 files)"]
  R1 --> V1["required checks"]
Loading

@opencode-agent opencode-agent Bot disabled auto-merge July 11, 2026 11:44

@opencode-agent opencode-agent Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

OpenCode reviewed the current-head bounded evidence and found no blocking issues.

Findings

No blocking findings.

Summary

Approval sufficiency: bounded evidence supplied affirmative approval evidence for changed files, coverage/docstring posture, risk surfaces, and current-head verification; approval is not based merely on the absence of known blockers.
Verification posture: CodeGraph evidence was initialized and bounded current-head evidence reviewed for changed-file evidence including ARCHITECTURE.md, src/main/java/com/clearfolio/viewer/artifact/ArtifactStore.java, src/main/java/com/clearfolio/viewer/artifact/FileSystemArtifactStore.java, src/main/java/com/clearfolio/viewer/artifact/InMemoryArtifactStore.java, src/main/java/com/clearfolio/viewer/config/ArtifactStoreConfig.java, and 11 more.
Linter/static: workflow/static review evidence is bounded by the current-head GitHub Checks gate and changed-file evidence.
TDD/regression: coverage execution evidence and focused changed hunks were reviewed from bounded-review-evidence.md.
Coverage: coverage execution evidence reports test coverage as not applicable because no supported changed source files or package manifests were found.
Docstring coverage: coverage execution evidence reports docstring coverage as not applicable because no supported changed source files or package manifests were found.
DAG: CodeGraph/source-backed behavior map connects ARCHITECTURE.md to the affected review, runtime, or workflow path and required checks.
PoC/execution: coverage-evidence job executed on the current head and reported PASS.
DDD/domain: workflow and repository-governance invariants were reviewed against changed files in bounded evidence.
CDD/context: CodeGraph evidence, changed-file history, and focused hunks were reviewed from bounded-review-evidence.md.
Similar issues: changed-file history evidence was reviewed for comparable local precedents.
Claim/concept check: bounded evidence, repository source, current-head workflow evidence, and, where numeric, scientific, statistical, or literature-backed claims are affected, original-paper/formula evidence and parameter-recovery expectations were used for claims.
Standards search: standards and external-source checks are delegated to configured OpenCode web_search/Context7/DeepWiki sources when applicable; no evidence-backed standards blocker is present in bounded evidence.
Compatibility/convention: changed workflow/script conventions, object naming, and reserved-word safety for schema/API/config/code surfaces were checked in bounded evidence.
Breaking-change/backcompat: deployment evidence and changed-file history were checked for backward-compatibility risk.
Performance: changed surfaces were checked for performance risk in bounded evidence.
Developer experience: changed automation, review, test, setup, and maintenance surfaces were checked for helpful or obstructive DX impact in bounded evidence.
User experience: connected user, operator, API, CLI, documentation, review-comment, status-check, rendering, and workflow-reader behavior was checked for contradictions against code, docs, and tests in bounded evidence.
Visual/DOM: Playwright visual, DOM locator, ARIA snapshot, console, and responsive evidence were checked when a web UI surface was present; for non-web surfaces, API/CLI/log/docs/workflow interaction evidence was reviewed instead.
Accessibility/i18n: accessibility, localization, and human-readable text surfaces were checked where UI, CLI, API message, docs, logs, or review text changed.
Supply-chain/license: dependency, package, model, container, and external-tool changes were checked in bounded evidence.
Packaging: package, build, test, lint, and security contracts were checked in bounded evidence.
Security/privacy: workflow-token, review-gate, and repository-automation security/privacy boundaries were checked in bounded evidence.

  • Result: APPROVE
  • Reason: PR introduces a disk-backed artifact store and PDF passthrough, with complete tests and no blocking issues.
  • Head SHA: 344fbc06bd7583621b3f827a5150a6bcb16313b9
  • Workflow run: 29174205435
  • Workflow attempt: 1

Changed-File Evidence Map

flowchart LR
  PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
  Evidence --> S1["Changed file (17 files)"]
  S1 --> I1["repository behavior"]
  I1 --> R1["Review risk: Changed file (17 files)"]
  R1 --> V1["required checks"]
Loading

@seonghobae seonghobae merged commit 9e1931e into main Jul 12, 2026
25 checks passed
@seonghobae seonghobae deleted the fix/pdf-passthrough-and-persistent-store branch July 12, 2026 00:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant