Skip to content

Store user images as IndexedDB assets by content hash#65

Open
Mr-Macharia wants to merge 2 commits into
Vrun-design:mainfrom
Mr-Macharia:feat/asset-store-v1
Open

Store user images as IndexedDB assets by content hash#65
Mr-Macharia wants to merge 2 commits into
Vrun-design:mainfrom
Mr-Macharia:feat/asset-store-v1

Conversation

@Mr-Macharia

Copy link
Copy Markdown

Summary

User-uploaded images and custom icons were stored as inline data: URLs on nodes. That payload was then duplicated across the live document, undo history, and snapshots, which bloated browser storage and made media-heavy diagrams expensive to save and undo.

This PR introduces asset-by-reference storage using the existing IndexedDB assets object store:

  • Encode/resize uploads (prefer WebP; keep SVG as-is; size limits)
  • Store bytes once under a content-hash id (sha256:…)
  • Nodes hold imageAssetId / iconAssetId instead of multi-MB data URLs
  • Resolve display URLs at render time via cached blob: URLs
  • Lazy-migrate legacy data: URLs into the asset store on document save
  • Fall back to data URLs if the store is disabled or a write fails

Local-first is unchanged — media stays in the user’s browser (IndexedDB). No new runtime dependencies.

Motivation / problem

  • Large images embedded in graph JSON inflate documents, undo stacks, and snapshots
  • Provider icons already use pack/shape refs; user media did not
  • The assets store was already declared in the IndexedDB schema but unused

Solution

Area Change
Rollout assetStoreV1 / VITE_ASSET_STORE_V1 (default on; set =0 for legacy behavior)
Storage src/services/storage/assetStore.ts (+ encode/hash/migration helpers)
Model imageAssetId / iconAssetId on node data
UI Image upload, icon picker, architecture custom icon, canvas image drop
Render useResolvedMediaUrl on image/icon/wireframe/custom/architecture nodes
Persist Best-effort migrate inline data URLs before local-first save
Docs Short note in ARCHITECTURE.md Persistence section

Guidelines checklist

  • TypeScript; no new npm runtime dependencies
  • Domain logic in src/services/storage/; UI stays thin
  • Feature gated behind rollout flag (src/config/rolloutFlags.ts)
  • No rename of legacy storage keys (flowmind_* etc.)
  • Migration path for existing documents with embedded data: URLs
  • Unit tests for hash, encode helpers, migration, media/icon state
  • Linked enhancement issue (happy to open one if preferred)

How to test

npm test
# focused:
npx vitest run src/services/storage/assetHash.test.ts \
  src/services/storage/assetEncode.test.ts \
  src/services/storage/assetMigration.test.ts \
  src/lib/nodeMediaState.test.ts \
  src/lib/nodeIconState.test.ts

Manual:

  1. Upload an image node / custom icon with the flag on (default).
  2. Confirm the node stores an asset id (not a huge data URL) after save/reload.
  3. Reload the app — image still renders; undo stack empty (unchanged product behavior).
  4. Optional: set VITE_ASSET_STORE_V1=0 and confirm legacy data-URL path still works.
  5. Open a diagram that already has embedded data URLs; after edit + autosave, media should migrate to asset ids when the store is available.

Out of scope / follow-ups

  • History gesture coalescing (drag/type still may over-record steps)
  • Document-scoped snapshots
  • Scheduled GC of unreferenced assets (helper exists; not wired to a UI/timer)
  • Export/import packages with asset sidecars
  • Persist full undo history (intentionally not done — session undo + snapshots remain the recovery model)

Notes for reviewers

  • Default flag is enabled so the improvement is real for users; disable via env if needed during rollout.
  • Document adapters still omit session undo history from durable docs (existing design).
  • AI chat image attachments are unchanged (still request-time base64).

Store uploaded images/icons in IndexedDB by content hash and reference
them from nodes via imageAssetId/iconAssetId. Gate with assetStoreV1
and lazy-migrate legacy data URLs on save.
Copilot AI review requested due to automatic review settings July 15, 2026 08:55

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR moves user-uploaded images and custom icons from inline data: URLs stored on nodes to content-addressed assets stored once in IndexedDB (referenced by sha256:… ids), reducing document/undo/snapshot bloat while keeping the app local-first.

Changes:

  • Added an IndexedDB-backed asset store (encode/resize/hash, cached blob: resolution, best-effort GC helpers) gated by assetStoreV1.
  • Updated node model + render/UI flows to prefer imageAssetId / iconAssetId and resolve display URLs at render time.
  • Added lazy migration that rewrites legacy inline data: URLs to asset references during local-first persistence, plus unit tests for key helpers.

Reviewed changes

Copilot reviewed 38 out of 38 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
src/services/storage/localFirstRuntime.ts Migrates legacy inline media to asset refs before saving local-first snapshots.
src/services/storage/assetTypes.ts Defines asset/store ingest types and encode defaults.
src/services/storage/assetStore.ts Implements asset store CRUD, ingest, display URL resolution, and GC helper.
src/services/storage/assetMigration.ts Implements node-level and batch migration from data: URLs to asset ids.
src/services/storage/assetMigration.test.ts Tests migration behavior with mocked asset store.
src/services/storage/assetHash.ts Adds content-hash asset id generation + validation helpers.
src/services/storage/assetHash.test.ts Tests asset id stability/validation.
src/services/storage/assetEncode.ts Adds encode/resize logic + data:/blob helpers and error type.
src/services/storage/assetEncode.test.ts Tests encode helper utilities and error typing.
src/lib/types.ts Extends node data with imageAssetId / iconAssetId.
src/lib/nodeStyleData.ts Adds iconAssetId to style field list for persistence/editing.
src/lib/nodeMediaState.ts Centralizes “prefer asset id, else URL” media reference logic.
src/lib/nodeMediaState.test.ts Tests media ref preference, collection, and inline detection.
src/lib/nodeIconState.ts Updates icon normalization/creation to support iconAssetId.
src/lib/nodeIconState.test.ts Tests icon state helpers with asset id support.
src/lib/nodeBulkEditing.ts Adds iconAssetId to bulk-editable icon capabilities.
src/hooks/useResolvedMediaUrl.ts Adds async resolution hook for asset ids to cached blob: URLs.
src/hooks/node-operations/useNodeOperationAdders.ts Extends add-image operation to pass optional imageAssetId.
src/hooks/node-operations/nodeFactories.ts Stores image nodes by imageAssetId when available.
src/config/rolloutFlags.ts Adds assetStoreV1 rollout flag (default enabled).
src/components/properties/NodeProperties.tsx Uses resolved URLs for display and writes asset ids on image upload.
src/components/properties/ImageUpload.tsx Switches upload path to asset ingest + encode error UI.
src/components/properties/IconPicker.tsx Adds icon asset ingest + resolved display support.
src/components/properties/families/ArchitectureNodeSection.tsx Adds custom icon ingest + resolved display support.
src/components/properties/bulkNodePropertiesModel.ts Tracks iconAssetId in bulk edit form state.
src/components/properties/BulkNodeProperties.tsx Wires iconAssetId through bulk icon upload flow.
src/components/ImageNode.tsx Resolves image display via useResolvedMediaUrl.
src/components/FlowEditorPanels.tsx Updates onAddImage signature to carry optional asset id/position.
src/components/flow-editor/useFlowEditorController.ts Propagates updated add-image signature.
src/components/flow-editor/panelProps.ts Updates command bar panel builder types for new add-image signature.
src/components/flow-editor/buildFlowEditorControllerParams.ts Updates controller param types for new add-image signature.
src/components/flow-canvas/useFlowCanvasDragDrop.ts Ingests dropped images into asset store before adding nodes.
src/components/CustomNode.tsx Resolves uploaded icon/image URLs for rendering without inline payloads.
src/components/custom-nodes/MobileNode.tsx Resolves image media for mobile wireframe rendering.
src/components/custom-nodes/BrowserNode.tsx Resolves image media for browser wireframe rendering.
src/components/custom-nodes/ArchitectureNode.tsx Resolves custom/provider icons via resolved media URLs.
src/components/command-bar/types.ts Updates command bar prop type for new add-image signature.
ARCHITECTURE.md Documents asset-by-reference persistence behavior and rollout flag.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +186 to +188
} catch {
// Migration is best-effort; always fall through to save.
}
Comment on lines +171 to +173
} catch {
// Keep picker usable if encode/store fails; user can retry.
}
Comment on lines +116 to +118
} catch {
// Keep inspector usable if encode/store fails.
}
Comment on lines 68 to 70
.catch(() => {
// Ignore failed drops; user can retry via the add-image control.
});
Log storage telemetry when save-time migration or image drops fail,
and show inline errors for custom icon uploads instead of swallowing.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants