Skip to content

Support blob type visualization - #72

Open
dentiny wants to merge 1 commit into
lance-format:mainfrom
dentiny:blob-visualization-upstream
Open

Support blob type visualization#72
dentiny wants to merge 1 commit into
lance-format:mainfrom
dentiny:blob-visualization-upstream

Conversation

@dentiny

@dentiny dentiny commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Closes #3

Hi team, since lance file format specializes for multi-model support, in this PR I want to add inline visualization for blob-type data, including videos, images, audios, etc.

In simple words, the backend detects media formats from binary signatures and returns MIME metadata with a base64 payload. The frontend renders recognized values as inline images, audio, or video.

Example UI display
image

Followup items

  • Since multi-modal data is significantly larger than scalar ones, it introduces non-trivial latency overhead; in the followup PR I will introduce UI and IO optimizations, mainly includes (1) parallel IO operations; (2) lazy load blob data
  • I have implemented the optimization in my fork https://github.com/dentiny/lance-data-viewer, and deployed to prod inside my workplace

AI usage disclaimer:
GPT-5.5 helped me make the code change, I deployed my own fork to k8s and verified it worked

@gordonmurray gordonmurray left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for implementing media detection and inline previews. This is a useful direction for issue #3, but I’m requesting changes before merge.

I need to define the binary/media response contract before this implementation can land. The viewer must not embed arbitrary-size images, audio, or video as base64 in every /rows response. That would make large binary datasets substantially slower and could worsen the existing large-payload problem. Browser-side loading="lazy" does not avoid transferring the base64 data already present in the JSON response.

The intended design should be:

  • small media may be returned inline under a documented size limit;
  • larger media returns metadata only, without a base64 payload;
  • loading large media happens through an explicit, bounded/lazy request or user action;
  • the API contract clearly distinguishes media metadata from inline content.

I’ll follow up with the exact size limit and endpoint contract. After that, please revise the implementation and add:

  • API-level tests for small and oversized media values;
  • compatibility tests for ordinary and fixed-size binary columns;
  • the media response contract in docs/spec.md;
  • the required [Unreleased] CHANGELOG.md entry referencing issue #3.

The preview UX is beneficial, but the current eager full-payload behavior is not safe to merge.

@dentiny

dentiny commented Aug 5, 2026

Copy link
Copy Markdown
Contributor Author

Hi @gordonmurray thanks for the feedback!

On inline storage fetch vs extra access, I'm thinking whether we could

  • Display the blob inline, if the lance blob storage type is storing it inline (instead of packed or URL)
  • For large blobs, we (1) display metadata on the blob (i.e., size) and a button; (2) when button clicked by user, fetch the externally stored blob and pop out a separate session to display

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Handling of binary data

2 participants