Skip to content

Move to raw binary#1

Open
Odyhibit wants to merge 9 commits into
SDCVO:masterfrom
Odyhibit:master
Open

Move to raw binary#1
Odyhibit wants to merge 9 commits into
SDCVO:masterfrom
Odyhibit:master

Conversation

@Odyhibit

Copy link
Copy Markdown

Summary

This PR improves the QR transfer format and encoder controls to make generated QR frames smaller, more predictable, and easier to tune.

The main change is a new v3 binary frame format. Instead of wrapping every QR frame in JSON and base64-encoding the payload, v3 stores the frame header and payload as raw bytes in QR byte mode. This removes JSON overhead, avoids base64 expansion, and keeps repeated metadata out of every frame.

What Changed

  • Added a v3 QR frame protocol using raw binary frames.
  • Replaced per-frame JSON/base64 payloads with direct Uint8Array QR byte-mode payloads.
  • Packed frame flags into a single byte.
  • Encoded frame index, total frame count, and data frame count as varints.
  • Moved filename, original size, encoded size, hash, and first-frame payload length into frame 0 metadata.
  • Kept non-metadata data frames as payload bytes only.
  • Kept Reed-Solomon parity compatible with v3 by protecting the frame bodies, including frame 0 metadata.
  • Updated decoding to read raw binary QR data through jsQR.binaryData where available.
  • Preserved compatibility paths for existing/legacy JSON QR frames.
  • Added a QR version selector instead of manually selecting chunk size.
  • Added a fixed QR byte-capacity lookup table for all QR versions and ECC levels.
  • The encoder now calculates the largest body size that fits the selected QR version and error correction level.
  • Replaced alert popups with inline encoder errors.
  • Pauses playback while the QR version slider is being dragged, then re-encodes when released.
  • Updated README docs and tests for the v3 binary framing behavior.

Motivation

The previous format repeated a JSON header in every QR frame and stored data as base64 text. That made each QR carry significantly less actual file data than its nominal chunk size. For example, a 500-character base64 chunk only represents about 375 bytes of file data, before also accounting for JSON header overhead.

The v3 format reduces that overhead by using the QR code’s byte mode directly:

"Q3" magic
flags byte
frame index varint
total frame count varint
data frame count varint
binary frame body

Frame 0 contains the transfer metadata and its portion of payload data. Other data frames contain only payload bytes. This keeps the format compact while still allowing the decoder to recover the file name and transfer metadata. When Reed-Solomon is enabled, frame 0’s metadata is included in the protected body data, so a missing metadata frame can still be recovered from parity.

## Encoder UX Changes

The old chunk-size control was replaced with a QR version slider. This makes the user-facing control match the actual QR capacity model more directly: QR version plus error correction level determines how many bytes can fit.

The encoder displays the calculated body bytes per frame, total frame count, QR capacity, and module size. This lets users choose between easier scanning with lower QR versions and fewer total frames with higher QR versions.

## Testing

npm test passes.

The test suite now covers v3 binary framing in addition to the existing GF(256), Reed-Solomon, recovery, path traversal, GIF bounds,
and full encode/recover/decompress coverage.

Comment thread decoder/index.html Outdated
@@ -519,6 +525,7 @@ <h3 style="font-size:0.95em;margin-bottom:8px;">Files in archive <span class="ba
for (var i = 0; i < barcodes.length; i++) {
if (barcodes[i].rawValue) camProcessQR(barcodes[i].rawValue);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does the browser BarcodeDetector browser API expose QR byte-mode payloads as raw bytes anywhere, or only as rawValue/DOMString text? For v3 arbitrary binary frames, is correct decoding therefore dependent on jsQR.binaryData rather than BarcodeDetector?

Or is it impossible to use BarcodeDetector API for arbitrary binary data(for those we use jsQR only?), and we can transfer only UTF8 text with BarcodeDetector?

Comment thread README.md Outdated
- Adjustable QR version and QR error correction level (L/M/Q/H)

**Decoding:**
- Live camera scan with native `BarcodeDetector` (hardware-accelerated) + jsQR fallback

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But for arbitrary binary data(not just UTF8 text that carries over base64 payload) that BarcodeDetector claim is still valid? Can BarcodeDetector do arbitrary binary data also?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No. jsQR is not a fallback for V3; it is required. It would probably be a good idea to remove the BarcodeDetector code from running each frame since it cannot recover binary data correctly, and just use jsQR.
If you want to see the difference, here is a link to V3 https://odyhibit.github.io/code-to-qr-code/
Try the same file in both versions.

@Odyhibit

Odyhibit commented Jun 2, 2026

Copy link
Copy Markdown
Author

updated the README, and removed the unused BarcodeDetector code.

Odyhibit added 2 commits June 12, 2026 00:13
…lose as we can (507) since we do not waste space with padding, but fill the entire QR code with data
removed the call to calcFixedSize every frame, since we now define the Version.
btnGif handler is now async: with a live progress update
decoder is now async with updates while decoding.
@Odyhibit

Copy link
Copy Markdown
Author

Changed the default size to version 17, which is close to Version 2's block size of 500 (507 in my example file). Optimized the inner loop.
Testing a file of 1.5 MB in both versions will show the difference plainly. The decoder is now async as well, and the visual feedback makes it feel snappier.
If you do a full encode -> decode on a file over 1MB, you should see a significant difference.

@Odyhibit

Copy link
Copy Markdown
Author

Added color channel mode.
Using a Version 20 QR code (97 x 97), which is the default size, this new version reduces the number of frames significantly. Using a test image 1,428,078 bytes (1.4 MB on disk), the original black and white base64 version is 3771 frames. At 3 fps, that is about 21 minutes to show them all. The new colorized binary version is 718 frames, which takes about 4 minutes. The encoding time is also reduced from approx 120 seconds initially, and anytime you adjust a setting, to approx 2 seconds with the optimized encoder.
Still not exactly practical, but closer.
The decoder still works with black and white images, so it's backwards compatible.
https://odyhibit.github.io/code-to-qr-code/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants