Skip to content

Add an objectStreams option to write PDF 1.5 object streams - #1813

Merged
blikblum merged 3 commits into
foliojs:masterfrom
tomsotte:feature/object_streams
Oct 8, 2026
Merged

blikblum merged 3 commits into
foliojs:masterfrom
tomsotte:feature/object_streams

Conversation

@tomsotte

@tomsotte tomsotte commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

What kind of change does this PR introduce?

A feature, closes #1810. It adds an opt-in objectStreams option that writes PDF 1.5 object streams and a cross-reference stream.

Objects without a stream are packed into compressed /ObjStm objects, up to 200 per stream. Examples are structure elements, annotations, font dictionaries and the catalog. A single /XRef stream replaces the xref table and the trailer. How much smaller a file gets depends on how many of these small objects it has, so tagged documents gain the most:

  • A tagged PDF/A-2a document with a large table went from 1.73 MB to 415 KB. For comparison, qpdf --object-streams=generate makes the same file 379 KB.
  • pdfkit's examples, which are mostly images and fonts, get 1–27% smaller.

Changes:

  • It's off by default, and the default output doesn't change. I checked byte for byte on the examples.
  • The logic is in a new lib/object_streams.js:
    • reference.js hands it the objects that have no stream.
    • document.js has it write the xref stream at the end.
    • Output keeps streaming, buffering at most 200 serialized objects.
  • _refEnd now resets _ended before calling _finalize. Writing the last object stream ends one more reference, and without this reset the xref would be written twice.
  • It throws:
    • with a pdfVersion below 1.5;
    • with PDF/A-1, which forbids object streams;
    • with encryption, which is possible per the spec but left out to keep this PR small;
    • when the file goes past 4 GB, because the xref stream writes offsets in 4 bytes.

Testing:

  • Unit tests cover the xref stream, the packed objects, splitting into several object streams, references ended late, compress: false, tagged PDF/A-2, and every case that throws.
  • With the option forced on for every document, all visual tests still pass. Every PDF from the test suite then passes qpdf --check, except the ones that already failed it, which have no pages or are encrypted.
  • With the option on, the examples render identically in poppler. veraPDF reports the same results with and without the option for PDF/A-2b, PDF/A-3b and PDF/UA.

Checklist:

  • Unit Tests
  • Documentation
  • Update CHANGELOG.md
  • Ready to be merged

I wrote this with the help of AI, then reviewed the code and checked the output myself.

…ence stream

Packs the objects that are not streams into compressed PDF 1.5 object
streams, and writes a cross-reference stream instead of the xref table,
making documents with many such objects, e.g. tagged ones, much smaller.
Off by default; requires a pdfVersion of 1.5 or higher, and cannot be
used with encryption or PDF/A-1.
@tomsotte
tomsotte marked this pull request as ready for review October 8, 2026 09:17
@blikblum

blikblum commented Oct 8, 2026

Copy link
Copy Markdown
Member

LGTM Please fix the conflict

@blikblum
blikblum merged commit 40a2113 into foliojs:master Oct 8, 2026
3 checks passed
asmyshlyaev177 pushed a commit to asmyshlyaev177/pdfkit that referenced this pull request Oct 10, 2026
…1813)

* Add an objectStreams option to write object streams and a cross-reference stream

Packs the objects that are not streams into compressed PDF 1.5 object
streams, and writes a cross-reference stream instead of the xref table,
making documents with many such objects, e.g. tagged ones, much smaller.
Off by default; requires a pdfVersion of 1.5 or higher, and cannot be
used with encryption or PDF/A-1.

* Name the cross-reference stream entry constants and reference foliojs#1810
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add opt-in PDF 1.5 object streams and cross-reference streams to reduce file size

2 participants