Separation fill colours of untouched objects become black after EPDFPage_ApplyRedactions + FPDFPage_GenerateContent

[AI-Daita]

After EPDFPage_ApplyRedactions + FPDFPage_GenerateContent, objects that were not under any redaction lose their Separation fill colour: a Separation-filled path and a /ImageMask painted with a Separation colour are both written back as black. DeviceCMYK fills on the same page are preserved.

Runtime: runtime-0d657f032cd680e6213442bbc078f9ab1d189f75 (linux-arm64 and linux-x64 release tarballs), used through its C API.

Minimal repro

One page, built with pikepdf:

import io, zlib, pikepdf
pdf = pikepdf.new()
tint = pikepdf.Dictionary(FunctionType=2, Domain=[0, 1], C0=[0, 0, 0, 0], C1=[0, 0.61, 0.97, 0], N=1)
sep = pikepdf.Array([pikepdf.Name.Separation, pikepdf.Name("/PANTONE#20165#20U"),
                     pikepdf.Name.DeviceCMYK, pdf.make_indirect(tint)])
mask = pdf.make_stream(zlib.compress(bytes(8 * 16)), Type=pikepdf.Name.XObject, Subtype=pikepdf.Name.Image,
                       Width=64, Height=16, ImageMask=True, BitsPerComponent=1, Filter=pikepdf.Name.FlateDecode)
font = pdf.make_indirect(pikepdf.Dictionary(Type=pikepdf.Name.Font, Subtype=pikepdf.Name.Type1,
                                            BaseFont=pikepdf.Name.Helvetica))
content = b"""
q /CS0 cs 1 scn 50 700 200 60 re f Q
q /CS0 cs 1 scn 300 700 200 0 0 60 cm /Im0 Do Q
q 0 0.61 0.97 0 k 50 600 200 60 re f Q
BT /F1 18 Tf 50 500 Td (Jane Doe) Tj ET
"""
res = pikepdf.Dictionary(ColorSpace=pikepdf.Dictionary(CS0=sep), XObject=pikepdf.Dictionary(Im0=mask),
                         Font=pikepdf.Dictionary(F1=font))
pdf.pages.append(pikepdf.Page(pikepdf.Dictionary(Type=pikepdf.Name.Page, MediaBox=[0, 0, 612, 792],
                                                 Resources=res, Contents=pdf.make_stream(content))))
pdf.save("separation.pdf")

Then, with the runtime:

1. FPDF_LoadMemDocument64, FPDF_LoadPage(doc, 0)
2. find "Jane Doe" with FPDFText_FindStart/FindNext, take its rects from FPDFText_GetRect
3. FPDFPage_CreateAnnot(page, FPDF_ANNOT_REDACT) + FPDFAnnot_SetRect + FPDFAnnot_AppendAttachmentPoints over those rects (no /IC, no overlay)
4. EPDFPage_ApplyRedactions(page, &removed) → returns true
5. FPDFPage_GenerateContent(page), FPDF_SaveAsCopy(..., FPDF_NO_INCREMENTAL)

Expected

Only "Jane Doe" is removed; everything else renders as before.

Actual

"Jane Doe" is removed correctly, but the colour operators of the untouched objects change:

before: /CS0 cs 1 scn   /CS0 cs 1 scn   0 0.61 0.97 0 k
after : 0 G             0 G             0 0.61000001 0.97000003 0 k   0 G

Rendered with the same runtime, at the centre of each object:

object	before	after
Separation-filled rect (not redacted)	(244, 128, 38)	(0, 0, 0)
/ImageMask painted with the Separation colour (not redacted)	(244, 128, 39)	(2, 2, 2)
DeviceCMYK rect (control)	(244, 128, 38)	(244, 128, 38)

The /ColorSpace resource of the page is also gone after saving.

Why it matters

Redaction of a single word rewrites the whole page's content stream, so the colour loss hits objects far away from the redacted area. We see it in real documents in two forms:

• print/corporate PDFs with spot colours (e.g. Pantone table cells, brand bars) turn black;
• scanned PDFs using MRC compression, where the text layer is a 1-bit /ImageMask painted with a Separation colour — after redacting one word, the rest of the page's text turns black or, depending on the tint, almost white.

DeviceN is likely affected the same way (not tested). As a workaround we currently compare renders before/after and rasterise pages that changed outside the redacted areas.

Happy to provide more samples if useful.

[seriouswebby]

We see the same thing on the same release (0d657f0), on darwin-arm64 and linux-x64 as well. It isn't limited to Separation fills:

• DeviceN, Indexed and Lab fills also turn black, and by the code CalGray and CalRGB would too.
• Strokes in these colour spaces turn black.
• Text drawn in these colour spaces turns black.
• Image masks painted in these colours turn black, as in your report.

It happens anywhere on a page that has a redaction, however far from the box. Device RGB/CMYK/Gray, ICC-based colours and tiling patterns are unaffected.

The generator seems to be the cause: RealizeColorSpaceObject only handles device and ICC-based spaces (see the TODO on line 679). When it returns nothing, EmitColor writes no colour, and ProcessGraphics ignores that, so the object falls back to black.

As a workaround, reading each text, path and image-mask object's colour with FPDFPageObj_GetFillColor / GetStrokeColor and writing it back with FPDFPageObj_SetFillColor / SetStrokeColor before EPDFPage_ApplyRedactions keeps the page pixel-identical.