XLSX files above 16MB are rejected

[james-francis-advanced]

Hello,

We're currently seeing an issue where XLSX files above 16MB are failing the content check, despite the file being seemingly valid and the start bytes matching the ZIP format (50 4B 03 04). Excel also opens the files without any error.

Claude suggests the following - I don't know enough about the file byte checking to know if this is accurate though, apologies if it is not.

The file is a perfectly valid XLSX — it starts with PK\x03\x04 and the tail is a standard ZIP EOCD record. The problem is a bug in MagicBytesValidator v2.1.6: Zip.EndsWith encodes a 22-byte EOCD pattern with a hardcoded 0x00 at byte offset 19, which is the most significant byte of the 4-byte central directory offset field. This file is 19MB so its central directory starts at 0x0126BCBC — that MSB is 0x01, not 0x00, so the check fails. Any ZIP-based file (XLSX, DOCX, PPTX, ODS, ODT, ODP) whose central directory sits beyond ~16.7MB into the file will hit this.

I realise this references v2.1.6 as this is the version we are using (for .NET 8 compatibility) but the problem seems to still exist in v2.2.1.

Thanks!

[kryptobi]

@james-francis-advanced thanks for reporting the bug, I'll take a look at it soon.

[Davilink]

I don't know if the OP use IFormFile with the use of stream, but if so, it might be related to this.
I used cursor to resume the problem

Note: I had issue with PDF so that why it's talk about pdf here, but it me be similar to other filetype validation

Incomplete stream read in Validator.IsValidAsync

Summary

Validator.IsValidAsync(Stream, …) allocates a buffer of fileStream.Length and fills it with a single ReadAsync, ignoring the number of bytes actually read. That violates Stream semantics and can leave the buffer only partially filled (trailing zeros), which breaks end-of-file checks (e.g. PDF %%EOF / EndsWith).

Affected version

Reproduced against 2.1.6 (same pattern still present in current Validator.cs on main).

Problematic code

fileStream.Position = 0;
var streamBuffer = new byte[fileStream.Length];
_ = await fileStream.ReadAsync(streamBuffer, cancellationToken); // return value discarded
return fileType.Matches(streamBuffer);

Why this is wrong

Stream.ReadAsync is allowed to return fewer bytes than requested. Callers must loop until EOF (or until Length bytes are read) unless they use an API that guarantees a full read.

This is especially relevant for:

disk-backed streams (e.g. ASP.NET Core IFormFile above MemoryBufferThreshold)
network / pipe streams
any stream that does not fill the buffer in one call
When the buffer is under-filled, formats that check trailing magic bytes fail even though the file is valid.

Expected behavior

Read the stream to completion into the buffer, e.g.:

var streamBuffer = new byte[fileStream.Length];
var totalRead = 0;
while (totalRead < streamBuffer.Length)
{
    var read = await fileStream.ReadAsync(
        streamBuffer.AsMemory(totalRead, streamBuffer.Length - totalRead),
        cancellationToken);
    if (read == 0)
        break;
    totalRead += read;
}

(or CopyToAsync into a MemoryStream / equivalent).

Impact

Valid large files (notably PDFs with EndsWith / %%EOF checks) can be reported as invalid when the source is not a fully buffered in-memory stream.