Skip to content

string: prevent negative slice bounds and preserve surrogate pairs during truncation - #1658

Open
GautamKumarOffical wants to merge 2 commits into
CodebuffAI:mainfrom
GautamKumarOffical:fix/string-truncation-surrogates-and-bounds
Open

GautamKumarOffical wants to merge 2 commits into
CodebuffAI:mainfrom
GautamKumarOffical:fix/string-truncation-surrogates-and-bounds

Conversation

@GautamKumarOffical

@GautamKumarOffical GautamKumarOffical commented Nov 13, 2026 •

Copy link
Copy Markdown

What

truncateStringWithMessage, truncateString, and capitalize now handle negative slice boundaries and astral code points (surrogate pairs) safely.

Why

  1. In truncateStringWithMessage, when maxLength is smaller than or equal to the truncation marker (message):

    • For remove: 'END', maxLength - suffix.length became negative. In JavaScript, str.slice(0, negative) counts backwards from the end of str. For example, truncating a 100-character string with maxLength: 20 returned a 120-character string (slice(0, -8) returning 92 chars + 28-char suffix), expanding the string rather than truncating it.
    • For remove: 'MIDDLE', length = Math.floor((maxLength - middle.length) / 2) became negative. str.slice(0, length) + middle + str.slice(-length) sliced nearly the entire string on both sides of the marker, returning 219 characters for an input of 100 characters.
    • For remove: 'START', when maxLength <= prefix.length, the slice index exceeded str.length, prepending the oversized prefix to an empty string.
  2. In truncateString and truncateStringWithMessage, slicing directly at maxLength or the message offset can cut an astral character (like an emoji or non-BMP symbol) in half, leaving a lone/dangling high surrogate (\uD800..\uDBFF). In UTF-8 encodings, JSON transmission, and database storage (such as database.ts storing truncated error messages), lone surrogates are either invalid, corrupt the text into replacement glyphs, or cause database write errors.

  3. In capitalize, str.charAt(0) split surrogate pairs when the string started with an astral character.

Fix

  • Guard maxLength <= marker.length in truncateStringWithMessage across all three removal strategies (END, START, MIDDLE), returning the marker sliced to maxLength.
  • Back off the cut index by one code unit when the character immediately before the boundary is a high surrogate (0xD800..0xDBFF), or advance by one when starting after a low surrogate (0xDC00..0xDFFF), ensuring surrogate pairs are never split.
  • Use Array.from in capitalize to iterate code points instead of UTF-16 code units.

Test

Added 11 new tests in common/src/util/__tests__/string.test.ts covering:

  • Standard truncation and length bounds.
  • Small maxLength bounds testing in truncateStringWithMessage for END, START, and MIDDLE modes.
  • Surrogate pair protection during truncation preventing lone surrogates.
  • Capitalization with ASCII and astral characters.
bun test common/src/util/__tests__/string.test.ts
 30 pass, 0 fail

Scope

common/ only, common/src/util/string.ts and its test file.

@codebuff-team

Copy link
Copy Markdown
Contributor

Good catch. The core bug is real and nasty: when maxLength is smaller than the truncation marker, str.slice(0, negative) in JS counts from the end rather than clamping to zero, so truncateStringWithMessage could expand a string instead of shrinking it (your 100→219 char MIDDLE example is correct). The guard clauses added for END/START/MIDDLE (checking maxLength <= marker.length before computing indices) are the right fix and I traced through the math - they produce exactly maxLength-length output in the small-bound cases, matching your tests.

The surrogate-pair handling is a legitimate secondary improvement (lone surrogates do break JSON/UTF-8/DB writes), and the capitalize fix using Array.from instead of charAt(0) is correct and minimal.

One nit: the surrogate-pair test for truncateStringWithMessage ('does not split surrogate pairs') has an inaccurate comment — it claims end = 50 - 28 = 22 landing on the rocket emoji, but the actual default suffix ('\n[TRUNCATED DUE TO LENGTH...]') is 30 chars, so end is actually 20, well before the emoji at index 21. The assertions still pass, but they pass trivially rather than exercising the surrogate-guard code path. Worth tightening if you revise, but it doesn't undermine the truncateString surrogate test, which does correctly exercise the guard.

Small, focused, in common/ only, with tests. Worth porting as-is; the comment fix above would make the test suite more honest about what it covers.

@codebuff-team codebuff-team added bot:triaged Classified by the community triage bot pr:port-candidate Worth porting into the private source tree labels Nov 13, 2026
@GautamKumarOffical

Copy link
Copy Markdown
Author

Good catch on the suffix length calculation — updated in ed6bdbe with index 20 so end = 21 lands directly after the high surrogate and exercises the back-off branch, asserting exact output and clean surrogate boundary preservation.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bot:triaged Classified by the community triage bot pr:port-candidate Worth porting into the private source tree

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants