Skip to content

Fix escaped-quote detection in JSON cleanup for strings ending with a backslash - #755

Draft
rootkiller6788 wants to merge 1 commit into
google:mainfrom
rootkiller6788:fix/cleanup-json-escaped-backslash
Draft

Fix escaped-quote detection in JSON cleanup for strings ending with a backslash#755
rootkiller6788 wants to merge 1 commit into
google:mainfrom
rootkiller6788:fix/cleanup-json-escaped-backslash

Conversation

@rootkiller6788

Copy link
Copy Markdown

Problem

cleanup_json in langfun/core/structured/schema/json.py detects an escaped double quote by checking only the immediately preceding character:

elif c == '"' and json_str[i - 1] != '\\':

This is incorrect when a closing quote is preceded by an escaped backslash. In JSON, \\" (two backslashes followed by a quote) means the string value ends with a literal backslash and the quote is a real closing quote. The old check treated that quote as escaped, so the scanner never left the string state and the trailing } was skipped, producing:

ValueError: Malformated JSON: missing 1 closing curly braces.

This affects any JSON-protocol lf.parse / lf.query response whose string value ends with a backslash (for example a Windows path like "C:\\", a regex fragment, or a LaTeX string).

Fix

Replace the single-character lookback with a helper that counts consecutive backslashes. A quote is escaped only when it is preceded by an odd number of backslashes, which is the correct JSON rule:

def _is_unescaped_quote(json_str, i):
  num_backslashes = 0
  j = i - 1
  while j >= 0 and json_str[j] == '\\':
    num_backslashes += 1
    j -= 1
  return num_backslashes % 2 == 0

Test

Added test_parse_with_escaped_backslash verifying that {"result": "C:\\"} parses to C:\ (previously raised JsonError). Existing tests, including the escaped-quote case in test_parse_basics, continue to pass.

… backslash

cleanup_json detected an escaped quote by looking at only the single
preceding character, so a closing quote that follows an escaped backslash
(e.g. a string value ending in a literal backslash) was misread as an
escaped quote. This caused otherwise-valid JSON to fail with 'Malformated
JSON: missing 1 closing curly braces'. Count consecutive backslashes and
treat the quote as escaped only when the count is odd.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant