fix(expr): Escape control characters in repr_py and repr_python

[leongdl]

TLDR. repr_py escaped two characters where CPython escapes a table, so a value holding a
newline, a carriage return or a NUL came out raw inside the quotes and CPython refused to parse it.
ExprValue::repr_python had the same defect and escaped nothing at all, rendering a\b as a\x08.
Both now call one function. Verified byte-identical to CPython 3.14's repr for all 1,112,064
Unicode scalar values, and round-tripping on 3.11 through 3.14. One PR fixes Python too: the
Python expression language is this crate through PyO3, so it needs a release rather than a
second patch. 18 mutations, all caught.

What was the problem/requirement? (What/Why)

Expression Language §2.2.6 defines repr_py as following Python's
repr, with the worked example
repr_py("hello\nworld") returning 'hello\\nworld'.

write_repr_py_string escaped \ and ' and nothing else
(crates/openjd-expr/src/functions/repr.rs:359). That covers what stops a literal closing early.
It omits what makes a literal invalid.

I rendered a<CH>b for every Unicode scalar value through the old code and fed each result to
ast.literal_eval, which is CPython's own parser. Of 1,112,064 values, three do not parse:

Code point	Emitted	CPython
U+000A	'a<LF>b'	SyntaxError: unterminated string literal
U+000D	'a<CR>b'	SyntaxError: unterminated string literal
U+0000	'a<NUL>b'	SyntaxError: source code string cannot contain null bytes

Three causes, not three affected values. Any string holding one fails, so 7 of 28 realistic
payloads break. U+000A is what a multi-line python -c program is made of, and U+000D is what a
value picked up from a Windows-authored file carries. Both reach repr_py on the ordinary
template path: openjd-model passes user-supplied parameter values into FormatString::new with
no host context or extension needed.

A second site, worse. ExprValue::repr_python (value.rs:1240) built literals with
format!("ExprValue('{s}')"), escaping nothing. It reaches Python as ExprValue.__repr__, so it is
what a developer sees when printing a value:

Input	Before	Parses as
it's	ExprValue('it's')	SyntaxError
a\b	ExprValue('a\b')	a\x08, silent corruption
hello\nworld	raw newline in the quotes	SyntaxError

The a\b row is the one worth pausing on. A __repr__ whose whole job is to show what a value
is showed a different value and said nothing about it. specs/expr/public-api.md:749 commented
this function // matches Python repr, which was false.

Example template, and where it breaks

RFC 0008's reference wrap-forwarding pattern round-trips the wrapped action through repr_py. The
task body is a two-line python -c program, which is the conformance suite's own portable-fixture
convention — 150 of 1,191 merged fixtures use a multi-line args entry.

steps:
- name: Step1
  script:
    actions:
      onRun:
        command: python
        # A newline inside the arg. Legal today: openjd-model's ArgString check
        # rejects control characters except \n and \r.
        args: ["-c", "import sys\nprint('GRAND_CHILD_RAN')"]
environments:
- specificationVersion: environment-2023-09
  extensions: [WRAP_ACTIONS, EXPR]
  environment:
    name: WrapEnv
    script:
      actions:
        onWrapTaskRun:
          command: python
          args:
          - "-c"
          # repr_py emits the newline RAW here, so the generated program is
          # `subprocess.run(['python', '-c', 'import sys` and then a line break.
          - "import subprocess,sys; sys.exit(subprocess.run([{{repr_py(WrappedAction.Command)}}]+{{repr_py(WrappedAction.Args)}}).returncode)"

Before, the grandchild never runs:

File "<string>", line 1
  ...subprocess.run(['python', '-c', 'import sys
                                     ^
SyntaxError: unterminated string literal

Upstream fixtures, both parked as expected failures:
openjd-specifications#162
(WRAP_ACTIONS/jobs/proposed/wrap-repr-py-escapes-newline-in-wrapped-args) and
EXPR/jobs/proposed/expr2.2.6--repr-py-newline-roundtrip on branch
conformance-func-lib-expected-failures. Also openjd-rs security finding 22.

What was the solution? (How)

One function, py_escape::write_py_string_literal, implementing CPython's rule. Both sites call
it.

Sharing it is the point rather than a tidiness gain. repr_py and repr_python render the same
literal and had drifted to two different wrong answers. One function cannot disagree with itself.

Fixing only repr_py's escape table was the smaller change and is not what this does, because it
leaves the second renderer wrong and leaves the next one free to be wrong again.

Walkthrough of the fix

Following the call path down, which is also the diff's order.

1. write_py_string_literal, the one rule. Delimiter selection, then the escape table. It
takes the whole literal including the quotes, not just the body, because the delimiter choice
depends on scanning the value — an escape routine taking the delimiter as a parameter would push
that scan onto every caller and let the two sites disagree again. py_string_literal is the
allocating form, for the format!-based callers in value.rs.

2. select_quote, why " is ever used. CPython prefers ', switching to " only to avoid
escaping an embedded '. A value holding both quote characters keeps ' and escapes it. That
third case is the one a reader gets wrong, and it is why the tests feed values containing both.

3. is_non_printable enumerates the PRINTABLE categories and negates. This looks backwards
and is deliberate. GeneralCategory is #[non_exhaustive], so a wildcard on the non-printable
side would treat a future variant as printable and emit it raw, which is the exact defect class
this module closes. This way an unrecognised category is escaped: uglier output, never wrong
output. Below U+0080 the answer is fixed for all time, so that range is decided arithmetically and
never consults the tables — 0.6 ns/char against 6.3, so an all-ASCII payload pays nothing.

4. write_escaped_code_point, and why the budget did not move. \xNN up to U+00FF, \uNNNN
up to U+FFFF, \UNNNNNNNN above. The widest looks alarming at ten characters but needs four
input bytes, so the worst ratio is \x00's four-for-one, under escaped_bound's
MAX_ESCAPE_EXPANSION of 6. Measured across all 1.1M code points: zero exceed it.

5. repr.rs loses its local escaper. write_repr_py and write_repr_py_ref call the shared
writer. The RangeExpr arm is routed too, although RangeExpr's Display emits only digits and
-, :, ,, so nothing there can need escaping — routed so no future reader has to re-derive
that.

6. value.rs routes six arms. String, Path, float-with-preserved-spelling, RangeExpr, the
list-element arm, and the _ fallback. The float, RangeExpr and _ routings are inert, since no
reachable input differs. Routed anyway so the file has one rule rather than one rule and five
audits.

7. One arm deliberately not routed. write_repr_py's _ arm is Unresolved, which renders
<unresolved[string]>. It is unreachable through evaluation: the evaluator returns an unresolved
result for a call with an unresolved argument, so the renderer never sees one. I tried 13 routes to
refute that and could not. Quoting it would change output nothing produces and no test could reach,
so it stays exactly as it was, with a comment saying why.

What is the impact of this change?

repr_py output changes for any value containing a control character, a non-printable non-ASCII
character, or a ' without a ". Every such output was previously wrong or unparseable, so no
correct output moves. ExprValue.__repr__ changes for the same values.

Nothing else moves. repr_json, repr_sh, repr_cmd, repr_pwsh, string() and
to_display_string() are byte-identical, verified by diffing 475 rendered values against
origin/main rather than by argument. repr_pwsh deliberately does not share the new module:
PowerShell doubles '' and admits a raw newline.

This PR fixes Python too, and that needs one more step. openjd-model-for-python has no
Python-side expression evaluator — openjd/expr/__init__.py re-exports the whole language from
the PyO3 extension, which pins openjd-expr from crates.io. What the upstream fixtures call two
implementations is one codebase seen twice, and Python picks the fix up from a release plus
cargo update -p openjd-expr, with no second patch to write. The Rust conformance lane installs
openjd-cli from crates.io, so it will not see this until then.

How was this change tested?

• cargo test --workspace: 7516 passed, 0 failed. The 25 pre-existing repr tests pass
  unmodified. cargo clippy --all-features --all-targets --workspace -- -D warnings, nightly
  cargo fmt --all --check, cargo doc --no-deps --workspace with -D warnings,
  cargo +1.94.1 check --workspace (MSRV), cargo deny check, and a wasm32-unknown-unknown
  release build are all clean. EXPR conformance 369/369 against the release CLI.
• CPython is the oracle, not my judgement. Every expected value in the new tests was read out
  of CPython rather than derived from the rules. A harness drives the shipped evaluator and
  compares: 28/28 byte-exact and 28/28 round-tripping through ast.literal_eval on CPython
  3.11.12, 3.12.10, 3.13.7 and 3.14.0b4.
• Full-range sweep. Against 3.14.0b4, whose Unicode 16.0 matches the crate's:
  1,112,064 / 1,112,064 byte-identical, zero mismatches. Against 3.13.7 the 5,185 divergences
  are all code points that version still calls unassigned, and round-trip holds at every version.
  That drift is one-directional and cannot break the parse contract, so the property the tests
  pin is round-trip rather than bytes.
• 60 new tests, mutation-checked. Independent reviewers each ran their own mutation set, 18 in
  total, across the escape table, the delimiter rule, each category, the escape
  widths, the ASCII fast path and all six call sites. All 18 caught. The reviewers found two real
  defects, both fixed here. One assertion was vacuous, because a single-character value can never
  contain its own delimiter: a value holding only ' flips the delimiter to ". And list[path]
  plus nested-list recursion were unprotected on both sites. I also deleted four tests as
  duplicates of unit-level facts, then re-verified the call-site mutations as still caught.
• Not verified: Windows and macOS CI matrix rows, the cross-user jobs, the vitest suite, and
  the full non-EXPR conformance corpus. The diff is platform-independent and touches no
  openjd-sessions code.

Was this change documented?

Yes, and the absence of documentation was part of the defect. specs/ said nothing about
repr_py's output, so nobody noticed the escape table was two entries long.

• specs/expr/function-library.md gains the output contract: the round-trip guarantee, the
  delimiter rule, the escape table, the eight non-printable categories, the Unicode-version
  caveat, and why repr_pwsh stays separate.
• specs/expr/public-api.md:749's false // matches Python repr comment is replaced with what
  repr_python actually promises.
• specs/expr/architecture.md lists py_escape.rs.
• Every function in the new module has a doc comment, including one for why the
  printable-category enumeration is inverted.

Worth raising upstream separately: Expression Language §2.2.6 should say that non-ASCII parity
depends on the implementation's Unicode version, since no two implementations can agree on an
unassigned code point. And Template Schemas §5.2 forbids Cc characters in <ArgString>, which
excludes the newlines 150 merged fixtures rely on — a spec conflict that gates promoting #162's
fixture, though not this fix.

Is this a breaking change?

No public signature changes, and py_escape is pub(crate). ExprValue::repr_python's output
changes for values it previously rendered unparseably, which is the fix rather than a break.

No new dependencies. The first revision added unicode-general-category; ea20772 removed it in
favour of the generated table, so the manifest and lock are back to where they started apart from
the 738-range NONPRINTABLE addition to unicode_tables.rs.

Regenerating did move the table module's header from CPython 3.14.7 to 3.14.0b4, the newest 3.14
available on the machine that ran it. All 13 pre-existing tables and the 1479-entry
TITLE_MAP came out byte-identical to the 3.14.7 output, so the only content change is the
added table. Worth regenerating on 3.14.7 if a maintainer prefers the release recorded.

Does this change impact security?

It closes a correctness defect and is not an injection fix — worth stating positively, since
"unescaped control characters in a quoting function" reads worse than it is. The old code escaped
' and \, so a literal could never be closed early; three crafted payloads were tried and all
three produced SyntaxError rather than parsing. Python compiles a whole module before executing
any of it, so a raw newline broke generation loudly instead of executing injected statements.

The new code strictly widens what is escaped, and the #[non_exhaustive] handling in step 3 fails
closed. No file, directory or permission behaviour changes.

---

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

[seant-aws]

should this be breaking or its okay to consider just a fix, as this was assumed there

[leongdl]

should this be breaking or its okay to consider just a fix, as this was assumed there

This is OK since it did not pass a new proposed conformance test. This PR changes nothing to current test expectations.