fix(tts): strip nested markup from extracted expressive tag values

[priyam-garg]

extract_and_strip recorded a wrapping tag's value as its raw inner content, so nesting leaked the inner tags' delimiters into the value:

<excited><loud>no way</loud></excited>
-> [("excited", "<loud>no way</loud>"), ("loud", "no way")]

ExpressiveTag.value is documented as "the tag's inner text" and is surfaced to the frontend via ATTRIBUTE_TRANSCRIPTION_EXPRESSION, so the markup reached consumers even though the transcript itself was clean.

Clean the inner text before recording it. The raw inner content is still returned, so the fixed-point loop's next pass records the nested tags on their own -- the tag list is unchanged, only the values are now text.

test_nested_emotion_prosody_strips_cleanly already promised "no leaked inner markup" but only asserted on the transcript; it now checks the tags too, and fails without this change.

[priyam-garg]

Summary

extract_and_strip records a wrapping tag's value as its raw inner content, so when markup nests, the outer tag keeps the inner tags' delimiters:

>>> extract_and_strip("<excited><loud>no way</loud></excited>", xml_tags=["excited", "loud"])
('no way', [('excited', '<loud>no way</loud>'), ('loud', 'no way')])
#                        ^^^^^^^^^^^^^^^^^^^^ markup, not text

ExpressiveTag.value is documented as "the tag's inner text", and split_all_markup surfaces it to the frontend via ATTRIBUTE_TRANSCRIPTION_EXPRESSION. So the transcript comes out clean while the tag payload still carries XML.

Nesting isn't exotic — combining an xAI emotion with a prosody wrapper produces exactly this:

raw = '<excited><loud><higher-pitch>no way</higher-pitch></loud></excited> <sound value="laugh"/> okay'
clean, tags = split_all_markup(raw)

clean  # 'no way  okay'                       <- correct
tags   # {'type': 'excited', 'value': '<loud><higher-pitch>no way</higher-pitch></loud>'}
       # {'type': 'sound',   'value': 'laugh'}
       # {'type': 'loud',    'value': '<higher-pitch>no way</higher-pitch>'}

Fix

Clean the inner text before recording it, in markup_utils._repl:

inner_text = extract_and_strip(inner, xml_tags=xml_tags)[0].strip() if inner else ""

The raw inner is still what gets returned into the text, so the fixed-point loop's next pass records the nested tags on their own. The tag list is unchanged — same tags, same order, same count. Only the values become text. The recursion terminates: inner is always strictly shorter than the match it came from.

Non-nested shapes are unaffected:

input	value
<spell>A7</spell>	"A7" (inner text, as before)
<emotion value="happy"/>	"happy" (attribute, as before)
<excited value="wow"><break time="500ms"/></excited>	"wow" — stripping the nested markup empties the inner text, so it falls back to the attribute rather than recording delimiters

Tests

test_nested_emotion_prosody_strips_cleanly already existed, and its comment already promised this:

combining emotion + prosody means nesting; the transcript must come out clean (no leaked inner markup)

…but it discarded the tag list and asserted only on the transcript, so the leak sat in the blind spot. It now asserts on the tags too.

Plus two focused cases in TestExtractAndStrip:

• test_nested_tag_value_is_inner_text_not_inner_markup
• test_nested_value_falls_back_to_attribute_when_inner_is_markup_only

All three fail on main and pass with the fix:

FAILED TestExtractAndStrip::test_nested_tag_value_is_inner_text_not_inner_markup
FAILED TestExtractAndStrip::test_nested_value_falls_back_to_attribute_when_inner_is_markup_only
FAILED TestXaiDialect::test_nested_emotion_prosody_strips_cleanly
3 failed, 53 passed

537 passing across the markup, transcript, expressive-toggle, TTS-fallback and agent-session suites. ruff format --check, ruff check, and mypy --strict on livekit.agents.tts are clean.

[priyam-garg]

Summary of where this landed after review, since the fix changed shape along the way.

The bug (9d315cb). extract_and_strip recorded a wrapping tag's value as its raw inner content, so nesting leaked the inner delimiters into a field documented as "the tag's inner text" and surfaced to the frontend:

>>> extract_and_strip("<excited><loud>no way</loud></excited>", xml_tags=["excited", "loud"])
('no way', [('excited', '<loud>no way</loud>'), ('loud', 'no way')])

The complexity fix (5a30ce1). My first attempt cleaned the value by recursing into extract_and_strip, which @devin-ai-integration correctly flagged: that made cleaning exponential in nesting depth (~4ms at depth 8, ~167ms at depth 14) because every level re-scanned everything below it from inside the fixed-point loop. A value only needs its delimiters deleted, not the full restructuring pass, so the fixed-point loop isn't needed there at all. Depth 14 is now 0.35ms, depth 20 is 0.56ms. Both patterns are also compiled once per tag set instead of per call.

The micro-optimization (7374da1). Copilot's point that plain inner text shouldn't pay for a scan still held on top of that — it's the common shape per streamed chunk. Short-circuited to inner.strip() when the inner content has no markup characters: 0.454µs → 0.158µs per wrapping tag.

Net effect on behaviour: the tag list is unchanged — same tags, same order, same count. Only the values became text. Every non-nested shape produces exactly what it did before.

Verification. The three nested-value tests fail against origin/main and pass here; test_deep_nesting_stays_cheap guards the complexity regression at 20 levels. 538 passing across the markup, transcript, expressive-toggle, TTS-fallback and agent-session suites, with ruff format --check, ruff check, and mypy --strict on livekit.agents.tts clean.

Happy to squash these into one commit if you'd prefer the history flat.

[Copilot]

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.