[QTI] Publish ships malformed or garbled item XML for control characters and non-UTF-8 declarations #6286

❌ This issue is not open for contribution. Visit Contributing guidelines to learn about the contributing process and how to find suitable issues.

Target branch: unstable

Observed behavior

Legacy input answer containing a character XML 1.0 forbids (U+0000–U+0008, U+000B, U+000C, U+000E–U+001F), e.g. "a\vb":

• convert_legacy_assessment_item_to_qti writes it unchanged into map-key and <qti-value> (convert.py#L368-L383); stdlib ElementTree serializes it as-is (base.py#L125-L128).
• No sized image: the malformed item ships. rewrite_qti_media_paths returns without parsing (media.py#L69-L70) and nothing validates the legacy path.
• Sized image: the parse fails and the question is left out of the package with an error log (archive.py#L299-L306).

Native item declaring a non-UTF-8 encoding, e.g. encoding="ISO-8859-1":

• parse_qti_xml encodes raw_data to UTF-8 and lxml decodes it as the declared encoding (validation.py#L57-L68), so café reads as café. validate_qti_item passes.
• set_qti_item_language reserializes as UTF-8 (media.py#L33-L34), so the shipped file declares UTF-8 and holds the mojibake.
• With no node or default language, the item ships as UTF-8 bytes still declaring ISO-8859-1.

Errors and logs

From validate_qti_item on the published legacy item:

PCDATA invalid Char value 11, line 2, column 533

Expected behavior

• Every published item is well-formed UTF-8 and its text matches raw_data.

User-facing consequences

• Learners get a question no conforming XML parser accepts, or garbled text in every non-ASCII word.

Steps to reproduce

1. Create a legacy input_question answered "a\vb" and publish: items/<id>.xml holds a raw U+000B.
2. In VALID_CHOICE_ITEM (tests/utils/qti/test_validation.py), set encoding="ISO-8859-1" and prefix the prompt with café : validate_qti_item is valid, and QTIExerciseGenerator._create_native_qti_item ships the prompt as café Select the correct answer.

Acceptance Criteria

• A legacy input answer with a forbidden character publishes well-formed XML, with or without a sized image.
• A native item declaring a non-UTF-8 encoding publishes text equal to raw_data, or fails validation.
• Publish tests cover both.

Context

unstable at 98b560e.

AI usage

rtibbles chose which issues to merge; Claude Code reproduced both cases on unstable and drafted this issue, which merges #6283.