[QTI] Legacy input question conversion produces wrong or unanswerable QTI #6196

❌ This issue is not open for contribution. Visit Contributing guidelines to learn about the contributing process and how to find suitable issues.

Overview

The #6003 conversion, which #6007 will run over every legacy item, turns several stored input question answer shapes into QTI that is wrong or can't be answered, and every result passes XSD validation. This needs fixing before the #6007 backfill runs.

Complexity: Medium
Target branch: unstable

Context

• The export path drops falsy answers before conversion (

  studio/contentcuration/contentcuration/utils/assessment/base.py

  Lines 342 to 356 in 8cd2daf

  	def _process_answers(self, assessment_item):
  	answer_data = json.loads(assessment_item.answers)
  	processed_answers = []
  	for answer in answer_data:
  	if answer["answer"]:
  	if isinstance(answer["answer"], str):
  	(answer["answer"], answer_images,) = self._process_content(
  	answer["answer"],
  	)
  	answer["images"] = answer_images
  	processed_answers.append(answer)
  	return self._sort_by_order(processed_answers, "answers")

  ), but convert_legacy_question_to_qti passes the stored answers through unchanged:

  studio/contentcuration/contentcuration/utils/assessment/qti/ingest.py

  Lines 26 to 47 in 8cd2daf

  	def convert_legacy_question_to_qti(question_data: dict) -> QTIConversionResult:
  	answers = json.loads(question_data.get("answers") or "[]")
  	for answer in answers:
  	if isinstance(answer.get("answer"), str):
  	answer["answer"] = strip_content_storage_placeholder(answer["answer"])
  	hints = json.loads(question_data.get("hints") or "[]")
  	for hint in hints:
  	if isinstance(hint.get("hint"), str):
  	hint["hint"] = strip_content_storage_placeholder(hint["hint"])
  	item = LegacyAssessmentItem(
  	type=question_data["type"],
  	question=strip_content_storage_placeholder(question_data.get("question") or ""),
  	answers=answers,
  	hints=hints,
  	randomize=question_data.get("randomize") or False,
  	assessment_id=question_data["assessment_id"],
  	title=question_data.get("assessment_id"),
  	language=question_data.get("language") or "en",
  	)
  	return convert_legacy_assessment_item_to_qti(item)

• _create_text_entry_interaction_and_response decides on float with Python's float(), and uses cardinality="multiple" when there is more than one answer:

  studio/contentcuration/contentcuration/utils/assessment/qti/convert.py

  Lines 207 to 239 in 8cd2daf

  	def _create_text_entry_interaction_and_response(
  	item: LegacyAssessmentItem,
  	) -> Tuple[Div, ResponseDeclaration]:
  	prompt = _create_html_content_from_text(item.question)
  	interaction_element = TextEntryInteraction(
  	response_identifier="RESPONSE",
  	expected_length=50, # Default expected length
  	placeholder_text="Enter your answer here",
  	)
  	# Text entry interaction is an inline element, so wrap it in a paragraph tag.
  	interaction_element = P(children=[interaction_element])
  	# prompt is already a list of elements, so just append the interaction to it.
  	prompt.append(interaction_element)
  	interaction = Div(children=prompt)
  	correct_values = []
  	values_float = []
  	for answer in item.answers:
  	if answer["correct"]:
  	correct_values.append(Value(value=str(answer["answer"])))
  	try:
  	float(answer["answer"])
  	values_float.append(True)
  	except ValueError:
  	values_float.append(False)
  	float_answer = bool(values_float) and all(values_float)
  	response_declaration = _response_declaration(
  	Cardinality.MULTIPLE if len(correct_values) > 1 else Cardinality.SINGLE,
  	BaseType.FLOAT if float_answer else BaseType.STRING,
  	correct_values,
  	)
  	return interaction, response_declaration

• Perseus derivation turns every text entry into an input question, whatever its base type:

  studio/contentcuration/contentcuration/utils/assessment/qti/perseus_derive.py

  Lines 150 to 156 in 8cd2daf

  	def _derive_text(interaction, item_body, declaration):
  	question = html_to_markdown([item_body])
  	answers = [
  	{"answer": value, "correct": True, "order": order}
  	for order, value in enumerate(_correct_values(declaration))
  	]
  	return exercises.INPUT_QUESTION, question, answers

• Stored answers and what convert_legacy_question_to_qti produces today:

  [""]                 -> string, correct response ""
  ["2", ""]            -> string, cardinality="multiple", correct {"2", ""}
  [false]              -> float, correct response "False"
  [false, "16508"]     -> float, correct {"False", "16508"}
  ["4\.62"]            -> string, correct response "4\.62"
  ["Sphere"]           -> string, correct response "Sphere", derived to a Perseus numeric input with no answer
  [{"answer": "5"}]    -> KeyError: 'correct'
  [{"answer": null}]   -> TypeError
  

• In live channels, 163 items have only blank answers and 53 have a JSON false answer.

The Change

• Blank ("") and JSON false answers should be dropped before conversion, as the export path already does.
• An answer with no correct key should count as correct, and a null answer should be dropped, instead of making conversion raise.
• An answer that extract_value reads as a number (with the whole-answer rule from [QTI] Perseus export publishes wrong answers for input questions that aren't plain numbers #6195, e.g. 4\.62, 1/2, 1,234) should convert to a float value in xsd:double form. The question should be float only when every answer converts.
• An input question with more than one accepted answer should get cardinality="single" with a <qti-mapping> and map_response, so entering any one of them is marked correct: the shape QTI editor emits cardinality="multiple" and match_correct for multi-answer Numeric and Text entry questions #6187 specifies for the editor.
• A string text entry should not be derived to Perseus, so its node publishes QTI only.
• Publishing should leave out items with no correct answer and log a warning.

How to Get There

• Call convert_legacy_question_to_qti({"type": "input_question", "question": "Q?", "answers": '[{"answer": "", "correct": true}]', "assessment_id": "a" * 32}) and read the qti-response-declaration in the returned XML. Repeat with each answer shape in Context.

Out of Scope

• Choice question conversion.
• Running the backfill ([QTI] Backfill all non-raw AssessmentItems to QTI items #6007).
• The editor's multi-answer serialization (QTI editor emits cardinality="multiple" and match_correct for multi-answer Numeric and Text entry questions #6187).
• The editor's numeric answer validation.

Acceptance Criteria

General

• convert_legacy_question_to_qti produces, for each stored answer list:
  • [""], [false], [], [{"answer": null}]: no correct response, no exception
  • ["2", ""]: float, cardinality="single", correct 2, match_correct
  • [false, "16508"]: float, correct 16508
  • ["4\.62"]: float, correct 4.62
  • ["1/2"]: float, correct 0.5; ["1,234"]: float, correct 1234
  • ["Sphere"]: string, correct Sphere
  • [{"answer": "5"}]: float, correct 5
  • ["13", "12"]: float, cardinality="single", map entries 13 and 12 with mapped-value="1.0", map_response
• Every item above passes validate_qti_item.
• An exercise containing a string text entry publishes a QTI package and no Perseus archive.
• An exercise with one answerless input question publishes its other items, leaves that item out, and logs a warning.

Testing

• Tests cover each answer list above, string text entry derivation, and publish skipping answerless items.

Testing

• pytest contentcuration/contentcuration/tests/utils/qti/test_ingest.py contentcuration/contentcuration/tests/utils/qti/test_convert.py contentcuration/contentcuration/tests/utils/qti/test_perseus_derive.py contentcuration/contentcuration/tests/utils/test_publish.py

AI usage

I directed a survey of input question answers on production and decided how each answer shape should convert. Claude Code wrote and ran the survey scripts, ran the conversion on those shapes, and drafted this issue section by section.