[m-messer]

Problem

Stacked on #281. Grading a number that combines the log of a constant with a large power of 10, such as log(0.1)*(10^8/ pi), ran past the 120 s Lambda timeout. The student got no grade, and the instance was held for the full timeout. This predates the refactor: the old grader stored empty responses for all 4 past submissions of this shape.

The cause is SymPy's simplify(), which calls logcombine. That rewrites -10^8*log(10)/pi as log(10**(-10**8)) and builds a 100-million-digit integer. Two places hit it: the "simplified" display in evaluation.py, and the comparison in do_comparison. Parsing is not involved.

Changes

• app/time_budget.py (new): nested time limits enforced with SIGALRM. They're checked between bytecodes, which interrupts this simplify promptly. TimeBudgetExceeded is a BaseException, so the except Exception around SymPy calls can't swallow it. A limit does nothing off the main thread. The limits are 5 s per comparison and 60 s per request.
• app/context/symbolic.py: the symbolic attempts in check_equality run under the comparison limit. If they run out of time, = falls back to comparing values to 50 digits. check_order, check_proportionality and equality equivalence also run under the limit and grade as incorrect if they run out of time.
• app/context/numerical.py: adds numerically_equal, and replace_pi moves to module level. The fallback substitutes exact pi, because float pi would add noise at 50 digits.
• app/evaluation.py:
  • The "simplified" display falls back to the raw response when it runs out of time.
  • The whole evaluation runs under the request limit. When that runs out, the request is graded incorrect with the new TIME_LIMIT_EXCEEDED feedback (app/feedback/symbolic.py).

Response vs answer	Before	After
log(0.1)*(10^8/ pi) vs (-1/(100*pi*k))*ln(r)+c	over 120 s	10 s, incorrect
900000+10^8* pi* log(100) vs 1.47*10**8	hangs	10 s, incorrect
log(0.1)*(10^8/ pi) vs -10^8*log(10)/pi	hangs	5 s, correct
-10^8/( pi) log(r) vs (-1/(100*pi*k))*ln(r)+c	0.15 s	0.15 s, incorrect

Not covered: the simplify calls in app/context/physical_quantity.py, and the variation code at symbolic.py:615/620. They're bounded only by the 60 s request limit, not by their own fallbacks.

Checklist

• Tests added/updated
  • regression tests for the cases above, plus the request running out of time, in symbolic_evaluation_test.py;
  • unit tests for the limits in app/tests/time_budget_test.py.
  • Tests shorten the limits, so they finish in seconds.
• Docs updated (app/docs.md) if user-facing behaviour changed: only the new feedback message is user-facing.

Testing

• pytest app/tests: 1,223 passed, plus the 1 known test_radians_to_frequency failure (it also fails on Restructure app: split tests, previews and comparison helpers into modules #281).

🤖 Generated with Claude Code

[m-messer]

Two findings from probing this branch locally:

1. The fallback gives a wrong grade. 99^9999999 against answer 1 (default params) returns is_correct: True after about 49 s. When the time budget runs out, a response should be graded incorrect (or "could not evaluate"), never correct.
2. The fix isn't on the refactor stack yet. This PR is based on main, so the stack top (feat/chained-equalities) still hangs (over 40 s) on 99^9999999 and on c - 100000000*log(Abs(1/10), E)/pi against -100000000*log(1/10)/pi.

🤖 Generated with Claude Code

[m-messer]

Finding 1 is fixed in 26d5f16. When the comparison budget ran out, numerically_equal converted values to complex. That turns 99^9999999 (about 9e1995634) into inf without raising an error, so the tolerance check inf <= 1e-30 * inf passed and the response was graded correct. It now returns None (not comparable) when any value isn't finite, so the response is graded incorrect. Two new tests cover it: a unit test on numerically_equal, and an end-to-end 99^9999999 vs 1 test (marked slow, about 45 s). Both fail without the fix.

Finding 2 is still open. feat/chained-equalities (#302) has neither this fix nor current main, so the fix will be ported onto the stack in a separate step.

🤖 Generated with Claude Code