Skip to content

Fail calculator parsers without a unary minus - #779

Open
SkyDrive2 wants to merge 1 commit into
sysprog21:mainfrom
SkyDrive2:judge-unary-minus
Open

SkyDrive2 wants to merge 1 commit into
sysprog21:mainfrom
SkyDrive2:judge-unary-minus

Conversation

@SkyDrive2

@SkyDrive2 SkyDrive2 commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

The Budget Sheet Formulas contract in problem-bank/variants.json allows a minus before a number or a parenthesized group, and Jim's clarification gives -(2+3) as an example, but none of the seven cases in problem-bank/judges.json had a minus without a left operand. A parser that reads every minus as binary therefore passed all seven.

Five cases now cover where a unary minus can stand and what it reads: -(2+3) expecting -5 (minus before a group), 1-(-2) expecting 3 (minus inside a group), -2+3 expecting 1 (minus on the first number only), - 7 expecting -7 (space after a leading minus) and -12 expecting -12 (minus before a two-digit number). The labels join the tricky-case list in tests/web/contract.rs. python3 scripts/gen-problems.py regenerated web/judges/budget-sheet-formulas.json, and the basic-calculator digest in tests/golden/task-posed.json is recomputed the way tests/test_task_package.py computes it.

A test in tests/browser/compiler-explorer.test.js, in the style of the valid-anagram one, runs six wrong parsers by name over the built-in cases and fails when no case rejects one of them. They share one recursive-descent parser and differ only in when a - that starts a term is read as a sign and what it negates: never (every minus is binary), only after a 0 is prepended to a leading minus, always and over the rest of the expression, only before a digit, only when it touches its operand with spaces kept, and always but over a single digit. That parser checks the ) that closes a group and fails on input left unread. Each case is the only one that rejects one of the last five parsers, so removing any case fails the test; I checked this by deleting each case from the generated judge in turn. The test also fails against the seven cases on main. The same test checks every expected value with evaluate from web/calculator.js, the task workspace's calculator, instead of a second parser of its own; the wrong parsers keep their own because that calculator has no switch for when a minus is a sign. The calculator refuses an unclosed group or input left unread, so a wrong expected value (such as - 7 expecting 7) or a malformed input (such as 1-(-2(, -(2+3( or -12)) fails the test too. It also reads *, ^, decimals, functions and a plus sign of its own, which the contract leaves out, so each input is first held to the contract's characters (digits, +, -, parentheses and spaces) with no plus sign of its own: 2*3, 1.5+1, +(2+3) or 1-(+2) fails the test.

The whitespace case is - 7 rather than - (2 + 3): every parser but one strips spaces, so - (2 + 3) would reject the same parser as -(2+3) and leave minus before a group unprotected. The two-digit case is -12 rather than -12+3 or 1-(-12): those also reject the parser that negates the rest or the one that prepends 0, so -2+3 or 1-(-2) would no longer be the only case that rejects it.

Each expected value was also checked against a sign-stack solution, which passes all twelve cases.

Verification on Ubuntu 26.04 in WSL2, Node v26.13.1, at this head on main bfadc5e: ./scripts/test.sh rust has no failed gates; node --test tests/browser/compiler-explorer.test.js passes 107/107; python3 -m unittest -k test_posed_bank_golden tests/test_task_package.py passes; python3 scripts/gen-problems.py --check is clean. The full ./scripts/test.sh run reports rust-style, review-memory-tests and git-hooks, and all three fail identically on main at 288a0c9 on this machine: rust-style measures against a stale origin/main in my clone and passes with --baseline upstream/main, and the other two fail the same tests without this change. Not run here: desktop clippy and tests (no libwebkit2gtk), cargo-audit, cargo-deny, ruff, shfmt and shellcheck.

Close #776

cubic-dev-ai[bot]

This comment was marked as resolved.

jserv

This comment was marked as resolved.

jserv

This comment was marked as resolved.

@yunhsuan0510 yunhsuan0510 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Validation performed on PR head 0d0ace8 (Node v26.13.0, Python 3.14.10):

  • node --test --test-name-pattern="basic-calculator cases reject parsers without a unary minus" tests/browser/compiler-explorer.test.js ? 1 passed, 0 failed, 0 skipped.
  • python scripts/gen-problems.py --check ? passed.
  • git diff upstream/pr/779^ upstream/pr/779 --check ? passed.
  • Two targeted mutation probes each pass all 9 current judge cases, but reject 1+-2 and - (2 + 3) respectively.

These are coverage findings; I did not reproduce a failure in an implementation under test.

Comment thread problem-bank/judges.json Outdated
"expected": -5
},
{
"label": "minus after an operator",

@yunhsuan0510 yunhsuan0510 Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we add a case where unary minus directly follows a binary operator, such as 1+-2 with expected value -1? In the current 1-(-2) case, the unary minus follows (, so it does not check a negative number immediately after +. I checked a restricted parser model that only accepts unary minus at the start of an expression or group: it passes all nine current cases but rejects 1+-2. The contract allows unary minus before a number, so this would cover that valid form and make the minus after an operator label accurate.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the probe. I left 1+-2 out for now because it is a question about the contract rather than about coverage. The original problem guarantees no two consecutive operators, so a solution that reads a sign only at the start of an expression or a group is correct there, and this case would fail it. The bank says only that s is a valid expression, and Jim's clarification says a minus can negate a number or a group without saying it may follow +. jserv's thread above renamed the label to minus inside a group, so the label no longer claims an operator before the minus. @jserv, should a minus directly after + or - be in scope? If so I will add 1+-2 with the start-of-group parser you described.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for clarifying the original problem constraint. I was reading the written contract, which allows unary minus before a number or parenthesized group but does not explicitly say it cannot follow a binary operator. If this variant retains the original “no two consecutive operators” restriction, I agree 1+-2 is out of scope; could we state that restriction in the variant contract so the test boundary is clear? Otherwise, 1+-2appears valid under the current wording.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, agreed the contract should say it either way. Whether 1+-2 is allowed is jserv's call, so I'll leave that to him.

Comment thread tests/browser/compiler-explorer.test.js Outdated
test("basic-calculator cases reject parsers without a unary minus", () => {
// Every minus is read as binary, so a term must start with a digit or "(".
const binaryMinusOnly = (s) => {
const text = s.replaceAll(" ", "");

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we add a whitespace case such as - (2 + 3) with expected value -5, and preserve the whitespace in a corresponding mutation probe? The contract allows spaces, but none of the current cases put whitespace between the unary sign and its operand. This helper removes spaces before evaluating its parser variants, so it cannot catch a parser that recognizes unary minus only when - touches the number or group. A targeted whitespace-sensitive parser model passes all nine current cases but rejects - (2 + 3).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes for better user experience.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a whitespace case and a parser that keeps spaces in d372191, as jserv agreed. The case is - 7 expecting -7 (space after a leading minus) rather than - (2 + 3): every other parser strips spaces first, so - (2 + 3) reads as -(2+3) to them and rejects the same parser as minus before a group, which would let that case be removed without the test noticing. - 7 is rejected only by the parser whose unary minus must touch its operand.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, that makes sense. - 7 isolates the whitespace-after-unary-minus case: - (2 + 3) would also be rejected by the parser variant that only supports unary minus before a number. The new whitespace-sensitive parser makes the added case independently useful. This addresses my coverage concern.

@jserv jserv left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Refactor the calculator into a scientific calculator that supports a wide range of mathematical expressions.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 issue found across 5 files (changes from recent commits).

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="problem-bank/judges.json">

<violation number="1" location="problem-bank/judges.json:5444">
P2: The new `-2+3` case only tests unary minus at the start of an expression. A parser that rejects unary minus immediately after a binary operator still passes these cases; add `1+-2` with expected value `-1` to cover that position.</violation>
</file>

Reply with feedback, questions, or to request a fix.

View guided diff | Re-trigger cubic

Comment thread problem-bank/judges.json
"expected": 3
},
{
"label": "minus on the first number only",

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: The new -2+3 case only tests unary minus at the start of an expression. A parser that rejects unary minus immediately after a binary operator still passes these cases; add 1+-2 with expected value -1 to cover that position.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At problem-bank/judges.json, line 5444:

<comment>The new `-2+3` case only tests unary minus at the start of an expression. A parser that rejects unary minus immediately after a binary operator still passes these cases; add `1+-2` with expected value `-1` to cover that position.</comment>

<file context>
@@ -5434,11 +5434,25 @@
         "expected": 3
+      },
+      {
+        "label": "minus on the first number only",
+        "input": [
+          "-2+3"
</file context>

@SkyDrive2

Copy link
Copy Markdown
Contributor Author

@jserv Could you clarify which calculator this refers to? This pull request only adds judge cases for Budget Sheet Formulas (basic-calculator), as scoped in #776, and does not touch the in-interview calculator in web/calculator.js. If you mean extending the Budget Sheet Formulas problem itself, for example with multiplication or functions, that changes the contract in problem-bank/variants.json; should that go in a separate issue?

jserv

This comment was marked as outdated.

@yuen0428 yuen0428 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the unary-minus judge cases and regression test. My local checks used Node v24.22.2 at d372191; the three relevant files are unchanged at the updated head, 2661e18.

The focused regression passed (1 passed, 0 skipped). An additional parser probe showed that both the correct parser and a faulty parser that reads only one digit after unary minus pass all 11 existing cases. The faulty parser rejects valid inputs -12+3 and 1-(-12), while the correct parser returns -9 and 13.

The two inline comments suggest adding a multi-digit negative-number case and a corresponding faulty-parser variant to preserve that coverage.

Comment thread problem-bank/judges.json
{
"label": "minus before a group",
"input": [
"-(2+3)"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we add a case with a multi-digit number directly after unary minus, such as -12+3 expecting -9, or 1-(-12) expecting 13?

I checked d372191 with Node v24.22.2. A recursive-descent parser deliberately changed to consume only one digit after unary minus passes all 11 current judge cases, as does the correct parser. However, the faulty parser rejects both examples above because it leaves the second digit unconsumed. All directly negated numeric literals in the current cases have one digit, so they do not catch this mistake. I also compared these files with the updated head and found no changes.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, confirmed. Added -12 expecting -12 instead: -12+3 and 1-(-12) also catch the parser that negates the rest or prepends 0, so -2+3 or 1-(-2) would no longer be the only case for it.

(s) => evaluate(s, { sign: digit }),
],
[
"the parser whose unary minus must touch its operand",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alongside the multi-digit negative-number case suggested in the judge data, could we add a parser variant that consumes only one digit after unary minus?

The current focused regression passes (1 passed, 0 skipped), but my additional faulty-parser probe also passes all 11 judge cases. A corresponding variant here would protect the new coverage: after adding the multi-digit case, removing it should make this regression fail because that faulty parser is no longer rejected. The probe checks that the entire input was consumed, so the multi-digit examples fail explicitly rather than silently returning a partial result.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added as the parser whose unary minus reads one digit; removing the -12 case now fails the test.

Comment on lines +376 to +378
`the built-in cases must reject ${name}`,
);
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing here checks that a correct parser passes the cases, so the test does not pin the expected values the judge now holds: with - 7 expecting 7 instead of -7, all five wrong parsers are still rejected and the test passes, while every correct candidate solution fails that case. The same happens if a later edit breaks the shared evaluate so it throws on any input. Running the parser whose minus is always a sign over every case closes both:

Suggested change
`the built-in cases must reject ${name}`,
);
}
`the built-in cases must reject ${name}`,
);
}
for (const { input, expected, label } of judges["basic-calculator"].cases) {
assert.equal(evaluate(input[0], { sign: () => true }), expected, label);
}

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Applied in 0518ecb; with - 7 expecting 7, the test now fails on that case.

jserv

This comment was marked as resolved.

Comment on lines +328 to +330
skip();
at += 1;
return value;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The group branch steps past whatever character ends the group without checking it is ), so the reference run below still accepts an input a correct solution refuses. With 1-(-2) changed to 1-(-2(, or -(2+3) to -(2+3(, evaluate still returns 3 and -5, every wrong parser is still rejected by its own case, and the test passes, while every correct candidate solution fails or raises on that case. Check the closing character as the new end-of-input check does:

Suggested change
skip();
at += 1;
return value;
skip();
if (text[at] !== ")") throw new Error(`no ")" at ${at}`);
at += 1;
return value;

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed, thanks! Broken inputs like 1-(-2( and -12) now make the test fail.

jserv
jserv previously requested changes Nov 19, 2026

@jserv jserv left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rebase onto latest main branch and resolve conflicts.

@jserv

jserv commented Nov 19, 2026

Copy link
Copy Markdown
Contributor

Could you clarify which calculator this refers to? This pull request only adds judge cases for Budget Sheet Formulas (basic-calculator), as scoped in #776, and does not touch the in-interview calculator in web/calculator.js.

Always improve code re-usability when possible.

@jserv
jserv dismissed their stale review November 20, 2026 01:25

No merge conflicts at the current head.

// The task workspace's calculator checks every expected value, and refuses
// an unclosed group or input left unread.
for (const { input, expected, label } of judges["basic-calculator"].cases) {
assert.equal(calculate(input[0]), expected, label);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

calculate reads a much wider grammar than evaluateFormula's contract (*, /, ^, decimals, 1e2, a unary +, pi, functions), so this check no longer pins the inputs the way the previous reference did. With a case such as 2*3 expecting 6, or 1.5+1 expecting 2.5, calculate returns the expected value and the test passes, while every correct candidate solution fails or raises on that case; the parser this replaced threw unread input at 1 on both. Checking each input against the contract's characters keeps the shared calculator and refuses those cases (a unary + would still get through):

Suggested change
assert.equal(calculate(input[0]), expected, label);
assert.match(input[0], /^[\d ()+-]+$/, label);
assert.equal(calculate(input[0]), expected, label);

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, used your regex, and added one more line so a unary + gets caught too. 2*3, 1.5+1 and +(2+3) all fail the test now.

The Budget Sheet Formulas contract allows a minus before a number or a
parenthesized group, yet no case had a minus without a left operand, so
a parser that reads every minus as binary passed all seven. Five cases
now put a minus before a group, inside a group, on the first number of
a sum, before a number after a space and before a two-digit number. A
browser test runs six such parsers by name, and each case is the only
one that rejects one of them, so removing any case fails it. The same
test holds every input to the contract's characters and checks every
expected value with the task workspace's calculator, which refuses an
unclosed group or input left unread.

Close sysprog21#776

Change-Id: I5b399c60a94503911b1d1ed2ee79aff46574bb83
@SkyDrive2

Copy link
Copy Markdown
Contributor Author

Always improve code re-usability when possible.

Got it. Rebased onto main, and the test now uses evaluate from web/calculator.js to check the answers instead of its own parser.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Budget Sheet Formulas accepts a solution that cannot parse a unary minus

4 participants