Repository navigation
Conversation
yunhsuan0510
left a comment
There was a problem hiding this comment.
Validation performed on PR head 0d0ace8 (Node v26.13.0, Python 3.14.10):
node --test --test-name-pattern="basic-calculator cases reject parsers without a unary minus" tests/browser/compiler-explorer.test.js? 1 passed, 0 failed, 0 skipped.python scripts/gen-problems.py --check? passed.git diff upstream/pr/779^ upstream/pr/779 --check? passed.- Two targeted mutation probes each pass all 9 current judge cases, but reject
1+-2and- (2 + 3)respectively.
These are coverage findings; I did not reproduce a failure in an implementation under test.
| "expected": -5 | ||
| }, | ||
| { | ||
| "label": "minus after an operator", |
There was a problem hiding this comment.
Could we add a case where unary minus directly follows a binary operator, such as 1+-2 with expected value -1? In the current 1-(-2) case, the unary minus follows (, so it does not check a negative number immediately after +. I checked a restricted parser model that only accepts unary minus at the start of an expression or group: it passes all nine current cases but rejects 1+-2. The contract allows unary minus before a number, so this would cover that valid form and make the minus after an operator label accurate.
There was a problem hiding this comment.
Thanks for the probe. I left 1+-2 out for now because it is a question about the contract rather than about coverage. The original problem guarantees no two consecutive operators, so a solution that reads a sign only at the start of an expression or a group is correct there, and this case would fail it. The bank says only that s is a valid expression, and Jim's clarification says a minus can negate a number or a group without saying it may follow +. jserv's thread above renamed the label to minus inside a group, so the label no longer claims an operator before the minus. @jserv, should a minus directly after + or - be in scope? If so I will add 1+-2 with the start-of-group parser you described.
There was a problem hiding this comment.
Thanks for clarifying the original problem constraint. I was reading the written contract, which allows unary minus before a number or parenthesized group but does not explicitly say it cannot follow a binary operator. If this variant retains the original “no two consecutive operators” restriction, I agree 1+-2 is out of scope; could we state that restriction in the variant contract so the test boundary is clear? Otherwise, 1+-2appears valid under the current wording.
There was a problem hiding this comment.
Yeah, agreed the contract should say it either way. Whether 1+-2 is allowed is jserv's call, so I'll leave that to him.
| test("basic-calculator cases reject parsers without a unary minus", () => { | ||
| // Every minus is read as binary, so a term must start with a digit or "(". | ||
| const binaryMinusOnly = (s) => { | ||
| const text = s.replaceAll(" ", ""); |
There was a problem hiding this comment.
Could we add a whitespace case such as - (2 + 3) with expected value -5, and preserve the whitespace in a corresponding mutation probe? The contract allows spaces, but none of the current cases put whitespace between the unary sign and its operand. This helper removes spaces before evaluating its parser variants, so it cannot catch a parser that recognizes unary minus only when - touches the number or group. A targeted whitespace-sensitive parser model passes all nine current cases but rejects - (2 + 3).
There was a problem hiding this comment.
Yes for better user experience.
There was a problem hiding this comment.
Added a whitespace case and a parser that keeps spaces in d372191, as jserv agreed. The case is - 7 expecting -7 (space after a leading minus) rather than - (2 + 3): every other parser strips spaces first, so - (2 + 3) reads as -(2+3) to them and rejects the same parser as minus before a group, which would let that case be removed without the test noticing. - 7 is rejected only by the parser whose unary minus must touch its operand.
There was a problem hiding this comment.
Thanks, that makes sense. - 7 isolates the whitespace-after-unary-minus case: - (2 + 3) would also be rejected by the parser variant that only supports unary minus before a number. The new whitespace-sensitive parser makes the added case independently useful. This addresses my coverage concern.
jserv
left a comment
There was a problem hiding this comment.
Refactor the calculator into a scientific calculator that supports a wide range of mathematical expressions.
0d0ace8 to
d372191
Compare
There was a problem hiding this comment.
1 issue found across 5 files (changes from recent commits).
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="problem-bank/judges.json">
<violation number="1" location="problem-bank/judges.json:5444">
P2: The new `-2+3` case only tests unary minus at the start of an expression. A parser that rejects unary minus immediately after a binary operator still passes these cases; add `1+-2` with expected value `-1` to cover that position.</violation>
</file>
Reply with feedback, questions, or to request a fix.
View guided diff | Re-trigger cubic
| "expected": 3 | ||
| }, | ||
| { | ||
| "label": "minus on the first number only", |
There was a problem hiding this comment.
P2: The new -2+3 case only tests unary minus at the start of an expression. A parser that rejects unary minus immediately after a binary operator still passes these cases; add 1+-2 with expected value -1 to cover that position.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At problem-bank/judges.json, line 5444:
<comment>The new `-2+3` case only tests unary minus at the start of an expression. A parser that rejects unary minus immediately after a binary operator still passes these cases; add `1+-2` with expected value `-1` to cover that position.</comment>
<file context>
@@ -5434,11 +5434,25 @@
"expected": 3
+ },
+ {
+ "label": "minus on the first number only",
+ "input": [
+ "-2+3"
</file context>
|
@jserv Could you clarify which calculator this refers to? This pull request only adds judge cases for Budget Sheet Formulas ( |
d372191 to
2661e18
Compare
yuen0428
left a comment
There was a problem hiding this comment.
Reviewed the unary-minus judge cases and regression test. My local checks used Node v24.22.2 at d372191; the three relevant files are unchanged at the updated head, 2661e18.
The focused regression passed (1 passed, 0 skipped). An additional parser probe showed that both the correct parser and a faulty parser that reads only one digit after unary minus pass all 11 existing cases. The faulty parser rejects valid inputs -12+3 and 1-(-12), while the correct parser returns -9 and 13.
The two inline comments suggest adding a multi-digit negative-number case and a corresponding faulty-parser variant to preserve that coverage.
| { | ||
| "label": "minus before a group", | ||
| "input": [ | ||
| "-(2+3)" |
There was a problem hiding this comment.
Could we add a case with a multi-digit number directly after unary minus, such as -12+3 expecting -9, or 1-(-12) expecting 13?
I checked d372191 with Node v24.22.2. A recursive-descent parser deliberately changed to consume only one digit after unary minus passes all 11 current judge cases, as does the correct parser. However, the faulty parser rejects both examples above because it leaves the second digit unconsumed. All directly negated numeric literals in the current cases have one digit, so they do not catch this mistake. I also compared these files with the updated head and found no changes.
There was a problem hiding this comment.
Thanks, confirmed. Added -12 expecting -12 instead: -12+3 and 1-(-12) also catch the parser that negates the rest or prepends 0, so -2+3 or 1-(-2) would no longer be the only case for it.
| (s) => evaluate(s, { sign: digit }), | ||
| ], | ||
| [ | ||
| "the parser whose unary minus must touch its operand", |
There was a problem hiding this comment.
Alongside the multi-digit negative-number case suggested in the judge data, could we add a parser variant that consumes only one digit after unary minus?
The current focused regression passes (1 passed, 0 skipped), but my additional faulty-parser probe also passes all 11 judge cases. A corresponding variant here would protect the new coverage: after adding the multi-digit case, removing it should make this regression fail because that faulty parser is no longer rejected. The probe checks that the entire input was consumed, so the multi-digit examples fail explicitly rather than silently returning a partial result.
There was a problem hiding this comment.
Added as the parser whose unary minus reads one digit; removing the -12 case now fails the test.
| `the built-in cases must reject ${name}`, | ||
| ); | ||
| } |
There was a problem hiding this comment.
Nothing here checks that a correct parser passes the cases, so the test does not pin the expected values the judge now holds: with - 7 expecting 7 instead of -7, all five wrong parsers are still rejected and the test passes, while every correct candidate solution fails that case. The same happens if a later edit breaks the shared evaluate so it throws on any input. Running the parser whose minus is always a sign over every case closes both:
| `the built-in cases must reject ${name}`, | |
| ); | |
| } | |
| `the built-in cases must reject ${name}`, | |
| ); | |
| } | |
| for (const { input, expected, label } of judges["basic-calculator"].cases) { | |
| assert.equal(evaluate(input[0], { sign: () => true }), expected, label); | |
| } |
There was a problem hiding this comment.
Applied in 0518ecb; with - 7 expecting 7, the test now fails on that case.
2661e18 to
0518ecb
Compare
0518ecb to
10027e6
Compare
| skip(); | ||
| at += 1; | ||
| return value; |
There was a problem hiding this comment.
The group branch steps past whatever character ends the group without checking it is ), so the reference run below still accepts an input a correct solution refuses. With 1-(-2) changed to 1-(-2(, or -(2+3) to -(2+3(, evaluate still returns 3 and -5, every wrong parser is still rejected by its own case, and the test passes, while every correct candidate solution fails or raises on that case. Check the closing character as the new end-of-input check does:
| skip(); | |
| at += 1; | |
| return value; | |
| skip(); | |
| if (text[at] !== ")") throw new Error(`no ")" at ${at}`); | |
| at += 1; | |
| return value; |
There was a problem hiding this comment.
Fixed, thanks! Broken inputs like 1-(-2( and -12) now make the test fail.
jserv
left a comment
There was a problem hiding this comment.
Rebase onto latest main branch and resolve conflicts.
Always improve code re-usability when possible. |
10027e6 to
8ebfbdd
Compare
| // The task workspace's calculator checks every expected value, and refuses | ||
| // an unclosed group or input left unread. | ||
| for (const { input, expected, label } of judges["basic-calculator"].cases) { | ||
| assert.equal(calculate(input[0]), expected, label); |
There was a problem hiding this comment.
calculate reads a much wider grammar than evaluateFormula's contract (*, /, ^, decimals, 1e2, a unary +, pi, functions), so this check no longer pins the inputs the way the previous reference did. With a case such as 2*3 expecting 6, or 1.5+1 expecting 2.5, calculate returns the expected value and the test passes, while every correct candidate solution fails or raises on that case; the parser this replaced threw unread input at 1 on both. Checking each input against the contract's characters keeps the shared calculator and refuses those cases (a unary + would still get through):
| assert.equal(calculate(input[0]), expected, label); | |
| assert.match(input[0], /^[\d ()+-]+$/, label); | |
| assert.equal(calculate(input[0]), expected, label); |
There was a problem hiding this comment.
Thanks, used your regex, and added one more line so a unary + gets caught too. 2*3, 1.5+1 and +(2+3) all fail the test now.
The Budget Sheet Formulas contract allows a minus before a number or a parenthesized group, yet no case had a minus without a left operand, so a parser that reads every minus as binary passed all seven. Five cases now put a minus before a group, inside a group, on the first number of a sum, before a number after a space and before a two-digit number. A browser test runs six such parsers by name, and each case is the only one that rejects one of them, so removing any case fails it. The same test holds every input to the contract's characters and checks every expected value with the task workspace's calculator, which refuses an unclosed group or input left unread. Close sysprog21#776 Change-Id: I5b399c60a94503911b1d1ed2ee79aff46574bb83
8ebfbdd to
02a8429
Compare
Got it. Rebased onto |
The Budget Sheet Formulas contract in
problem-bank/variants.jsonallows a minus before a number or a parenthesized group, and Jim's clarification gives-(2+3)as an example, but none of the seven cases inproblem-bank/judges.jsonhad a minus without a left operand. A parser that reads every minus as binary therefore passed all seven.Five cases now cover where a unary minus can stand and what it reads:
-(2+3)expecting-5(minus before a group),1-(-2)expecting3(minus inside a group),-2+3expecting1(minus on the first number only),- 7expecting-7(space after a leading minus) and-12expecting-12(minus before a two-digit number). The labels join the tricky-case list intests/web/contract.rs.python3 scripts/gen-problems.pyregeneratedweb/judges/budget-sheet-formulas.json, and thebasic-calculatordigest intests/golden/task-posed.jsonis recomputed the waytests/test_task_package.pycomputes it.A test in
tests/browser/compiler-explorer.test.js, in the style of thevalid-anagramone, runs six wrong parsers by name over the built-in cases and fails when no case rejects one of them. They share one recursive-descent parser and differ only in when a-that starts a term is read as a sign and what it negates: never (every minus is binary), only after a0is prepended to a leading minus, always and over the rest of the expression, only before a digit, only when it touches its operand with spaces kept, and always but over a single digit. That parser checks the)that closes a group and fails on input left unread. Each case is the only one that rejects one of the last five parsers, so removing any case fails the test; I checked this by deleting each case from the generated judge in turn. The test also fails against the seven cases onmain. The same test checks every expected value withevaluatefromweb/calculator.js, the task workspace's calculator, instead of a second parser of its own; the wrong parsers keep their own because that calculator has no switch for when a minus is a sign. The calculator refuses an unclosed group or input left unread, so a wrong expected value (such as- 7expecting7) or a malformed input (such as1-(-2(,-(2+3(or-12)) fails the test too. It also reads*,^, decimals, functions and a plus sign of its own, which the contract leaves out, so each input is first held to the contract's characters (digits,+,-, parentheses and spaces) with no plus sign of its own:2*3,1.5+1,+(2+3)or1-(+2)fails the test.The whitespace case is
- 7rather than- (2 + 3): every parser but one strips spaces, so- (2 + 3)would reject the same parser as-(2+3)and leaveminus before a groupunprotected. The two-digit case is-12rather than-12+3or1-(-12): those also reject the parser that negates the rest or the one that prepends0, so-2+3or1-(-2)would no longer be the only case that rejects it.Each expected value was also checked against a sign-stack solution, which passes all twelve cases.
Verification on Ubuntu 26.04 in WSL2, Node v26.13.1, at this head on
mainbfadc5e:./scripts/test.sh rusthas no failed gates;node --test tests/browser/compiler-explorer.test.jspasses 107/107;python3 -m unittest -k test_posed_bank_golden tests/test_task_package.pypasses;python3 scripts/gen-problems.py --checkis clean. The full./scripts/test.shrun reportsrust-style,review-memory-testsandgit-hooks, and all three fail identically onmainat 288a0c9 on this machine:rust-stylemeasures against a staleorigin/mainin my clone and passes with--baseline upstream/main, and the other two fail the same tests without this change. Not run here: desktop clippy and tests (no libwebkit2gtk), cargo-audit, cargo-deny, ruff, shfmt and shellcheck.Close #776