[GP-02] Support the power operator `^` in the expression language, as in Antares Simulator

[aoustry]

Affected Component: Expression parsing / grammar (expression/, grammar/)

Type of Change: Minor — backward-compatible new feature (new operator)

Description

Antares Simulator supports the binary power operator ^ in the GEMS expression language; GemsPy does not. A library YAML that is valid for the C++ interpreter therefore fails to load in GemsPy (^ is not even a lexer token, so parsing raises a ParsingException).

Reference behaviour in Antares Simulator

Grammar (src/expressions/antlr-interface/Expr.g4) — right-associative, binding tighter than * / and + -, and available in the three expression rules:

expr
    : ...
    | <assoc=right> expr '^' expr              # power
    | expr op=('/' | '*') expr                 # muldiv
    ...
shift_expr
    : <assoc=right> shift_expr '^' right_expr  # shiftPower
    ...
right_expr
    : <assoc=right> right_expr '^' right_expr  # rightPower
    ...

Semantics (docs/user-guide/modeler/09-expressions.md, "Power operator"):

This binary operator ^ is used within any expression, but with following restrictions. In the context of a linear problem construction (any context but extra-output), its operands can only be literals or parameters. Within an extra-output expression, references to variables are allowed.

Implementation-wise, ^ maps to FunctionNode(FunctionNodeType::pow, base, exponent):

• EvalVisitor::visitPow evaluates it with std::pow (post-solve / constant evaluation);
• VariabilityVisitor::visitPow requires the exponent to be constant, and propagates the base's variability;
• ReadLinearExpressionVisitor::visitPower (linear-problem construction) rejects a non-scalar exponent ("exponent must be constant") and raises the constant left-hand expression to that power.

So p^2, 2^p, p^(1 + q) are fine in constraints/objective (parameters and literals only), x^2 is rejected there, and myVar^(2 + myParam) is allowed in extra-outputs.

This is exactly the split GemsPy already implements for floor, ceil, abs, round, min, max: allowed on degree-0 expressions in constraints/bounds/objective, allowed on variables in extra-outputs (post-solve evaluation).

Precedence: GemsPy must follow the standard mathematical convention

Decision: -2^2 must parse as -(2^2) = -4, i.e. ^ binds tighter than unary minus, as in standard mathematical notation (and as in Python, MATLAB, Julia, LaTeX-era convention). 2^-3 must remain valid and parse as 2^(-3).

This is a deliberate deviation from the current Antares Simulator behaviour. In ANTLR4 the order of alternatives sets precedence, and Antares declares '-' expr # negation before the power alternative, so -2^2 currently parses there as (-2)^2 = 4. A separate issue will be opened on Antares Simulator to align it with the standard convention.

Concretely, in grammar/Expr.g4 the power alternative must sit above negation (and above muldiv):

expr
    : atom                                     # unsignedAtom
    | portFieldExpr                            # portField
    | <assoc=right> expr '^' expr              # power      <-- above negation
    | '-' expr                                 # negation
    ...

which gives: -2^2 = -(2^2), 2^-3 = 2^(-3) (unary minus is reachable as a primary alternative on the right of ^), 2^3^2 = 2^(3^2), 2*3^2 = 2*(3^2), 2^3*2 = (2^3)*2.

The same convention must hold inside time-shift expressions, and the shift_expr / right_expr sub-grammar needs care to get there — copying Antares' alternatives verbatim would not be enough:

• signedAtom (op=('+'|'-') atom) binds the leading sign to a bare atom, so x[t-2^2] would parse as (-2)^2 — the sign's operand must be widened to a power-capable operand for the convention to hold;
• shift_expr '^' right_expr puts a full right_expr on the right of ^, and since that is a separate rule it is entered at precedence 0 and greedily swallows * / / — so x[t-2^2*3] would parse as 2^(2*3) instead of (2^2)*3. The right-hand side of ^ needs a dedicated operand rule (atom or parenthesised expression, right-associatively chained) rather than right_expr.

Both points are worth reporting on the Antares side as well.

Proposed scope

1. Grammar (grammar/Expr.g4) — add ^ to expr, right_expr and the shift sub-grammar with the precedence described above. Regenerate the ANTLR parser with grammar/generate-parser.sh (do not hand-edit expression/parsing/antlr/). Note the script's output path is stale (src/gems/expression/...) and should be fixed to src/gems_craft/expression/parsing/antlr as part of this work.
2. AST (gems_craft/expression/expression.py) — new PowerNode(BinaryOperatorNode) plus __pow__ / __rpow__ on ExpressionNode, so the Python API can write param("p") ** 2 alongside the parsed string form.
3. Parsing visitor (expression/parsing/parse_expression.py) — visit methods for the new alternatives.
4. Visitor interface (expression/visitor.py) — new abstract power(...) method, implemented in every visitor: CopyVisitor, PrinterVisitor, EqualityVisitor, ExpressionDegreeVisitor, TimeScenarioIndexingVisitor, UsesSumConnectionsOnVisitor, _PortFieldExpressionChecker, _ForbidBarePortFieldVisitor, EvaluationVisitor, VectorizedBuilderBase (+ LinearExpressionBuilder override in linearize.py), _ShiftAmountEvaluator, ShiftValidityVisitor. Adding __pow__ to the SupportsOperations protocol lets ExpressionVisitorOperations provide the default for the arithmetic visitors.
5. Degree / linearity (expression/degree.py, consumed by _forbid_nonlinear in model/resolve_library.py) — the rule that keeps parity with Antares: exponent must have degree 0; if the base has degree 0 the result is degree 0; otherwise degree = base_degree * exponent when the exponent is a non-negative integer literal, inf otherwise. inf in a constraint/objective/bound context then produces the existing "Non-linear expression is not allowed in ..." error, while extra-outputs still accept it.
6. Evaluation — numpy.power in the vectorized builders and ** in EvaluationVisitor, so x^2 works in extra-outputs and in post-solve port-field definitions, like the other nonlinear operators.
7. Docs — expression-syntax section (documenting the -2^2 = -(2^2) convention explicitly) + docs/CHANGELOG.md entry under Unreleased.

Open question for triage: whether to also accept ** as an alias in parsed strings. Recommendation is no — Antares only accepts ^, and keeping the two languages identical is the point of this issue.

Results Impact

No change to solver results for existing studies. This is a purely additive grammar/AST change: ^ is currently a parse error, so no existing GEMS library or system file can contain it, and no existing expression takes a different path. New alternatives are added to the grammar rules but the precedence of the existing operators is unchanged (power is inserted above muldiv and negation, which does not reorder any + - * / parse).

The only behavioural change is that expressions previously rejected at parse time are now accepted, and expressions using ^ on variables outside extra-outputs are rejected with the existing non-linearity error rather than a parse error.

Note that until the Antares side is aligned, -2^2 evaluates to -4 in GemsPy and 4 in Antares Simulator; the same model would then give different results in the two interpreters. This is the one intentional divergence, and it is the reason for the companion Antares issue.

Validation Strategy

Unit tests:

• tests/unittests/gems_craft/expressions/parsing/test_expression_parsing.py — p^2, 2^p, p^(1+q), x[t-1]^2, and precedence/associativity cases compared against the equivalent explicitly-parenthesised AST: -2^2 == -(2^2), 2^-3 == 2^(-3), 2^3^2 == 2^(3^2), 2*3^2 == 2*(3^2), 2^3*2 == (2^3)*2, plus the shift-context cases x[t-2^2] == x[t-4] and x[t-2^2*3] == x[t-12].
• test_printer.py / test_equality.py / copy round-trip — parse → print → re-parse keeps the same AST, with the parentheses needed by right-associativity and by the unary-minus precedence.
• degree tests — degree 0 for p^2, 2 for x^2, inf for x^p, and ValueError("Non-linear expression is not allowed in ...") for x^2 in a constraint (mirroring the existing test_lib_parsing.py non-linearity tests).
• tests/unittests/gems_runner/expression/test_evaluation.py and test_simulation_table_extra_outputs.py — numeric evaluation of myVar^(2 + myParam) in an extra-output, including negative and fractional exponents.
• test_vectorized_linear_expr_builder.py — parameter-only powers built into a linear problem give the same coefficients as the manually expanded expression.

E2E: solver equivalence must hold for the existing reference studies (none use ^); one reference study (or an existing one extended) should exercise ^ in a parameter expression and in an extra-outputs expression, with expected values documented.

Cross-check against Antares Simulator on the same model to confirm identical numeric output — excluding the -2^2 precedence case, which is expected to differ until the companion Antares issue is resolved.

Process Checklist

Step 1 — Issue Creation

• Issue created and linked to process GP-02

Step 2 — Triage

• Process confirmed applicable
• Assigned to responsible contributor
• Priority and milestone set (if applicable)

Step 3 — Impact Analysis ⚠️ extended results analysis required

• Affected modules identified
• Results impact explicitly stated: change intended / no change intended
• If results expected to change: before/after difference described and justified
• Breaking vs backward-compatible change determined

Step 4 — Implementation

• Code changes implemented
• Backward compatibility maintained unless explicitly intended as breaking

Step 5 — Testing & Validation

• If results expected to change: reference studies updated with new expected values
• If results must not change: solver equivalence confirmed via tests
• Unit tests cover changed components

Step 6 — CI Validation

• Type checking passes (mypy)
• Formatting passes (black, isort)
• All tests pass in CI (pytest)

Step 7 — Review & Merge

• PR reviewed; documentation changes included

Step 8 — Versioning

• pyproject.toml version bumped

Step 9 — Supporting Files

• AGENTS.md reviewed for impact and updated if needed

Step 10 — Release

• If a release is needed: follow the release process in the Developer Guidelines