fix(engine): correct invalid-JSON error type and mis-targeted sync test - #193
Conversation
Two pre-existing bugs surfaced during the SonarCloud remediation (#191): 1. engine/security/{P2_9_llm_schemas,_5_llm_security}.py raised pydantic ValidationError with a plain string on the JSON-parse path. In pydantic v2 ValidationError is not string-constructible, so the invalid-JSON branch raised TypeError instead of a meaningful error. Raise ValueError (which pydantic.ValidationError subclasses, so broad-catch callers are unaffected); schema-shape failures still raise pydantic ValidationError via model_validate. Drop the now-unused import. 2. tests/integration/test_sync_handler.py asserted a non-existent SyncGenerator.resolve_endpoint and only passed because pytest.raises swallowed the AttributeError. Rewrite it to exercise the real RULE 3 path: handle_sync rejects an unknown entity_type with ValidationError ("No sync endpoint for entity type") during endpoint resolution, before any Neo4j access (mock driver + real domain loader). test_invalid_json updated to assert the corrected ValueError. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01USEi3CurG9BEbx5bTHHGMx
|
✅ PR reviewable size is within recommended limits |
L9 Audit Harness Report
Step Results
Architecture Audit Findings
See Spec Coverage
See Next StepsAll checks passed. Safe to merge. |
There was a problem hiding this comment.
Pull request overview
This PR fixes two pre-existing engine issues: (1) invalid-JSON handling in LLM helpers raising TypeError due to incorrect pydantic.ValidationError construction in Pydantic v2, and (2) a sync test that was asserting a non-existent method instead of exercising the real unknown-entity-type rejection path.
Changes:
- Replace invalid
ValidationError("...")construction on JSON parse failures with a properValueError(preserving schema-validationValidationErrorbehavior viamodel_validate). - Update LLM JSON validation tests to assert
ValueErrorwith an “invalid JSON” message. - Rewrite the “unknown entity_type” sync test to call
handle_syncand assert the realengine.handlers.ValidationErrorrejection path.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| tests/test_algorithmic_upgrades.py | Updates invalid-JSON expectation from TypeError to ValueError with message match. |
| tests/integration/test_sync_handler.py | Replaces a non-exercising test (nonexistent method) with an async handler-based assertion of unknown entity type rejection. |
| engine/security/P2_9_llm_schemas.py | Fixes invalid-JSON error path to raise ValueError instead of incorrectly constructing Pydantic ValidationError. |
| engine/security/_5_llm_security.py | Fixes invalid-JSON error path to raise ValueError (but docstring still needs to reflect this). |
| logger.exception("LLM returned invalid JSON") | ||
| raise ValidationError(f"LLM output is not valid JSON: {e}") | ||
| # pydantic's ValidationError is not string-constructible in v2; raise a | ||
| # ValueError (its superclass) so the JSON-parse failure surfaces cleanly. | ||
| msg = f"LLM output is not valid JSON: {e}" | ||
| raise ValueError(msg) from e |
…utput Addresses PR review: the JSON-parse path now raises ValueError (not pydantic ValidationError), so the docstring Raises section is updated to reflect the real public contract. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01USEi3CurG9BEbx5bTHHGMx
|
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 4 out of 4 changed files in this pull request and generated no new comments.
Suppressed comments (1)
tests/integration/test_sync_handler.py:55
handle_sync()enforcesTENANT_ALLOWLISTbefore endpoint resolution. If the environment (CI/dev) sets a allowlist that doesn’t includeplasticos, this test will fail with a different ValidationError than expected. Consider pinningTENANT_ALLOWLISTviamonkeypatchfor determinism, and callinit_dependencieswith keyword args (most tests do; e.g.tests/integration/test_handlers.py:67).
from engine.handlers import ValidationError, handle_sync, init_dependencies
from engine.state import get_state
get_state().reset()
init_dependencies(AsyncMock(), domain_loader)



Summary
Fixes the two pre-existing engine bugs surfaced (and flagged, not fixed) during the SonarCloud remediation in #191. Small, focused, behavior-correcting — 4 files.
Bug 1 — invalid-JSON path raised
TypeErrorinstead of a real errorengine/security/P2_9_llm_schemas.pyandengine/security/_5_llm_security.pyconstructed a pydanticValidationErrorfrom a plain string on the JSON-parse branch. In pydantic v2ValidationErroris not string-constructible (needsfrom_exception_data), sovalidate_llm_json("not json", …)raisedTypeErrorwhile building the error — masking the actual "invalid JSON" condition.ValueErroron the JSON-parse failure.pydantic.ValidationErrorsubclassesValueError, so any caller doing broadexcept ValueErrorstill catches schema-shape failures too — no behavior regression.ValidationErrorviamodel_validate(e.g. the destructive-Cypher validator), unchanged.ValidationErrorimport inP2_9_llm_schemas.py.Bug 2 — test asserted a method that doesn't exist
tests/integration/test_sync_handler.py::test_sync_unknown_entity_type_raisescalledSyncGenerator.resolve_endpoint(...), which does not exist; the test only passed becausepytest.raises(Exception/AttributeError)swallowed the resultingAttributeError. It never exercised the behavior its docstring claims ("RULE 3: unknown entity_type is rejected").handle_syncresolvesentity_typeto a declared sync endpoint and raises the engineValidationError("No sync endpoint for entity type …") when none matches. That rejection happens during endpoint resolution, before any Neo4j access, so the test uses the real domain loader + a mock driver (no testcontainers needed).test_invalid_jsonupdated to assert the correctedValueError(was asserting the buggyTypeError).Validation (CI-pinned toolchain)
ruff check/ruff format --check✅ ·mypy engine/✅ (139 files) ·tools/contract_scanner.py✅ no violationstest_algorithmic_upgrades.py,test_sync_handler.py): 30 passed, 2 skipped (the 2 skips are the Neo4j-only sync tests; the new RULE-3 test runs without Neo4j by design)Generated by Claude Code