Skip to content

Add --allow CODE to exclude specific problem codes from validate's pass/fail decision #2

Description

@imnasnainaec

Problem

sil-lift validate --strict is meant as a CI conformance gate (see the interop guide), but real-world FieldWorks/FLEx exports trip several warning-level findings that are expected FLEx quirks rather than defects — uri-not-rfc being the clearest example (documented policy in docs/en/guides/validate.md). Right now the only lever is --no-check-media, which is hardcoded to one specific code (missing-media) and fully suppresses it from output rather than just excluding it from the strict decision.

A caller who wants --strict for genuine regressions but doesn't want it to fail on a known-tolerated code (e.g. uri-not-rfc) currently has no way to express that short of post-processing --format json output themselves.

Proposed solution

Add a repeatable --allow CODE flag to validate:

sil-lift validate export.lift --strict --allow uri-not-rfc

Semantics: problems whose code is in the allow-list are still collected and printed/emitted (so a human or a CI log still sees them), but they're excluded from the errors/warnings counts that decide the exit code and from --strict escalation. This differs from --no-check-media, which drops missing-media findings entirely.

Sketch of the affected logic in _cmd_validate (src/sil_lift/_cli.py):

def _cmd_validate(args: argparse.Namespace) -> int:
    problems = _collect_problems(args)
    allowed = set(args.allow)
    counted = [p for p in problems if p.code not in allowed]
    errors = sum(1 for p in counted if p.level == "error")
    warnings = len(counted) - errors
    failed = bool(errors) or (args.strict and bool(warnings))
    ...

For --format json, add an additive summary.allowed count alongside the existing errors/warnings.

Open questions

  • Scope: should --allow apply to error-level codes too (e.g. the trait/field-in-range-element schema errors that are deliberately kept as errors per policy), or warnings only? The mechanism above is symmetric either way — it's a decision about what we want to invite people to silence.
  • SemVer: --format json's shape is a documented, tested interface (see docs/en/guides/lift-export-interop.md). Adding summary.allowed is additive/safe for a minor bump but should be called out explicitly in the CHANGELOG since CI consumers may assert on summary's exact keys.
  • Unknown codes: should --allow on a code that never appears (typo, or a code that doesn't exist) warn/error, or silently no-op? Leaning toward silent no-op for forward-compatibility (a code retired in a later version shouldn't break an existing --allow list).

Alternatives considered

  • A dedicated --flex flag that downgrades a fixed, tool-chosen set of "FLEx-known" warnings to an info level. Rejected: uri-not-rfc is the only warning that's genuinely FLEx-specific under current policy (missing-media, undefined-range-value are generic, source-agnostic data problems a gate might legitimately want to fail on), and introducing a third Problem.level value widens the documented/SemVer-covered JSON schema for one code's benefit. --allow CODE covers the same need generally, without the tool prescribing what counts as "FLEx-known."

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions