Skip to content

feat: preserve cross-scan finding relationships - #574

Open
mldangelo-oai wants to merge 6 commits into
mainfrom
mdangelo/codex/scan-comparison-storage
Open

feat: preserve cross-scan finding relationships#574
mldangelo-oai wants to merge 6 commits into
mainfrom
mdangelo/codex/scan-comparison-storage

Conversation

@mldangelo-oai

@mldangelo-oai mldangelo-oai commented Aug 20, 2026

Copy link
Copy Markdown
Collaborator

Summary

Preserve confirmed, uncertain, and related relationships between findings in the SQLite workbench so later comparisons can reuse known identities without collapsing independent findings. #575 remains stacked on this storage work.

No user-facing CLI commands, flags, accepted values, or defaults change, and this PR adds no model calls.

Changes

  • Validate complete comparison payloads before replacing saved data.
  • Reuse stable identities and confirmed links while preserving uncertainty and coverage semantics.
  • Return saved related-finding metadata through batched lookups.
  • Add migration 31 with two lookup indexes without rewriting existing history.
  • Use UTF-8 stdin for comparison payloads, with an existing-format fallback for older plugins.
  • Keep legacy records without target IDs scoped to their repository.
  • Preserve the current matcher's input until feat: match repeated findings across scan history #575 consumes the stored historical groups.
  • Merge current origin/main at fa9dfa491a9ed464fbfca9481dc493784a8ef1fc.

Testing

At d7bca7f46e0ff0a8bf1f32080000364d927aef17:

  • bun test --timeout 30000 --seed 12345 ./tests-ts: 1,603 passed, 28 platform-specific skips, 0 failed.
  • Changed-file history, runtime, CLI, and publication-store suites: 314 passed, 11 platform-specific skips, 0 failed.
  • pnpm run types, pnpm run format, pnpm run build, Python syntax parsing for the changed workbench modules, and git diff --check: passed.
  • Packed-package inspection and installed-package smoke: passed for 255 archive entries and 111 bundled plugin files.
  • Regression coverage includes legacy repository scoping, related-pair validation, grouped coverage and uncertainty, migration upgrades, large UTF-8 comparison payloads, current matcher isolation, SQLite batching, and older-plugin fallback.

Risk and rollout

Migration 31 builds indexes under SQLite's normal write lock but does not rewrite saved comparisons or completed scan artifacts.

Older custom plugins keep confirmed and uncertain results, but cannot preserve related links and remain subject to operating-system argument limits for large comparison payloads.

Merge this PR before #575.

Public disclosure review

  • No customer, partner, prospect, or user identities, data, or identifying details are included.
  • No credentials, personal data, private source, scan findings, or nonpublic links or tickets are included.
  • I reviewed the branch name, title, description, commits, changes, comments, logs, screenshots, attachments, and links for public disclosure.

@github-actions github-actions Bot added the enhancement New feature or request label Aug 20, 2026
@mldangelo-oai

Copy link
Copy Markdown
Collaborator Author

@codex review

Please review the current head, cd4b7a67c7c729fac3e4ccc1c33c963abe974374. This is the scan-history storage and compatibility portion of #567. The existing matcher remains in place on this branch.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. You're on a roll.

Reviewed commit: cd4b7a67c7

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@mldangelo-oai

Copy link
Copy Markdown
Collaborator Author

@codex review

The description now includes the final QA results and merge order. Please review the current head, cd4b7a67c7c729fac3e4ccc1c33c963abe974374. The code has not changed since the earlier review.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. 🎉

Reviewed commit: cd4b7a67c7

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Aug 20, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review Completed 2026-08-22T14:53:14.391103Z d7bca7f Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

Copy link
Copy Markdown
Collaborator Author

@codex review

Please review the current head, 02a2e338f9f515a7a7d76d38a17e238bcbd32be0. The PR description includes the latest cleanup and QA results.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Already looking forward to the next diff.

Reviewed commit: 02a2e338f9

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@mldangelo-oai

Copy link
Copy Markdown
Collaborator Author

@codex review

Please review the current head, aa2ed4119c0334b896cef05c426ec1b3a5fea86f. This additive update merges current main, including #573. The PR description has the fresh compatibility and full-suite QA results. Please check migration compatibility, confirmed-link reuse, uncertainty, and coverage handling.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Delightful!

Reviewed commit: aa2ed4119c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

ianw-oai
ianw-oai previously approved these changes Aug 21, 2026
Keep legacy repositories separate when preparing matching inputs, simplify finding alias bookkeeping, and retain the shared workbench stdin API. Use platform-aware Python probes and exercise comparison payloads larger than command-line limits.
@mldangelo-oai

Copy link
Copy Markdown
Collaborator Author

@codex review

Please review exact head 67d380ec17b191bbf31258d0fd6ef4873676de47 against main. This additive update preserves main's stdin and Python-resolution fixes, scopes legacy records without target IDs by repository, simplifies finding-alias bookkeeping, and exercises large comparison payloads. Please check relationship grouping, coverage and uncertainty, migration compatibility, and the transport fallback for older plugins.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Another round soon, please!

Reviewed commit: 67d380ec17

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@mldangelo-oai mldangelo-oai changed the title feat: preserve finding relationships in scan history feat: preserve cross-scan finding relationships Aug 21, 2026
@mldangelo-oai

Copy link
Copy Markdown
Collaborator Author

@codex security review

Please security-review exact head 6ad300c against main. Focus on stored relationship validation, legacy repository scoping, stdin transport, credential isolation, and custom-plugin compatibility.

@mldangelo-oai

Copy link
Copy Markdown
Collaborator Author

@codex review

Please review exact head 6ad300c against main. Focus on relationship grouping, coverage and uncertainty semantics, migration compatibility, current matcher input isolation, and the fallback for older custom plugins.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Something went wrong. Try again later by commenting “@codex review”.

Unknown error
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Breezy!

Reviewed commit: 6ad300c882

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@mldangelo-oai

Copy link
Copy Markdown
Collaborator Author

@codex review

Please review current head d7bca7f46e0ff0a8bf1f32080000364d927aef17.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d7bca7f46e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread sdk/typescript/src/cli.ts
Comment on lines +1492 to +1495
await dependencies.matchFindings({
before: input.before,
after: input.after,
}),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Wire historical groups through the matcher

When compare-scans supplies the newly added knownFindingGroups, this call explicitly reconstructs the input using only before and after; the built-in matcher also still emits only matches and uncertain. Thus ordinary scans match/scans compare executions can neither use confirmed cross-scan identities nor generate the new related pairs, leaving the added persistence and display path unreachable and allowing later matching to contradict saved history. Pass the group data through and extend the matcher input/output schema, including the --all path.

AGENTS.md reference: sdk/typescript/AGENTS.md:L16-L16

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants