Repair the factory feedback comment (step 1) - #156
Merged
Conversation
Implement section 5 of docs/feedback-threads.md, and record the section 6 spike result. Step 2 (review threads) stays unbuilt. The hubspot scope check printed 10 questions where 5 exist. The extractor truncated all 53 open-question bullets in the repository at the first line break. The snowflake pipeline review asked 2 questions that the same run had already refuted. Changes: - Add text-similarity.ts. It decides when two questions say the same thing. The threshold is 0.6. The measured band is narrow: the highest score between two different questions is 0.5882, and the lowest score between a true duplicate pair is 0.6000. A test pins both sides. - scope-gate.ts keeps the whole bullet. It joins each continuation line. - mergeOpenQuestions merges a dossier bullet into the report question that already says it. The report text always wins, because isMaterialOpenQuestion fires on the report paraphrase and not on the longer dossier bullet. - findings.ts drops a lint finding that a review round refuted. - format-scope-check.ts suppresses duplicate soft questions. - format-pipeline-review.ts collapses the nits into a details element. - format-summary.ts posts a short issue comment with a link. Three deviations from the design document, each with evidence: - 5.2 asks for a prefix rule. No prefix pair exists in any of the 60 run records, so the rule removes nothing. Containment overlap replaces it. - 5.4 asks to extend partitionFindings to the scope path. That function keys on target plus where, and the scope path carries bare strings. - 5.5 asks to shrink both long copies. cmd-pr.ts passes an empty prUrl, because the pull request does not exist yet at that moment. The pull request body stays the canonical copy. Only the issue comment shrinks. The spike answer: a file-level review thread does not go Outdated after a commit rewrites the file. A line-level control thread on the same file does. Section 7.3 needs no change. Tests go from 112 to 157. Every fixture is a copy under pipeline/src/, because pipeline-ci.yml triggers only on the pipeline path. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements section 5 of
docs/feedback-threads.md, and records the section 6 spike result. Step 2 (review threads) stays unbuilt.The problem
Three defects made the factory comment worse than the work behind it.
external.md is missingafter the same run statedexternal.md existsThe change
text-similarity.ts(new). It decides when two questions say the same thing. The threshold is0.6. The measured band is narrow: the highest score between two genuinely different questions is0.5882, and the lowest score between a true duplicate pair is0.6000. A test pins both sides, so a silent retune fails the suite.scope-gate.tskeeps the whole bullet. It joins each continuation line.mergeOpenQuestionsmerges a dossier bullet into the report question that already says it. The report text always wins.findings.tsdrops a lint finding that a review round refuted. The rule is narrow on purpose: a<file>.md is missing.problem, and only when a disputed line claims that same file exists.format-scope-check.tssuppresses duplicate soft questions.format-pipeline-review.tscollapses the nits into a<details>element. The old threshold never fired: the highest nit count in all 60 run records is 9, and the branch needs more than 12.format-summary.ts(new) posts a short issue comment with a link. It measures 446 characters against 3800 for the full comment.Results
Tests go from 112 to 157.
tsc --noEmitis clean.Three deviations from the design document
Each one is measured, and each is recorded in
disagreements_with_designterms in the commit message.softoropen_questionsarray across all 60 run records, so the rule takes 10 questions in and returns 10. Containment overlap replaces it.isMaterialOpenQuestionfires on the short report paraphrase and not on the longer dossier bullet. The merge keeps the report text instead.partitionFindingsto the scope path. That function keys ontargetpluswhere, and the scope path carries bare strings.cmd-pr.ts:101passes an emptyprUrl, because the pull request does not exist yet at that moment. The pull request body stays the canonical copy. Only the issue comment shrinks.The document also stated the hubspot baseline as 9 questions. The real baseline is 10: 1 decision plus 9 soft.
The spike
A file-level review thread does not go Outdated after a commit rewrites the file. A line-level control thread on the same file does, which proves the test worked. Section 7.3 needs no change.
The spike ran on a throwaway pull request, not on #143.
guide/issue-142-hubspotis the live resume source for issue #142, so a spike commit would change what the next factory run reads. Every spike artifact is cleaned up:pulls/commentsreturns[], #143 sits at the same sha with 0 review comments, and the throwaway branch is deleted.Section 6 records the required field set, the verbatim 422 error text, the identifiers the API returned, and one correction: REST
positiondoes not becomenullon an outdated thread. The field that changes isline.Two items for later, not fixed here
severity: "blocker"with nodimension, soisGatedemotes them to "Optional nits". They are in the twogoogle-big-queryrecords. Collapsing the nits hides them one click deeper. A test pins the current behaviour, so a later fix shows up as a test change.0.4286. Section 5.1 fixes it for future runs only.🤖 Generated with Claude Code