Repository navigation
ci: check license headers with skywalking-eyes - #62
Conversation
d51573d to
9f8b87e
Compare
vchamarthi
left a comment
There was a problem hiding this comment.
Thanks for this! The exemptions are well reasoned (agent-read text, uv-generated locks, templates), the action is pinned to the v0.9.0 commit, and the CONTRIBUTING note makes the rule easy to follow. One thing to sort out before merge: main has moved on since this branch was cut, see inline.
| # persona prompt or skill being measured. | ||
| - 'CLAUDE.md' | ||
| - 'dokimi/data/agent_profiles/**' | ||
| - 'dokimi/data/skills/**' |
There was a problem hiding this comment.
I think this needs a rebase on main before merge: #57 added .agent-eval/, and 25 of its files have no SPDX header (the Dockerfile, README.md, every check.py, the hidden tests, the *.patch files and the oracle_answer.md files). Since pull_request runs check out the merge ref, this job should fail here and on main once it lands. The task fixtures look like they belong with the other verbatim exemptions, since a header in oracle.patch/setup.patch, oracle_answer.md or a hidden test changes what the agent sees or what gets applied. Could we add .agent-eval/tasks/** to paths-ignore and put headers on .agent-eval/Dockerfile and .agent-eval/README.md?
There was a problem hiding this comment.
Rebased on main (96e355a). On .agent-eval/ I kept the split from e80f53c rather than ignoring all of tasks/**: the files that reach the agent or the workspace verbatim are exempt (*.patch, which covers setup.patch/oracle.patch/review.patch, and oracle_answer.md, copied to answer.txt). check.py is run by the harness and the hidden/ tests are copied in only after the agent finishes (run.py _hidden_tests, after process_metrics), so a header there changes nothing the agent sees or what gets applied. Dockerfile and README.md have headers. Happy to widen it to tasks/** if you would rather not reason per file.
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #62 +/- ##
=======================================
Coverage 89.10% 89.10%
=======================================
Files 204 204
Lines 23557 23557
Branches 3779 3779
=======================================
Hits 20990 20990
Misses 1862 1862
Partials 705 705 🚀 New features to boost your workflow:
|
9f8b87e to
595043b
Compare
|
Checked at 595043b by merging main into it and running skywalking-eyes 0.9.0 (Docker image), plus the suite, ruff, mypy, zizmor and actionlint. The 65 files are pure insertions (334 added lines, 0 deleted; the old file equals the new one minus the inserted lines, shebangs stay on line 1, the AST and
The |
antonwolfy
left a comment
There was a problem hiding this comment.
Review at f239ba1. The headers are right: 334 pure insertions, the YAML, TOML and requirements files still load, the docs still build with --strict, and the exemptions match what agents read verbatim. The problem is the merge. The job goes red as soon as this branch meets current main, and nothing in make ci-local or AGENTS.md would tell a contributor about the rule.
What I ran, on a Linux review host at f239ba1:
make ci-local: 18 passed, 1 failed, 4 skipped; 4,293 tests. The one failure isschema-compat: the branch is 4 commits behind main and lacks the twotool_preference*.v1.jsonschemas that main added (#53), so it reports them as removed. It is a stale-base artifact, not something this PR does; it goes away after the rebase. No Python lines changed, so the changed-line coverage gate had nothing to measure.- A header scan of the head tree (the config's ignore list, the header form, the first 120 characters): 498 tracked files, 0 without a header. Applying the same scan to
git merge-tree origin/main HEAD(the tree CI checks out on apull_requestrun; the merge has no conflicts): 507 files, 1 without a header. - Targeted tests for every consumer of the 65 files: the YAML registries and explore tasks,
pyproject.toml, the requirements files, the docs tests,mkdocs build --strict, and the guidance self-scan (fail 0; claims go from 198 to 200 because of the new CONTRIBUTING text). The.agent-evalcheck.pyshebangs stay on line 1 and the docstrings stay the first statement. The tasks apply their patches onto the workspace's start commit, so headers on main do not touch them. - I did not run the license-eye binary itself. Everything about its matching is from the pattern as written, simulated with Python
re.
Checked, no issues
- Content:
git diff --numstatshows 0 deletions in all 65 files; only.licenserc.yaml, theci.ymljob and thedokimi/CONTRIBUTING.mdnote carry anything other than a header. - Workflow:
zizmor --offline --min-severity mediumreports no findings; the checkout SHA and# v7.0.1comment are identical to the other 12 uses in the workflows; the job setspersist-credentials: falseand inheritscontents: read. - Exemptions: each one is justified (JSON, uv locks, templates, license texts, agent-read text, eval oracles). No exempt file carries a header and no other file is missing one.
- Docs metrics: the repo scores its own docs with
benchmarks/spec.v1.yaml. The header moves the readability and coverage scores of 19 of 21 docs by less than 0.01, far below thescore_drop_warnof 0.03.
f239ba1 to
ff5dd7a
Compare
antonwolfy
left a comment
There was a problem hiding this comment.
LGTM at ff5dd7a. The earlier problems are fixed: the branch is rebased on main, tool-preference.md has its header, the rule is in AGENTS.md and both CONTRIBUTING.md files, and the pattern's tail now rejects -only, + and OR. Two nits inline.
What I ran, on a Linux review host at ff5dd7a:
make ci-local: 18 passed, 1 failed, 4 skipped. The one failure,changelog.yml:fragment, is local only:ci-localdoes not see PR labels, and with this PR'sskip-changeloglabel the Changelog fragment check passes on GitHub. Tests, mypy, schema-compat, zizmor, pre-commit, pip-audit, bandit, package smoke andmkdocs build --strictpass. No Python lines changed, so the changed-line coverage gate had nothing to measure.- A header scan of the tracked tree with the config's ignore list and the pattern, simulated with Python
re(I did not run the license-eye binary): 534 tracked files, 455 not ignored, 453 carry a matching header. The other two aredokimi/LICENSEanddokimi/NOTICE. They are in the ignore list only as bareLICENSEandNOTICE, and the License header check passes on GitHub at ff5dd7a, so 0.9.0 matches a bare name at any depth. - Probes of the pattern's tail with
re(see the nit inline). - Diff against the base: 394 pure insertions in 72 files,
ast.dumpof every changed.pyfile identical to the base, all changed YAML and TOML files parse to the same content, shebangs still on line 1. 742 targeted tests pass on the registries, packaging, docs andci_localconsumers. All 12 eval patches still passgit apply --checkon their pinned base with the header hunks added.
Checked, no issues
- Links:
dokimi/CONTRIBUTING.md#license-of-contributionsresolves to the## License of contributionsheading; the docker command matches the workflow'sconfigandmode. - Doc snippets: the header lines in
adding-doc-source.mdandcontributing-metric.mdsit before the docstring, so it stays the module docstring. - Workflow: the job sets
persist-credentials: falseand uses the same checkout SHA as the other workflows; the action is pinned to the v0.9.0 commit.
| # with a line break or "(C)" never matches). The tail rejects a longer id | ||
| # (Apache-2.0-only, Apache-2.0+) and an "Apache-2.0 OR ..." expression. | ||
| pattern: |- | ||
| Copyright \d{4}(-\d{4})? Intel Corporation SPDX-License-Identifier: Apache-2\.0(\s*$|\s+[^O ]|\s+O[^R]) |
There was a problem hiding this comment.
Nit: the tail only denies OR, so it is narrower than the comment above it says, and it rejects one valid case.
Simulated with Python re on the whitespace-normalised header (I did not run license-eye):
- Accepted:
Apache-2.0 or MIT,Apache-2.0 AND MIT,Apache-2.0 WITH LLVM-exception. - Rejected:
Apache-2.0 OR MIT,Apache-2.0-only,Apache-2.0+. - Rejected although the header is correct: a file whose first token after the header starts with
OR, such asORDER = 1orOR_X = 1(Overview = 1passes). No tracked file does this today.
Suggestion: let OR followed by a non-space character through, in RE2 syntax (no lookahead): Apache-2\.0(\s*$|\s+[^O ]|\s+O[^R]|\s+OR[^\s]). Or keep the form and word the comment as "rejects OR expressions, -only and +".
There was a problem hiding this comment.
Took your tail in 9c59446: \s+OR[^\s]. Checked with license-eye 0.9.0, not simulated. ORDER = 1 and OR_X = 1 after the header now pass. Apache-2.0 OR MIT, -only and + are still rejected. The comment now says it rejects OR expressions, -only and +, and that AND/WITH expressions still pass.
| comment: never | ||
|
|
||
| # The header may follow a shebang line or YAML front matter. | ||
| license-location-threshold: 120 |
There was a problem hiding this comment.
Nit: a threshold of 120 characters leaves room for only a short front matter block, so "the header may follow YAML front matter" (here and in dokimi/CONTRIBUTING.md) holds only in the short case.
The HTML-comment header ends about 90 characters into a Markdown file. With a 110-character front matter block (name: plus a one-line description:), the SPDX id starts past character 120; in the same simulation it is not seen. No non-exempt tracked file has front matter today, so nothing fails now. A new doc page with a normal front matter would.
Suggestion: raise the threshold (the largest offset where the id ends in the tree is 99), or say in both places that the header goes first.
There was a problem hiding this comment.
Checked with license-eye 0.9.0: in check mode the threshold is not applied. A Markdown header after 260 or 700+ characters of front matter passes at 120, and so does a .py header at line 201. The threshold only affects where fix inserts a header. 9c59446 says this next to license-location-threshold, and the top comment says the check does not enforce the position. CONTRIBUTING.md keeps "after any front matter", which is true for any length.
Port oneDAL's license header check: a license-header job in ci.yml runs apache/skywalking-eyes in check mode with .github/.licenserc.yaml. Every tracked file carries the two-line SPDX header the tree already uses; the 52 files that lacked it get it. Exempt: JSON (no comment syntax), license texts, GitHub templates, uv-generated lock files, and text agents read verbatim (CLAUDE.md, agent profiles, skills). Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
.agent-eval (intel#57) landed after this branch's base: its check.py scripts, hidden tests, Dockerfile and README get the header. Oracle and setup patches and oracle_answer.md are exempt, like the other text applied or read verbatim. CLAUDE.md became AGENTS.md (intel#56); the exemption follows. Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
The two-line pattern with "(C)" never matched with skywalking-eyes 0.9.0: a header with any year but 2026 failed the check, whatever the comment in the config promised, so the first file added in 2027 would turn CI red. A one-line pattern without "(C)" matches 2027 and 2024-2026 and still rejects another holder, another SPDX id and a missing header. Signed-off-by: Alexander Rybkin <alexander.rybkin@intel.com>
… check - Headers on the files main gained since the branch was cut: CHANGELOG.md, changelog.d/README.md, docs/releasing.md, context-window.md and tool-preference.md. - Exempt scriv fragments and the fragment template: `scriv collect` moves their text into CHANGELOG.md, which carries the header. - Match on the pattern alone and end it after the id. With 0.9.0 `content` is a substring match and `spdx-id` accepts "Apache-2.0 OR MIT"; the pattern now rejects that, Apache-2.0-only and Apache-2.0+. - State the rule and the local docker command where contributors and agents look: dokimi/CONTRIBUTING.md, root CONTRIBUTING.md, AGENTS.md, the PR template, and the new-file skeletons in the extension guides. Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
ff5dd7a to
9c59446
Compare
… may sit The tail rejected a correct header followed by a word starting with "OR" (ORDER = 1); it now rejects only an "OR" expression. license-eye 0.9.0 check finds the header anywhere in the file, so license-location-threshold only affects fix, and front matter of any length can precede the header. Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
|
@vchamarthi - review is blocked - please take a look |
Summary
Ports oneDAL's license header check (apache/skywalking-eyes, check mode) to dokimi. Header is the two-line SPDX form the tree already uses in 357 files, not oneDAL's full Apache paragraph.
Exempt (in
.licenserc.yaml): JSON (no comments),LICENSE/NOTICE/.gitignore/CODEOWNERS, issue/PR templates, uv-generatedrequirements*.lock.txt, and text agents read verbatim (AGENTS.md,data/agent_profiles/**,data/skills/**) and eval oracles (.agent-eval/tasks/**/*.patch,oracle_answer.md) — a header there would change the measured input. Eval tasks pin abasecommit, so headers on main do not touch what the patches apply to.Evidence
license-eye header check→valid: 359, invalid: 53(52 files +.gitworktree file)After:
valid: 425, invalid: 0, ignored: 72(rebased on main incl. docs: move the agent guidance to AGENTS.md, remove CLAUDE.md #56, eval: dokimi's own repository evals in .agent-eval #57)make ci-local:0 failed, 19 passed, 4 skipped(4252 tests, mypy, zizmor, pre-commit, mkdocs--strict, guidance self-scan).ci-localskips the new job (it isuses:only); the config header has the docker command for a local run.Merge Danger
Door: two-way
Header-only content changes plus one new CI job; revert is a single commit.
Blast Radius: CI
New PRs adding a file without the header fail the new job. Open PRs touching the 63 files may hit trivial conflicts on the first lines.
🤖 Generated with Claude Code