Skip to content

ci: check license headers with skywalking-eyes - #62

Merged
napetrov merged 5 commits into
intel:mainfrom
napetrov:license-header-check
Oct 16, 2026
Merged

napetrov merged 5 commits into
intel:mainfrom
napetrov:license-header-check

Conversation

@napetrov

@napetrov napetrov commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Ports oneDAL's license header check (apache/skywalking-eyes, check mode) to dokimi. Header is the two-line SPDX form the tree already uses in 357 files, not oneDAL's full Apache paragraph.

 .github/
+├── .licenserc.yaml          # pattern: Copyright (C) <year> Intel Corporation + SPDX Apache-2.0
 └── workflows/ci.yml
+    └── license-header       # skywalking-eyes/header@v0.9.0 (SHA-pinned, same as oneDAL)
 dokimi/
+    52 files get the header  # 8 tests .py, YAML registries/data, pyproject, requirements*.txt, 34 .md (HTML comment)
 .agent-eval/
+    11 files get the header  # check.py scripts, hidden tests, Dockerfile, README
     CONTRIBUTING.md          # + note: new files need the header

Exempt (in .licenserc.yaml): JSON (no comments), LICENSE/NOTICE/.gitignore/CODEOWNERS, issue/PR templates, uv-generated requirements*.lock.txt, and text agents read verbatim (AGENTS.md, data/agent_profiles/**, data/skills/**) and eval oracles (.agent-eval/tasks/**/*.patch, oracle_answer.md) — a header there would change the measured input. Eval tasks pin a base commit, so headers on main do not touch what the patches apply to.

Evidence

Merge Danger

Door: two-way

Header-only content changes plus one new CI job; revert is a single commit.

Blast Radius: CI

New PRs adding a file without the header fail the new job. Open PRs touching the 63 files may hit trivial conflicts on the first lines.

🤖 Generated with Claude Code

@vchamarthi vchamarthi left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this! The exemptions are well reasoned (agent-read text, uv-generated locks, templates), the action is pinned to the v0.9.0 commit, and the CONTRIBUTING note makes the rule easy to follow. One thing to sort out before merge: main has moved on since this branch was cut, see inline.

Comment thread .github/.licenserc.yaml
# persona prompt or skill being measured.
- 'CLAUDE.md'
- 'dokimi/data/agent_profiles/**'
- 'dokimi/data/skills/**'

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this needs a rebase on main before merge: #57 added .agent-eval/, and 25 of its files have no SPDX header (the Dockerfile, README.md, every check.py, the hidden tests, the *.patch files and the oracle_answer.md files). Since pull_request runs check out the merge ref, this job should fail here and on main once it lands. The task fixtures look like they belong with the other verbatim exemptions, since a header in oracle.patch/setup.patch, oracle_answer.md or a hidden test changes what the agent sees or what gets applied. Could we add .agent-eval/tasks/** to paths-ignore and put headers on .agent-eval/Dockerfile and .agent-eval/README.md?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rebased on main (96e355a). On .agent-eval/ I kept the split from e80f53c rather than ignoring all of tasks/**: the files that reach the agent or the workspace verbatim are exempt (*.patch, which covers setup.patch/oracle.patch/review.patch, and oracle_answer.md, copied to answer.txt). check.py is run by the harness and the hidden/ tests are copied in only after the agent finishes (run.py _hidden_tests, after process_metrics), so a header there changes nothing the agent sees or what gets applied. Dockerfile and README.md have headers. Happy to widen it to tasks/** if you would rather not reason per file.

@codecov-commenter

codecov-commenter commented Oct 15, 2026 •

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 89.10%. Comparing base (f28810d) to head (9c59446).
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@           Coverage Diff           @@
##             main      #62   +/-   ##
=======================================
  Coverage   89.10%   89.10%           
=======================================
  Files         204      204           
  Lines       23557    23557           
  Branches     3779     3779           
=======================================
  Hits        20990    20990           
  Misses       1862     1862           
  Partials      705      705           
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@napetrov
napetrov force-pushed the license-header-check branch from 9f8b87e to 595043b Compare October 15, 2026 22:03
@xaleryb

xaleryb commented Oct 16, 2026

Copy link
Copy Markdown
Contributor

Checked at 595043b by merging main into it and running skywalking-eyes 0.9.0 (Docker image), plus the suite, ruff, mypy, zizmor and actionlint. The 65 files are pure insertions (334 added lines, 0 deleted; the old file equals the new one minus the inserted lines, shebangs stay on line 1, the AST and __doc__ of the 17 .py files are unchanged, no CRLF); the pins match their tags; the check fails on a missing header, another holder, another SPDX id and a new file without a header.

  1. After a rebase the job goes red: dokimi/docs/tool-preference.md, added by feat(tools): tools preference - what moves a model's choice between two equal tools #53, has no header (invalid: 1). Open PRs that will hit the same on their new files: feat(context): context trace, the prompt size of every model call in a Claude Code trace #68 (context-window.md, tool-preference.md), docs: preparing a repository for coding agents #65 (preparing-a-repository.md), refactor(novelty): the bundle record owns its validity #44 (three new test_novelty_*.py), build: versioned releases from 0.1.0 — changelog fragments, working wheel, release workflow #39 (CHANGELOG.md, changelog.d/*, and fragment_template.md.j2, a Jinja template where a header may render into the output).
  2. The pattern never matched: the two-line form with (C) matches nothing, so only a 2026 header passes and the first file added in 2027 fails. Pushed f239ba1: the one-line Copyright \d{4}(-\d{4})? Intel Corporation SPDX-License-Identifier: Apache-2\.0 accepts any year and still rejects other holders and licences (checked on 2019, 2024-2026, 2027, Acme, MIT and a missing header).

The .agent-eval/tasks/** request from the CHANGES_REQUESTED review is handled differently (only *.patch and oracle_answer.md are ignored, the graders get headers): harmless as far as I can see, but the reviewer has not confirmed it. Root CONTRIBUTING.md gets a header without the requirement note that dokimi/CONTRIBUTING.md has.

@antonwolfy antonwolfy left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review at f239ba1. The headers are right: 334 pure insertions, the YAML, TOML and requirements files still load, the docs still build with --strict, and the exemptions match what agents read verbatim. The problem is the merge. The job goes red as soon as this branch meets current main, and nothing in make ci-local or AGENTS.md would tell a contributor about the rule.

What I ran, on a Linux review host at f239ba1:

  • make ci-local: 18 passed, 1 failed, 4 skipped; 4,293 tests. The one failure is schema-compat: the branch is 4 commits behind main and lacks the two tool_preference*.v1.json schemas that main added (#53), so it reports them as removed. It is a stale-base artifact, not something this PR does; it goes away after the rebase. No Python lines changed, so the changed-line coverage gate had nothing to measure.
  • A header scan of the head tree (the config's ignore list, the header form, the first 120 characters): 498 tracked files, 0 without a header. Applying the same scan to git merge-tree origin/main HEAD (the tree CI checks out on a pull_request run; the merge has no conflicts): 507 files, 1 without a header.
  • Targeted tests for every consumer of the 65 files: the YAML registries and explore tasks, pyproject.toml, the requirements files, the docs tests, mkdocs build --strict, and the guidance self-scan (fail 0; claims go from 198 to 200 because of the new CONTRIBUTING text). The .agent-eval check.py shebangs stay on line 1 and the docstrings stay the first statement. The tasks apply their patches onto the workspace's start commit, so headers on main do not touch them.
  • I did not run the license-eye binary itself. Everything about its matching is from the pattern as written, simulated with Python re.

Checked, no issues

  • Content: git diff --numstat shows 0 deletions in all 65 files; only .licenserc.yaml, the ci.yml job and the dokimi/CONTRIBUTING.md note carry anything other than a header.
  • Workflow: zizmor --offline --min-severity medium reports no findings; the checkout SHA and # v7.0.1 comment are identical to the other 12 uses in the workflows; the job sets persist-credentials: false and inherits contents: read.
  • Exemptions: each one is justified (JSON, uv locks, templates, license texts, agent-read text, eval oracles). No exempt file carries a header and no other file is missing one.
  • Docs metrics: the repo scores its own docs with benchmarks/spec.v1.yaml. The header moves the readability and coverage scores of 19 of 21 docs by less than 0.01, far below the score_drop_warn of 0.03.

Comment thread .github/workflows/ci.yml Resolved
Comment thread dokimi/CONTRIBUTING.md Resolved
Comment thread .github/.licenserc.yaml Resolved
@napetrov
napetrov force-pushed the license-header-check branch from f239ba1 to ff5dd7a Compare October 16, 2026 17:53
@napetrov napetrov added the skip-changelog No user-visible change: bypasses the changelog fragment check label Oct 16, 2026

@antonwolfy antonwolfy left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM at ff5dd7a. The earlier problems are fixed: the branch is rebased on main, tool-preference.md has its header, the rule is in AGENTS.md and both CONTRIBUTING.md files, and the pattern's tail now rejects -only, + and OR. Two nits inline.

What I ran, on a Linux review host at ff5dd7a:

  • make ci-local: 18 passed, 1 failed, 4 skipped. The one failure, changelog.yml:fragment, is local only: ci-local does not see PR labels, and with this PR's skip-changelog label the Changelog fragment check passes on GitHub. Tests, mypy, schema-compat, zizmor, pre-commit, pip-audit, bandit, package smoke and mkdocs build --strict pass. No Python lines changed, so the changed-line coverage gate had nothing to measure.
  • A header scan of the tracked tree with the config's ignore list and the pattern, simulated with Python re (I did not run the license-eye binary): 534 tracked files, 455 not ignored, 453 carry a matching header. The other two are dokimi/LICENSE and dokimi/NOTICE. They are in the ignore list only as bare LICENSE and NOTICE, and the License header check passes on GitHub at ff5dd7a, so 0.9.0 matches a bare name at any depth.
  • Probes of the pattern's tail with re (see the nit inline).
  • Diff against the base: 394 pure insertions in 72 files, ast.dump of every changed .py file identical to the base, all changed YAML and TOML files parse to the same content, shebangs still on line 1. 742 targeted tests pass on the registries, packaging, docs and ci_local consumers. All 12 eval patches still pass git apply --check on their pinned base with the header hunks added.

Checked, no issues

  • Links: dokimi/CONTRIBUTING.md#license-of-contributions resolves to the ## License of contributions heading; the docker command matches the workflow's config and mode.
  • Doc snippets: the header lines in adding-doc-source.md and contributing-metric.md sit before the docstring, so it stays the module docstring.
  • Workflow: the job sets persist-credentials: false and uses the same checkout SHA as the other workflows; the action is pinned to the v0.9.0 commit.

Comment thread .github/.licenserc.yaml Outdated
# with a line break or "(C)" never matches). The tail rejects a longer id
# (Apache-2.0-only, Apache-2.0+) and an "Apache-2.0 OR ..." expression.
pattern: |-
Copyright \d{4}(-\d{4})? Intel Corporation SPDX-License-Identifier: Apache-2\.0(\s*$|\s+[^O ]|\s+O[^R])

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: the tail only denies OR, so it is narrower than the comment above it says, and it rejects one valid case.

Simulated with Python re on the whitespace-normalised header (I did not run license-eye):

  • Accepted: Apache-2.0 or MIT, Apache-2.0 AND MIT, Apache-2.0 WITH LLVM-exception.
  • Rejected: Apache-2.0 OR MIT, Apache-2.0-only, Apache-2.0+.
  • Rejected although the header is correct: a file whose first token after the header starts with OR, such as ORDER = 1 or OR_X = 1 (Overview = 1 passes). No tracked file does this today.

Suggestion: let OR followed by a non-space character through, in RE2 syntax (no lookahead): Apache-2\.0(\s*$|\s+[^O ]|\s+O[^R]|\s+OR[^\s]). Or keep the form and word the comment as "rejects OR expressions, -only and +".

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Took your tail in 9c59446: \s+OR[^\s]. Checked with license-eye 0.9.0, not simulated. ORDER = 1 and OR_X = 1 after the header now pass. Apache-2.0 OR MIT, -only and + are still rejected. The comment now says it rejects OR expressions, -only and +, and that AND/WITH expressions still pass.

Comment thread .github/.licenserc.yaml
comment: never

# The header may follow a shebang line or YAML front matter.
license-location-threshold: 120

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: a threshold of 120 characters leaves room for only a short front matter block, so "the header may follow YAML front matter" (here and in dokimi/CONTRIBUTING.md) holds only in the short case.

The HTML-comment header ends about 90 characters into a Markdown file. With a 110-character front matter block (name: plus a one-line description:), the SPDX id starts past character 120; in the same simulation it is not seen. No non-exempt tracked file has front matter today, so nothing fails now. A new doc page with a normal front matter would.

Suggestion: raise the threshold (the largest offset where the id ends in the tree is 99), or say in both places that the header goes first.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checked with license-eye 0.9.0: in check mode the threshold is not applied. A Markdown header after 260 or 700+ characters of front matter passes at 120, and so does a .py header at line 201. The threshold only affects where fix inserts a header. 9c59446 says this next to license-location-threshold, and the top comment says the check does not enforce the position. CONTRIBUTING.md keeps "after any front matter", which is true for any length.

napetrov and others added 4 commits October 16, 2026 11:29
Port oneDAL's license header check: a license-header job in ci.yml runs
apache/skywalking-eyes in check mode with .github/.licenserc.yaml. Every
tracked file carries the two-line SPDX header the tree already uses; the
52 files that lacked it get it. Exempt: JSON (no comment syntax), license
texts, GitHub templates, uv-generated lock files, and text agents read
verbatim (CLAUDE.md, agent profiles, skills).

Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
.agent-eval (intel#57) landed after this branch's base: its check.py scripts,
hidden tests, Dockerfile and README get the header. Oracle and setup
patches and oracle_answer.md are exempt, like the other text applied or
read verbatim. CLAUDE.md became AGENTS.md (intel#56); the exemption follows.

Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
The two-line pattern with "(C)" never matched with skywalking-eyes 0.9.0:
a header with any year but 2026 failed the check, whatever the comment in
the config promised, so the first file added in 2027 would turn CI red.
A one-line pattern without "(C)" matches 2027 and 2024-2026 and still
rejects another holder, another SPDX id and a missing header.

Signed-off-by: Alexander Rybkin <alexander.rybkin@intel.com>
… check

- Headers on the files main gained since the branch was cut: CHANGELOG.md,
  changelog.d/README.md, docs/releasing.md, context-window.md and
  tool-preference.md.
- Exempt scriv fragments and the fragment template: `scriv collect` moves
  their text into CHANGELOG.md, which carries the header.
- Match on the pattern alone and end it after the id. With 0.9.0 `content`
  is a substring match and `spdx-id` accepts "Apache-2.0 OR MIT"; the
  pattern now rejects that, Apache-2.0-only and Apache-2.0+.
- State the rule and the local docker command where contributors and
  agents look: dokimi/CONTRIBUTING.md, root CONTRIBUTING.md, AGENTS.md,
  the PR template, and the new-file skeletons in the extension guides.

Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
@napetrov
napetrov force-pushed the license-header-check branch from ff5dd7a to 9c59446 Compare October 16, 2026 18:29
… may sit

The tail rejected a correct header followed by a word starting with "OR"
(ORDER = 1); it now rejects only an "OR" expression. license-eye 0.9.0
check finds the header anywhere in the file, so license-location-threshold
only affects fix, and front matter of any length can precede the header.

Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
@napetrov

Copy link
Copy Markdown
Contributor Author

@vchamarthi - review is blocked - please take a look

@napetrov
napetrov merged commit b17766c into intel:main Oct 16, 2026
25 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

skip-changelog No user-visible change: bypasses the changelog fragment check

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants