Skip to content

test, docs: leftovers from the reviews of #43, #44, #59, #69 and #75 - #87

Merged
napetrov merged 1 commit into
intel:mainfrom
antonwolfy:fu/review-leftovers
Oct 17, 2026
Merged

napetrov merged 1 commit into
intel:mainfrom
antonwolfy:fu/review-leftovers

Conversation

@antonwolfy

Copy link
Copy Markdown
Contributor

Summary

Points from the reviews of #43, #44, #59, #69 and #75 that were neither changed nor answered before those PRs merged. Only tests and text change here, no package code.

Changes

Tests

Docs

Risk

None to behaviour. No changelog fragment: nothing under dokimi/dokimi/ changes.

Testing

On a Linux review host:

  • make ci-local: 19 passed, 0 failed, 4 skipped (the Docker catalog smoke, test-latest, docs deploy, semgrep). 5,118 tests passed, 1 skipped.
  • Each new test kills its mutant:
    • load_questions(args.questions) without validate=False in refine;
    • except KeyError instead of ValueError around entry_point_only in shorten;
    • isinstance(question_id(r), str) in the gate;
    • a dropped isinstance(arm, dict) in from_artifact;
    • a dropped isinstance(s, dict) in unreproducible_arms.

Checklist

  • Tests added/updated and pytest passes
  • ruff, mypy, and schema checks pass
  • Docs updated (or N/A) — see the docs-drift checklist in CONTRIBUTING.md
  • No ad-hoc run artifacts committed
  • New files carry the SPDX license header (or are exempt in .github/.licenserc.yaml)

intel#69 and intel#75

Points from those reviews that were neither changed nor answered, and
that change no package code.

Tests:
- intel#43: questions refine reads its input without validating it; a set
  whose duplicate record has `"topics": "perf"` is repaired and written.
  Reading it strictly instead passed the suite.
- intel#44: shorten turns a bundle block it cannot slice into an exit that
  names the atoms file, not a traceback; an entry-point record without
  a stored outside_share reports the share its counts give (83%).
- intel#69: the cross-judge gate refuses one damaged question among eight
  with int ids (a gate that read str ids only passed); from_artifact
  skips a non-dict arm next to a served one; unreproducible_arms skips
  records, arms, evaluations and scores of the wrong type.

Docs:
- intel#59: README says to add --all-atoms to the tier command and to run
  --estimate-only first, and names the credential variables repo reach
  passes on.
- intel#69, intel#75: re-flow the long lines in evaluating-treatments.md, the
  runner comment in ci.yml and the coverage bullet in CONTRIBUTING.md.

Signed-off-by: Anton Volkov <antonwolfy@gmail.com>
@codecov-commenter

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 90.14%. Comparing base (9753cc7) to head (3e276e4).
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@            Coverage Diff             @@
##             main      #87      +/-   ##
==========================================
+ Coverage   90.06%   90.14%   +0.07%     
==========================================
  Files         205      205              
  Lines       24193    24193              
  Branches     3892     3892              
==========================================
+ Hits        21790    21808      +18     
+ Misses       1724     1713      -11     
+ Partials      679      672       -7     

see 4 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@napetrov
napetrov merged commit 7ad4bdf into intel:main Oct 17, 2026
20 checks passed
napetrov added a commit to napetrov/dokimi that referenced this pull request Oct 18, 2026
Resolve conflicts with intel#81 (empty-question rule): the judge paths keep the
record objects and call require_question_texts / require_question_text on
the parsed records before the first judge call; the duplicate RecordError
question check in Judge.evaluate_answers is replaced by intel#81's
EmptyQuestionError (exit 2). checklist_jobs, pairwise_jobs and
scored_answers read records. A record without question_id stays a
RecordError (intel#80) in the degradation pairs; unreproducible_arms keeps
intel#87's skip of malformed records.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RW8mAicb3ibXdzHMV83wDn
Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
@antonwolfy
antonwolfy deleted the fu/review-leftovers branch October 19, 2026 11:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants