Skip to content

Update - #267

Merged
MrIbrahem merged 17 commits into
mainfrom
update
Jul 22, 2026
Merged

Update#267
MrIbrahem merged 17 commits into
mainfrom
update

Conversation

@MrIbrahem

@MrIbrahem MrIbrahem commented Jul 22, 2026

Copy link
Copy Markdown
Collaborator

Summary by CodeRabbit

  • Bug Fixes

    • Fixed journal information being skipped when matching records are found.
    • Improved handling of empty content and empty sections.
    • Corrected category and page-result processing.
    • Improved data merging and sorting for assessment and page-view records.
    • Refined recent-page retrieval and page-assessment processing.
  • Improvements

    • Expanded support for flexible input values across API and database tools.
    • Improved logging clarity during timestamp processing.
  • Developer Experience

    • Strengthened code quality, formatting, linting, and type-checking configurations.

@gemini-code-assist

Copy link
Copy Markdown
Contributor

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

@coderabbitai

coderabbitai Bot commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Important

Review skipped

Too many files!

This PR contains 107 files, which is 7 over the limit of 100.

To get a review, narrow the scope:
• coderabbit review --committed # exclude uncommitted changes
• coderabbit review --dir # limit to a subdirectory
• coderabbit review --base # compare against a closer base

Upgrade to a paid plan to raise the limit.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: d52de4dd-bf45-4bf0-b9ff-a28f3b69a14a

📥 Commits

Reviewing files that changed from the base of the PR and between 50c9508 and 1d20ca5.

📒 Files selected for processing (109)
  • src/copy_to_en/revid.py
  • src/db/analytics/replica_db.py
  • src/db/mdapi_sql/services/sql_for_mdwiki.py
  • src/db/mdapi_sql/services/sql_qids.py
  • src/db/mdapi_sql/services/sql_qids_others.py
  • src/db/mdapi_sql/sql_td_bot.py
  • src/md_core/fix_cs1/bot.py
  • src/md_core/fix_cs1/fix_cs_params/bot.py
  • src/md_core/mdpy/fixref/make_title_bot.py
  • src/md_core/mdpy/others/export.py
  • src/md_core/mdpy/others/our.py
  • src/md_core/p11143_bot/bot.py
  • src/md_core/p11143_bot/wd_helps.py
  • src/md_core/unlinked_wb/bot.py
  • src/md_core/unlinked_wb/hlps.py
  • src/md_core/updates/Medicine_articles.py
  • src/md_core/updates/c2023.py
  • src/md_core/updates/listo.py
  • src/md_core_helps/apis/cat_cach.py
  • src/md_core_helps/apis/mdwiki_api_call.py
  • src/md_core_helps/apis/mw_views.py
  • src/md_core_helps/apis/wd_bots/wd_post_new.py
  • src/md_core_helps/apis/wd_bots/wd_rest_new.py
  • src/md_core_helps/apis/wd_bots/wikidataapi_post.py
  • src/md_core_helps/apis/wiki_api.py
  • src/md_core_helps/apis/wikidataapi.py
  • src/md_core_helps/one_time/prior/text_bot.py
  • src/md_core_helps/one_time/priorviews/bots/gt_blame.py
  • src/md_core_helps/one_time/wikiblame/bot.py
  • src/mdwiki_api/mdwiki_page.py
  • src/newapi/__init__.py
  • src/newapi/api_client/client.py
  • src/newapi/client_wiki/api_utils/bot_edit/bot_edit_by_templates.py
  • src/newapi/client_wiki/api_utils/wd_sparql.py
  • src/newapi/client_wiki/categories/category_db.py
  • src/td_core/after_translate/bots/add_to_mdwiki.py
  • src/td_core/after_translate/bots/add_to_pages_users_db.py
  • src/td_core/after_translate/bots/get_pages.py
  • src/td_core/after_translate/start_work.py
  • src/td_core/copy_data/by_qid/sitelinks.py
  • src/td_core/copy_data/by_title/all_articles.py
  • src/td_core/copy_data/copy_assessments.py
  • src/td_core/copy_data/copy_enwiki_pageviews.py
  • src/td_core/copy_data/copy_refs_2.py
  • src/td_core/copy_data/copy_word_2.py
  • src/td_core/copy_data/copy_word_table.py
  • src/td_core/db_work/check_titles.py
  • src/td_core/db_work/check_titles_helps.py
  • src/td_core/db_work/days_7.py
  • src/td_core/db_work/get_red.py
  • src/td_core/fix_user_pages/bot.py
  • src/td_core/fix_user_pages/del.py
  • src/td_core/fix_user_pages/fix_it_db.py
  • src/td_core/fix_user_pages/fix_it_db_new.py
  • src/td_core/fix_user_pages/user_bot.py
  • src/td_core/mdcount/bots/links.py
  • src/td_core/mdcount/ref_words_bot.py
  • src/td_core/mdpages/create_qids.py
  • src/td_core/mdpages/find_qids.py
  • src/td_core/mdpyget/enwiki_views.py
  • src/td_core/mdpyget/getas.py
  • src/td_core/mdpyget/pages_list.py
  • src/td_core/td_dirs.py
  • src/td_core/td_other_qids/make_list.py
  • src/wprefs/api.py
  • src/wprefs/bot.py
  • src/wprefs/bot1.py
  • src/wprefs/bots/Duplicatenew2.py
  • src/wprefs/bots/duplicate.py
  • src/wprefs/files.py
  • src/wprefs/infobox.py
  • src/wprefs/wpref_text.py
  • src1/copy_text/bot.py
  • src1/copy_text/files_list.py
  • src1/copy_text/html_bot.py
  • src1/copy_text/scan_files.py
  • src1/copy_text/text_bot.py
  • src1/copy_to_en/bot.py
  • src1/copy_to_en/bots/fix_refs_names.py
  • src1/copy_to_en/bots/ref.py
  • src1/copy_to_en/bots/ref2.py
  • src1/copy_to_en/mdwikicx.py
  • src1/copy_to_en/medwiki.py
  • src1/find_replace_bot/bot.py
  • src1/find_replace_bot/one_job.py
  • src1/fix_use/mtab.py
  • src1/md_core/add_rtt/bot.py
  • src1/md_core/add_rtt/pup.py
  • src1/md_core/add_rtt/r_column_bots/add_r_column.py
  • src1/md_core/add_rtt/r_column_bots/pup_table.py
  • src1/md_core/add_rtt/remove.py
  • src1/md_core/commons_svg/bot.py
  • src1/md_core/mdpy/fix_duplicate.py
  • src1/md_core/mdpy/fixred.py
  • src1/md_core/mdpy/orred.py
  • src1/md_core/mdpy/replace1.py
  • src1/named_param/named_param.py
  • src1/newupdater/mdapi.py
  • src1/newupdater/med.py
  • src1/newupdater/new_updater/MedWorkNew.py
  • src1/newupdater/new_updater/chembox.py
  • tests/src/md_core/add_rtt/add_r_col.py
  • tests/src/md_core/add_rtt/add_r_col_to_file.py
  • tests/src/md_core/add_rtt/test.py
  • tests/src/md_core/add_rtt/tests/add_r_col_to_file.py
  • toolforge/c9/logger_config.py
  • toolforge/c9/pwb.py
  • toolforge/c9/user_conf.py
  • toolforge/user-config.py

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • Review on demand using usage pricing

Walkthrough

This PR revises Python tooling configuration, modernizes type annotations, adjusts API and category processing, fixes journal matching, changes data-copy flows, and cleans up several one-time utility scripts.

Changes

Repository modernization

Layer / File(s) Summary
Tooling and type-checking configuration
pyproject.toml
Black, isort, Ruff, and Pyright settings are revised, with Pyright narrowed to src and switched to strict checking.
Typing and public API contracts
src/db/..., src/md_core/..., src/newapi/...
Union annotations, collection imports, public parameter types, return types, and internal attribute annotations are modernized.
API response and category processing
src/newapi/client_wiki/bot_api.py, src/newapi/client_wiki/categories/category_db.py
Recent-page and assessment calls are adjusted, and category parsing now handles integer depth, list-shaped pages, merged results, and timestamp sorting.
Runtime data handling and processing
src/md_core/..., src/td_core/copy_data/...
Journal matching, section extraction, empty-content checks, logging, link retrieval, and intermediate data merging are updated.
Utility script cleanup and naming
src/md_core_helps/one_time/..., src1/...
Test flags are removed, counters and helpers are renamed, word-count calls are updated, and import/docstring formatting changes are applied.

Estimated code review effort: 4 (Complex) | ~45 minutes

Possibly related PRs

Poem

A rabbit hops through types so neat,
And finds old bugs beneath its feet.
APIs reshape, logs softly fade,
Fresh data paths through burrows wade.
Test flags vanish—bright and clean,
“Strict checks!” thumps the bunny queen.

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (1 warning, 1 inconclusive)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 14.71% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
Title check ❓ Inconclusive The title is too generic and does not describe the main change in the pull request. Use a concise, specific title that reflects the primary change, such as the tooling/type-hint refactor or the most important functional fix.
✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch update

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 8

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (2)
src/newapi/client_wiki/bot_api.py (1)

479-482: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

Bound the three-hour recentchanges window correctly.
src/newapi/client_wiki/bot_api.py:479-482rcstart=now-3h with the default rcdir=older and no rcend walks backward from that cutoff, so this can return changes older than three hours. Set rcdir="newer" with an explicit rcend, or swap the bounds.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/newapi/client_wiki/bot_api.py` around lines 479 - 482, Correct the
three-hour recent-changes window in the three_houers branch by pairing rcstart
with an explicit current-time rcend and using rcdir="newer", or equivalently
swapping the bounds. Ensure the request only returns changes from the last three
hours and does not include older results.

Source: MCP tools

src/newapi/client_wiki/categories/category_db.py (1)

74-99: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

self.limit isn't coerced to int like self.depth is, despite the tightened int annotation.

Line 42 declares self.limit: int, but Line 88 keeps self.limit = kwargs.get("limit") or 0 without any int() cast — unlike self.depth, which now gets a try/except int(...) guard just above (Lines 74-78). If limit is passed as a non-numeric string (previously valid under the old int | str type), self.limit will hold a str at runtime, and the later self.limit > 0 comparisons (used in get_cat_new/subcatquery_) will raise TypeError.

🐛 Proposed fix: mirror the depth-parsing pattern
-        self.limit = kwargs.get("limit") or 0
+        try:
+            self.limit = int(kwargs.get("limit") or 0)
+        except ValueError:
+            logger.error(f"self.limit != int: {kwargs.get('limit')}")
+            self.limit = 0
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/newapi/client_wiki/categories/category_db.py` around lines 74 - 99,
Coerce the limit option to an integer in the initializer, mirroring the existing
depth parsing and fallback behavior. Update the self.limit assignment to handle
invalid values safely by logging the conversion error and defaulting to 0, so
self.limit remains an int for later comparisons in get_cat_new and subcatquery_.
🧹 Nitpick comments (1)
src/newapi/client_wiki/api_utils/txtlib.py (1)

14-14: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Redundant str | Any annotation — simplifies to Any for type checkers.

Since Any is compatible with any type, str | Any conveys the same information as plain Any to static analyzers; the str part is misleading given the function still runtime-validates via isinstance(text, str).

-def extract_templates_and_params(text: str | Any) -> list[dict[str, Any]]:
+def extract_templates_and_params(text: Any) -> list[dict[str, Any]]:
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/newapi/client_wiki/api_utils/txtlib.py` at line 14, Update the
extract_templates_and_params parameter annotation from str | Any to Any, leaving
the existing runtime isinstance(text, str) validation and function behavior
unchanged.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@pyproject.toml`:
- Line 251: Update the Pyright configuration around typeCheckingMode to use a
less restrictive base mode instead of "strict", then explicitly enable the
desired strict type-checking diagnostics while retaining the intended
lower-severity or disabled report overrides. Ensure the resulting configuration
preserves the targeted checks without strict mode overriding those settings.

In `@src/md_core_helps/apis/mdwiki_api_call.py`:
- Around line 226-227: Update Get_template_pages and its
api_new.Get_template_pages delegate to thread the limit argument through and use
it for the gtilimit request parameter, preserving caller-selected caps;
alternatively remove limit from the wrapper API if configurability is not
intended.

In `@src/md_core_helps/one_time/prior/add_old.py`:
- Around line 125-130: Deep-copy data before the iteration and mutation flow
that produces data_new, ensuring nested entries are independent from the
original data. Keep the existing data_new == data comparison and write behavior,
so mutated copies are serialized when changes occur.

In `@src/newapi/client_wiki/api_utils/bot_edit/__init__.py`:
- Line 18: Update the delay handling in the bot edit request body so truthy
string values are coerced to integers before the last-edit-time safety check.
Ensure invalid strings are rejected or handled consistently, and preserve the
existing guard behavior for valid numeric delays in the code surrounding the
isinstance(delay, int) check.

In `@src/newapi/client_wiki/categories/category_db.py`:
- Around line 136-139: Guard the namespace conversion in params_work so the
default self.ns value "all" does not reach int(). Preserve the existing "0"/"10"
page handling and apply the "subcat" branch only when self.ns is numeric and
equals 14, while allowing get_cat_new to proceed normally for "all".
- Around line 202-233: Update the request configuration used by get_cat_new so
the API requests formatversion=2, matching pages_table_work’s list-based
iteration over query.pages. Preserve the existing pages_table_work processing
and ensure the format-version parameter is applied to this category request.

In `@src/td_core/copy_data/copy_enwiki_pageviews.py`:
- Around line 40-43: Replace truthiness-based fallback checks with
key-membership checks in the SQL merge paths: update the condition in
src/td_core/copy_data/copy_enwiki_pageviews.py lines 40-43 and the corresponding
condition in src/td_core/copy_data/copy_assessments.py lines 41-42 to use title
not in new_data, preserving existing falsy values.

In `@src/td_core/mdcount/bots/links.py`:
- Around line 78-79: Update the `vav` assignment after `Get_page_links` to
convert the links mapping keys into a mutable list using the requested
`list(result.get("links", {}))` form, so subsequent `vav[:]` and `vav.remove()`
operations work correctly.

---

Outside diff comments:
In `@src/newapi/client_wiki/bot_api.py`:
- Around line 479-482: Correct the three-hour recent-changes window in the
three_houers branch by pairing rcstart with an explicit current-time rcend and
using rcdir="newer", or equivalently swapping the bounds. Ensure the request
only returns changes from the last three hours and does not include older
results.

In `@src/newapi/client_wiki/categories/category_db.py`:
- Around line 74-99: Coerce the limit option to an integer in the initializer,
mirroring the existing depth parsing and fallback behavior. Update the
self.limit assignment to handle invalid values safely by logging the conversion
error and defaulting to 0, so self.limit remains an int for later comparisons in
get_cat_new and subcatquery_.

---

Nitpick comments:
In `@src/newapi/client_wiki/api_utils/txtlib.py`:
- Line 14: Update the extract_templates_and_params parameter annotation from str
| Any to Any, leaving the existing runtime isinstance(text, str) validation and
function behavior unchanged.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 83d59808-6c27-4ba0-a15c-9669a972c1a2

📥 Commits

Reviewing files that changed from the base of the PR and between 341745d and 50c9508.

📒 Files selected for processing (29)
  • pyproject.toml
  • src/db/analytics/replica_db.py
  • src/md_core/fix_cs1/bots/pmid.py
  • src/md_core/p11143_bot/wd_helps.py
  • src/md_core/updates/io.py
  • src/md_core_helps/apis/mdwiki_api_call.py
  • src/md_core_helps/one_time/WHOem/find_views_by_lang.py
  • src/md_core_helps/one_time/prior/add_old.py
  • src/md_core_helps/one_time/prior/read5.py
  • src/md_core_helps/one_time/priorviews/bots/count_words.py
  • src/md_core_helps/one_time/priorviews/bots/gt_blame.py
  • src/md_core_helps/one_time/priorviews/bots/sections_links.py
  • src/md_core_helps/one_time/priorviews/find/find_blame.py
  • src/md_core_helps/one_time/priorviews/find/find_creator.py
  • src/md_core_helps/one_time/priorviews/find/find_translators.py
  • src/md_core_helps/one_time/priorviews/find/find_views.py
  • src/md_core_helps/one_time/priorviews/find/find_word.py
  • src/md_core_helps/one_time/wikiblame/bot.py
  • src/newapi/api_client/client.py
  • src/newapi/client_wiki/api_utils/bot_edit/__init__.py
  • src/newapi/client_wiki/api_utils/txtlib.py
  • src/newapi/client_wiki/bot_api.py
  • src/newapi/client_wiki/categories/category_db.py
  • src/newapi/client_wiki/pages/super_page.py
  • src/td_core/copy_data/copy_assessments.py
  • src/td_core/copy_data/copy_enwiki_pageviews.py
  • src/td_core/mdcount/bots/links.py
  • src1/md_core/add_rtt/pup.py
  • src1/newupdater/__init__.py
💤 Files with no reviewable changes (5)
  • src/md_core_helps/one_time/WHOem/find_views_by_lang.py
  • src1/md_core/add_rtt/pup.py
  • src/md_core_helps/one_time/priorviews/find/find_translators.py
  • src/md_core_helps/one_time/priorviews/find/find_views.py
  • src/md_core_helps/one_time/priorviews/find/find_word.py

Comment thread pyproject.toml
Comment thread src/md_core_helps/apis/mdwiki_api_call.py
Comment thread src/md_core_helps/one_time/prior/add_old.py
Comment thread src/newapi/client_wiki/api_utils/bot_edit/__init__.py
Comment thread src/newapi/client_wiki/categories/category_db.py
Comment thread src/newapi/client_wiki/categories/category_db.py Outdated
Comment thread src/td_core/copy_data/copy_enwiki_pageviews.py
Comment thread src/td_core/mdcount/bots/links.py
@MrIbrahem
MrIbrahem merged commit 3436dde into main Jul 22, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant