Skip to content

perf(worker): eliminate N+1 query in commit parent lookup - #2227

Open
sentry[bot] wants to merge 1 commit into
mainfrom
seer/perf/worker-commit-parent-n1
Open

perf(worker): eliminate N+1 query in commit parent lookup#2227
sentry[bot] wants to merge 1 commit into
mainfrom
seer/perf/worker-commit-parent-n1

Conversation

@sentry

@sentry sentry Bot commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

This PR refactors the fetch_appropriate_parent_for_commit function in apps/worker/services/repository.py to address an N+1 query issue.

Problem:
The original implementation performed up to two database queries for each BFS level of a commit's ancestor tree. For repositories with deep commit histories, this resulted in a large number of redundant database calls (e.g., ~60 queries per task invocation observed in WORKER-SM9), leading to significant performance overhead.

Solution:

  1. The function now first traverses the entire in-memory ancestor tree to collect all relevant commit IDs.
  2. A single batched database query is then executed to fetch the commitid, branch, and message status for all collected ancestor IDs.
  3. An in-memory lookup map is created from the query results.
  4. The logic for selecting the closest parent (with and without a message, including branch tie-breaking) is then replayed entirely in-memory using this map, preserving the original behavior.

This change drastically reduces the number of database queries from N+1 to a single query, improving the performance of the CommitUpdate task.

Legal Boilerplate

Look, I get it. The entity doing business as "Codecov" is owned by Harness, Inc. In 2026 Harness acquired Codecov and as a result Harness is going to need some rights from me in order to utilize my contributions in this PR. So here's the deal: I retain all rights, title and interest in and to my contributions, and by keeping this boilerplate intact I confirm that Harness can use, modify, copy, and redistribute my contributions, under Harness's choice of terms.

Fixes WORKER-SM9

This PR was automatically generated by Sentry. You can adjust this setting at any time.

@thomasrockhu-codecov

thomasrockhu-codecov commented Sep 9, 2026

Copy link
Copy Markdown
Contributor
✨ Harness AI Code Review View in Harness →

Risk: LOW

This PR refactors fetch_appropriate_parent_for_commit in apps/worker/services/repository.py to eliminate an N+1 query, replacing per-BFS-level database queries with a single batched query and an in-memory replay of the closest-parent selection. The refactor is behaviorally equivalent to the original: BFS levels are collected in the same order, the single query fetches the same (commitid, branch, message) set filtered by repo and non-deleted, and the new _possibly_filter_out_branch_from_list mirrors _possibly_filter_out_branch (single-candidate short-circuit, then first branch match, else None). Message-existence semantics and the closest_parent_without_message fallback are preserved, and existing BFS-path tests continue to cover the changed code.

No issues found above the confidence threshold.

@codecov-notifications

codecov-notifications Bot commented Sep 9, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 92.85714% with 2 lines in your changes missing coverage. Please review.
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
apps/worker/services/repository.py 92.85% 2 Missing ⚠️

📢 Thoughts on this report? Let us know!

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@codecov

codecov Bot commented Sep 9, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 92.85714% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 91.63%. Comparing base (17e0f08) to head (b658a30).
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
apps/worker/services/repository.py 92.85% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2227      +/-   ##
==========================================
- Coverage   91.63%   91.63%   -0.01%     
==========================================
  Files        1337     1337              
  Lines       53262    53285      +23     
  Branches     1647     1647              
==========================================
+ Hits        48805    48826      +21     
- Misses       4136     4138       +2     
  Partials      321      321              
Flag Coverage Δ
workerintegration 58.51% <75.00%> (+0.01%) ⬆️
workerunit 90.55% <92.85%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

@sentry

sentry Bot commented Sep 9, 2026

Copy link
Copy Markdown
Contributor Author

⚠️ Seer needs additional GitHub permissions

Seer wants to keep iterating on this pull request to get CI passing, but the Sentry GitHub App installation is missing permissions it needs to read the failing checks and push a fix.

Review and accept the updated permissions to let Seer continue: https://github.com/organizations/codecov/settings/installations/86101127/permissions/update

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant