Skip to content

perf(background_tasks): project task reads, separate exact count, index the default order - #833

Merged
antosubash merged 8 commits into
mainfrom
fix/s11-perf-background-tasks
Nov 5, 2026
Merged

antosubash merged 8 commits into
mainfrom
fix/s11-perf-background-tasks

Conversation

@antosubash

Copy link
Copy Markdown
Owner

Performance fix for the background-task status poll and the admin task list (local performance review, 2026-11-05).

  • Status polls and the task list load only the columns their DTOs return (dto_columns: load_only + raiseload). Status reads keep owner_user_id, tenant_id, resource_type and resource_id for authorization, and the list keeps the retry arguments. Large result/traceback JSON is no longer fetched or decoded.
  • The list runs a separate exact count(*) plus a projected page query instead of count() OVER () over wide rows. Stale-page clamping is unchanged, now through simple_module_db.pagination.total_pages.
  • New expression index ix_background_tasks_task_execution_queued_order on (queued_at IS NULL), queued_at DESC (migration c5d8e2f1a903). newest_first() defines that ordering once for the index, the list and the status lookup. The status lookup previously sorted NULL queued_at first on Postgres and last on SQLite.
  • The index page's "any executions?" check is a LIMIT 1 probe (has_any()) instead of a counted list call.

Measured with 100k executions on SQLite (method in docs/perf/2026-11-05-background-task-reads.md):

Measurement Before After
First list page, including exact total 1,418 ms 3.5 ms
Poll with a 1 MB result 3.10 ms 0.48 ms
Peak traced Python memory per poll 2.0 MB 23 KB

Pipeline: /optimize (1 commit), two review passes with triage kept locally.
Local CI: make lint ✓ · make test-py (8 workers) 7861 passed ✓ · make test-js ✓ · make migrations-roundtrip-pg ✓ (fresh database).

Note: c5d8e2f1a903 and #832's e805a1b2c3d4 both revise e731a1b2c3d4; whichever merges second re-points its down_revision.

https://claude.ai/code/session_01M9neheZZEe3sVpDi2S3zT4

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 5, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
🔒 Security Review ✅ Completed 2026-11-05T16:18:16.856764Z c7639e5 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Nov 5, 2026 •

Copy link
Copy Markdown

Deploying simple-module-python with  Cloudflare Pages  Cloudflare Pages

Latest commit: 5500185
Status: ✅  Deploy successful!
Preview URL: https://ba8fda04.simple-module-python.pages.dev
Branch Preview URL: https://fix-s11-perf-background-task.simple-module-python.pages.dev

View logs

# Conflicts:
#	CHANGELOG.md
# Conflicts:
#	docs/database/migrations.md
#	host/tests/test_migration_graph.py
@antosubash
antosubash merged commit 8fa5bd8 into main Nov 5, 2026
17 checks passed
@antosubash
antosubash deleted the fix/s11-perf-background-tasks branch November 5, 2026 22:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant