Skip to content

Fix #1395: Scheduler initialized but not processing tasks (uvicorn vs direct call differenc - #1982

Merged
bittergreen merged 1 commit into
dev-v2.0.22from
bugfix/autodev-1395
Jul 2, 2026
Merged

bittergreen merged 1 commit into
dev-v2.0.22from
bugfix/autodev-1395

Conversation

@Memtensor-AI

Copy link
Copy Markdown
Collaborator

Description

Fixes issue #1395: under Docker + uvicorn with the local in-memory message queue (DEFAULT_USE_REDIS_QUEUE=false), GET /product/scheduler/allstatus always returned {waiting:0, in_progress:0, pending:0, ...} even though the consumer thread was alive and dispatching MEM_READ tasks. Root cause: handle_scheduler_allstatus only aggregated per-stream entries whose key starts with "scheduler:" — these only exist for the Redis-backed queue monitor. The local queue monitor returns just flat top-level totals (running, remaining, pending), so the prefix filter collapsed the summary to zero regardless of actual queue depth or dispatcher state.

Fix: in src/memos/api/handlers/scheduler_handler.py::handle_scheduler_allstatus, detect whether per-stream entries are present; if not and mem_scheduler.use_redis_queue is False, fall back to the flat totals so in_progress, waiting, and pending reflect live local-queue state. Redis-queue aggregation is untouched. The API response model is unchanged, so no OpenAPI regeneration is required; completed/failed/cancelled remain zero for local queues (documented behavior — persistent task status tracking is Redis-only).

Tests: added 3 regression tests in tests/api/test_scheduler_handler_allstatus.py covering local-non-zero, local-empty, and Redis per-stream paths. Initial run of the local-non-zero test produced the expected RED (assert 0 == 2); after the fix all three pass. Wider regression sweep across tests/api/test_server_router.py + tests/mem_scheduler/ + the new file: 79 passed / 1 skipped. Four pre-existing failures in tests/api/test_mcp_serve.py are unrelated to this change (confirmed by stashing the diff and reproducing on the base branch HEAD). Ruff lint + format clean on the touched files. The secondary symptom in the issue (missing scheduler INFO logs on Docker stdout) is caused by memos/log.py pinning the console handler to WARNING when DEBUG=False; this is documented in the design doc as out of scope to avoid a broader operational change.

Related Issue (Required): Fixes #1395

Type of change

Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Refactor (does not change functionality, e.g. code style improvements, linting)
  • Documentation update

How Has This Been Tested?

Automated tests are pending.

  • Unit Test
  • Test Script Or Test Steps (please provide)
  • Pipeline Automated API Test (please provide)

Checklist

  • I have performed a self-review of my own code
  • I have commented my code in hard-to-understand areas
  • I have added tests that prove my fix is effective or that my feature works
  • I have created related documentation issue/PR in MemOS-Docs (if applicable)
  • I have linked the issue to this PR (if applicable)
  • I have mentioned the person who will review this PR

@MatthewZhuang, @CarltonXiang, @syzsunshine219, @World-controller please review this PR.

Reviewer Checklist

handle_scheduler_allstatus only aggregated per-stream entries whose key
starts with "scheduler:" — these only exist for the Redis-backed message
queue. Under DEFAULT_USE_REDIS_QUEUE=false (Docker, no Redis) the
TaskScheduleMonitor returns just the flat top-level totals (running,
remaining, pending) without any per-stream entries, so the prefix filter
collapsed the summary to zero regardless of actual queue depth or
dispatcher in-flight count. Users observed scheduler "initialized but not
processing tasks" because /product/scheduler/allstatus always reported
all zeros even though the consumer thread was alive and dispatching
MEM_READ tasks.

Detect whether per-stream entries are present; if not and
mem_scheduler.use_redis_queue is False, fall back to the flat totals so
in_progress / waiting / pending reflect the live state of the local
queue and dispatcher. Redis-queue aggregation is unchanged.

Fixes #1395
@Memtensor-AI

Copy link
Copy Markdown
Collaborator Author

✅ Automated Test Results: PASSED

No applicable test scope for the changed files — automated tests skipped. Changed paths do not map to any configured scope (env.yaml source_mapping). Manual review recommended.

Branch: bugfix/autodev-1395

@Memtensor-AI
Memtensor-AI changed the base branch from dev-20260624-v2.0.22 to dev-v2.0.22 June 30, 2026 12:25
@CarltonXiang
CarltonXiang requested a review from bittergreen July 2, 2026 07:45
@CarltonXiang CarltonXiang added the area:core MOS 编排层 / 框架底座 / 跨模块问题 label Jul 2, 2026
@Memtensor-AI

Copy link
Copy Markdown
Collaborator Author

Automated Test Results: PASSED

Cloud test-engine rerun against dev-v2.0.22 completed successfully.

  • Run: tr-327fdb0a-fb7 on cloud test-engine 10012
  • Scope: memos_python_core/changed-repo-python
  • Result: 3 passed, 0 failed, 0 skipped

This replaces the earlier no-scope signal for this PR. Manual code review is still required before merge.

@bittergreen bittergreen left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice

@bittergreen
bittergreen merged commit e3f4e51 into dev-v2.0.22 Jul 2, 2026
16 checks passed
@bittergreen
bittergreen deleted the bugfix/autodev-1395 branch July 2, 2026 11:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ai:generated Generated or modified by AI | 由 AI 生成或修改 area:core MOS 编排层 / 框架底座 / 跨模块问题 types:bug Something isn't working | 功能异常

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants