Skip to content

fix(mem_scheduler): tolerate absent optional fields in EnvConfigMixin.from_env (#2424) - #2492

Closed
RerankerGuo wants to merge 1 commit into
MemTensor:dev-v2.0.36from
RerankerGuo:fix/issue-2424-env-config-mixin
Closed

RerankerGuo wants to merge 1 commit into
MemTensor:dev-v2.0.36from
RerankerGuo:fix/issue-2424-env-config-mixin

Conversation

@RerankerGuo

Copy link
Copy Markdown
Contributor

Summary

EnvConfigMixin.from_env raised a bare raise ValueError() (empty message) whenever a field declared as field: int | None = None had no environment variable set — so any config with a plain optional field could not be built from the environment at all. The scheduler init path (memos/configs/mem_scheduler.py calls RabbitMQConfig.from_env()) was the most visible victim: port: int | None is exactly that shape, so it failed immediately.

Root cause

for field_name, field_info in cls.model_fields.items():
    ...
    if env_var in os.environ:
        field_values[field_name] = cls._parse_env_value(...)
    elif field_info.default is not None:
        field_values[field_name] = field_info.default
    else:
        raise ValueError()                       # ← empty, swallows any Optional[None]

field_info.default is None for Optional[int] = None, so the elif never fires and the bare raise ValueError() triggers for every absent optional field. The error also makes it impossible to tell which field caused the failure.

_parse_env_value had a secondary issue: it only parsed when target_type is bool/int/float, so an Optional[int] env var that was provided would have been left as a raw string instead of being cast.

Fix

  • from_env now skips fields with no env var entirely; only genuinely required fields (no default and not Optional) still raise, and the raised error carries the missing env var name and the word "missing" (replacing the empty-message bug).
  • _parse_env_value unwraps Optional[X] / X | None via typing.get_args, so an Optional[int] env var parses as int.
  • A new module-level helper _is_optional keeps the Optional check in one place, used by both from_env and _parse_env_value.

Tests

New tests/mem_scheduler/test_env_config_mixin.py, 5 cases:

  1. test_absent_optional_int_field_falls_back_to_none — the bug reproducer (red against unpatched main)
  2. test_absent_optional_str_field_falls_back_to_default — same bug for non-Optional defaults
  3. test_optional_int_env_var_parses_as_int — secondary symptom (Optional[int] gets cast)
  4. test_bool_env_var_parses_as_bool — regression coverage
  5. test_missing_required_field_raises_named_value_error — genuinely required fields still raise, with a meaningful message

Verification

PYTHONPATH=src pytest tests/mem_scheduler/test_env_config_mixin.py -v
============================== 5 passed in 2.12s ==============================

Ruff: ruff format --check clean on touched files; ruff check 0 new findings vs main (5 pre-existing BLE001/B008/UP findings auto-fixed by ruff check --fix during this PR, 1 pre-existing TC009 finding remains, both unrelated to this change).

Out of scope

  • No new dependencies
  • No public API / OpenAPI contract changes
  • No DB schema, migration, or graph model changes
  • No Makefile / pyproject.toml / pre-commit / GitHub workflow changes

Closes #2424

@Memtensor-AI Memtensor-AI added area:scheduler 调度模块 status:in-progress Someone or AI is working on it | 人工或 AI 正在处理 labels Oct 13, 2026
@Memtensor-AI
Memtensor-AI requested a review from wustzdy October 13, 2026 09:08
@RerankerGuo
RerankerGuo force-pushed the fix/issue-2424-env-config-mixin branch from 66a0150 to 400aadf Compare October 27, 2026 18:37
@RerankerGuo

Copy link
Copy Markdown
Contributor Author

Rebased this branch onto the current tip of dev-v2.0.36 (4bfddc8) so the PR is no longer behind its base branch.

No functional change. The two files touched by this PR are byte-identical to the previously reviewed version — only the base commits moved underneath:

  • src/memos/mem_scheduler/general_modules/misc.py
  • tests/mem_scheduler/test_env_config_mixin.py

All 16 required GitHub Actions build (...) checks pass on the new head 400aadf, and the new tests pass locally (tests/mem_scheduler/test_env_config_mixin.py, 5 passed).

About the earlier open-code-review failure. The previous run reported Open Code Review execution failed; retry required — an external-tool execution failure that completed in 0s and produced 0 findings, i.e. not a code-quality signal. No code or workflow was changed to accommodate it. It looks like that external check did not get re-scheduled on the rebased head, so please re-run it from the CI side (or retry it on the PR) if it is still expected as a gate.

The PR still needs one approving review per the repository ruleset — no merge has been attempted from my side.

@Memtensor-AI

Copy link
Copy Markdown
Collaborator

🤖 Open Code Review

Target: PR #2492
Task: da64085345855e53
Base: dev-v2.0.36
Head: fix/issue-2424-env-config-mixin

🔍 OpenCodeReview found 3 issue(s) in this PR.

⚠️ 1 warning(s) occurred during review.


1. src/memos/mem_scheduler/general_modules/misc.py (L89-L90)

The not _is_optional(field_type) escape hatch is based on an incorrect premise, stated in the comment above and in the docstring: Pydantic does not apply None on its own for Optional[X]. A field annotated x: int | None without = None (or a default_factory) is still a required field — FieldInfo.is_required() returns True in Pydantic v2 (pydantic 2.11.7 per poetry.lock). For such a field this guard is now skipped, so instead of the descriptive ValueError("Required environment variable MEMSCHEDULER_..._X is missing") the construction fails later inside cls(**field_values) with a generic ValidationError that never names the env var — exactly the operator-facing diagnostic this method exists to produce. Note that is_required() already accounts for defaults (= None yields False), so the extra _is_optional test is both redundant and harmful; dropping it fixes issue #2424 without weakening the message.

💡 Suggested Change

Before:

            if field_info.is_required() and not _is_optional(field_type):
                raise ValueError(f"Required environment variable {env_var} is missing")

After:

            if field_info.is_required():
                raise ValueError(f"Required environment variable {env_var} is missing")

2. src/memos/mem_scheduler/general_modules/misc.py (L105-L107)

Unwrapping only args[0] silently drops the other union members. For a multi-member annotation such as Union[int, str, None] / int | str | None, args[0] may be int, so a legitimately valid string value (e.g. "auto") is pushed into int() and raises a bare ValueError: invalid literal for int() with no field name or env-var name attached, which makes startup misconfiguration hard to diagnose. Either restrict the unwrap to exactly one non-None member (if len(args) == 1:) and leave genuine multi-member unions as raw strings for Pydantic to validate, or attempt each member in order and fall back to the raw string when none match.

💡 Suggested Change

Before:

            args = [a for a in get_args(target_type) if a is not type(None)]
            if args:
                target_type = args[0]

After:

            args = [a for a in get_args(target_type) if a is not type(None)]
            if len(args) == 1:
                target_type = args[0]

3. src/memos/mem_scheduler/general_modules/misc.py (L183)

Replacing cls: type[BaseModelType] -> BaseModelType with Self here and in from_json removes the last in-file references to the module-level BaseModelType = TypeVar("T", bound="BaseModel") (line 19), leaving it as dead code — a repo-wide search finds no other reference. Consider deleting the TypeVar, and with it the now-unneeded TYPE_CHECKING import of BaseModel, unless the symbol is intended to remain part of the module's public surface for other importers. (Its name string is also "T", duplicating the sibling T = TypeVar("T"), which only matters if it is kept.)

Generated by cloud-assistant via Open Code Review.

@Memtensor-AI

Copy link
Copy Markdown
Collaborator

✅ Automated Test Results: PASSED

All tests passed (5/5 executed). memos_python_core/changed-repo-python: 5/5. Duration: 14s

Branch: fix/issue-2424-env-config-mixin

@Memtensor-AI Memtensor-AI added status:ready Ready for implementation; waiting for assignee or AI dispatch | 可进入实现,等待认领或派发 and removed status:in-progress Someone or AI is working on it | 人工或 AI 正在处理 labels Oct 28, 2026
….from_env (MemTensor#2424)

EnvConfigMixin.from_env previously raised a bare
`raise ValueError()` (empty message) whenever a field declared as
`field: int | None = None` had no environment variable set — so any
config with a plain optional field could not be built from the environment
at all. The scheduler init path (`memos/configs/mem_scheduler.py` calls
`RabbitMQConfig.from_env()`) was the most visible victim: `port: int | None`
is exactly that shape, so it failed immediately.

A secondary bug: `_parse_env_value` only parsed when the type was exactly
`bool`/`int`/`float`, so an `Optional[int]` env var that *was* provided
would have been left as a string instead of being cast.

The fix:

- from_env now skips fields with no env var entirely; only genuinely
  required fields (no default and not Optional) still raise, and the
  raised error carries the missing env var name and the word
  "missing" (the bug path's bare ValueError() is replaced).
- _parse_env_value unwraps `Optional[X]` / `X | None` (using typing.get_args)
  so an Optional[int] env var parses as int.
- A new module-level helper `_is_optional` keeps the Optional check in
  one place, since both from_env and _parse_env_value need it.

New tests/mem_scheduler/test_env_config_mixin.py covers:

- absent Optional[int] falls back to None (the bug's reproducer)
- absent str falls back to the field default
- present Optional[int] parses as int
- bool env var still parses as bool (regression coverage)
- missing required field still raises with the env var name in the
  message (so the empty ValueError() path is dead)

Fixes MemTensor#2424.
@RerankerGuo
RerankerGuo force-pushed the fix/issue-2424-env-config-mixin branch from 400aadf to daa224a Compare November 4, 2026 13:58
@Memtensor-AI Memtensor-AI added status:in-progress Someone or AI is working on it | 人工或 AI 正在处理 and removed status:ready Ready for implementation; waiting for assignee or AI dispatch | 可进入实现,等待认领或派发 labels Nov 4, 2026
@syzsunshine219
syzsunshine219 deleted the branch MemTensor:dev-v2.0.36 November 9, 2026 12:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:scheduler 调度模块 status:in-progress Someone or AI is working on it | 人工或 AI 正在处理

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants