refactor(background_tasks): follow the module layout (moves only)

CHANGELOG.md

@@ -85,6 +85,7 @@ All notable changes to this project are documented in this file. The format is b
 - `mail` follows the module layout: `mail.core.{constants,settings,state,data_compatibility}` and `mail.delivery.{sender,runtime}`. `mail.settings` and `mail.constants` still import, with a `DeprecationWarning`, until 0.0.46.
 - `site_lock` follows the module layout: `site_lock.core.{constants,settings,state}` and `site_lock.web.{middleware,page,rate_limit}` (the unlock template moves with the page). `site_lock.settings` still imports, with a `DeprecationWarning`, until 0.0.46.
 - `sharing` follows the module layout: `sharing.core.{constants,settings,services,service,policy,audience,tenant_cleanup,user_cleanup}`, `sharing.web.{deps,present,endpoints}`, and the page components and types move to `ui/`. `sharing.policy`, `sharing.constants` and `sharing.endpoints.api` still import, with a `DeprecationWarning`, until 0.0.46; new code imports `sharing.core.policy`.
+- `background_tasks` follows the module layout: `background_tasks.core.*` (constants, settings, services, config guards, health, invalidation, `sync_db`, tracking), `background_tasks.worker.*` (`celery_app`, `runner`, signals, task routing, heartbeats, worker settings, fleet inspection), `background_tasks.{executions,outbox,logs,web}.*`; the page components and TS helpers move to `ui/`. `background_tasks.{runner,celery_app,settings,sync_db,tenant_context,config_guards,task_routing,worker_settings,outbox_transport,outbox_worker}` still import, with a `DeprecationWarning`, until 0.0.46, so existing `scripts/run_worker.py` files keep working. Celery task names, durable event and handler names and the package-root exports are unchanged; module-name loggers move with their modules.
 - `dashboard` follows the module layout: `dashboard.core.constants`, `dashboard.web.{endpoints,stats,doctor}`, and the page components move from `pages/components/` to `ui/components/`. Nothing outside the module imported the old paths, so no aliases remain.
 - `feature_flags` follows the module layout: `feature_flags.core.{constants,service,invalidation,tenant_cleanup}`, `feature_flags.web.{deps,endpoints}`, and the page components and types move to `ui/`. `feature_flags.constants` still imports, with a `DeprecationWarning`, until 0.0.46.
 - `keycloak` follows the module layout: `keycloak.core.{constants,settings,state}`, `keycloak.identity.{provider,oidc,renewal,directory,persistence,tokens,jwks}` and `keycloak.web.endpoints`. `keycloak.{constants,settings,provider,jwks,tokens}` still import, with a `DeprecationWarning`, until 0.0.46.

docs/framework/leases.md

@@ -45,7 +45,7 @@ A lease does not refresh itself. Pick a TTL longer than the work, or call `refre
 
 ## Backends
 
-The backend is process-global because a Celery worker has no `app.state`. `create_app` installs it when the app is built, and the worker bootstrap (`background_tasks.runner.build_runner_celery`) installs the same kind from the same settings:
+The backend is process-global because a Celery worker has no `app.state`. `create_app` installs it when the app is built, and the worker bootstrap (`background_tasks.worker.runner.build_runner_celery`) installs the same kind from the same settings:
 
 | Configured | Backend | Notes |
 |---|---|---|

docs/framework/multi-tenancy.md

@@ -300,7 +300,7 @@ An unscoped event's handler runs with no tenant bound — never as the platform
 — so a public webhook's payload cannot drive a cross-tenant read. See
 [background_tasks](../modules/background_tasks.md).
 
-The worker never builds the app, so `background_tasks.sync_db` attaches the
+The worker never builds the app, so `background_tasks.core.sync_db` attaches the
 same listeners to its own session class and reads `multi_tenant` from the host
 settings (`scripts/run_worker.py`): task bodies get the same fail-closed rules
 as request code. It also passes `default_tenant`, so an unbound insert in a

docs/module-layout.md

@@ -32,8 +32,8 @@ checkers, not a framework runtime lookup.
 | `py.typed` | PEP 561 consumers; `framework/cli/simple_module_cli/templates/module/pyproject.toml.tpl:33` selects the whole package for the wheel, including this marker. |
 | `module.py` | `framework/core/simple_module_core/discovery.py:80` loads the `simple_module` entry point; the scaffold declares its target in `framework/cli/simple_module_cli/templates/module/pyproject.toml.tpl:15`. |
 | `models.py` or `models/` | `framework/db/simple_module_db/migrations.py:74` imports the module's models for metadata; `framework/core/simple_module_core/diagnostics/_tenancy.py:94` and `framework/core/simple_module_core/diagnostics/_expression_index.py:52` do the same. A `models/` package must load its tables. |
-| `tasks.py` | `modules/background_tasks/background_tasks/celery_app.py:215` autodiscovers with `related_name="tasks"`; `framework/core/simple_module_core/diagnostics/_heavy_task_imports.py:65` checks this root file (SM028). |
-| `task_config.py` | `modules/background_tasks/background_tasks/task_routing.py:46` imports `<package>.task_config` for a module's Celery routes and beat schedule. |
+| `tasks.py` | `modules/background_tasks/background_tasks/worker/celery_app.py:215` autodiscovers with `related_name="tasks"`; `framework/core/simple_module_core/diagnostics/_heavy_task_imports.py:65` checks this root file (SM028). |
+| `task_config.py` | `modules/background_tasks/background_tasks/worker/task_routing.py:46` imports `<package>.task_config` for a module's Celery routes and beat schedule. |
 | `contracts/` | Public imports for sibling modules; the scaffold consumer is `framework/cli/simple_module_cli/templates/module/__PACKAGE__/web/service.py.tpl:15`. Keep public SQLModel DTOs, events and real extension contracts here. |
 | `pages/` | `framework/hosting/simple_module_hosting/manifest.py:97` discovers pages; `framework/hosting/simple_module_hosting/page_globs.py:34` emits their Vite glob; `framework/core/simple_module_core/diagnostics/_pages.py:170` checks page names. |
 | `pages/api.ts`, `pages/api.generated.ts` | `framework/cli/simple_module_cli/openapi_types.py:130` selects the root pages directory for generation; `framework/cli/simple_module_cli/templates/module/__PACKAGE__/pages/api.ts.tpl:8` consumes the generated types. |
@@ -215,7 +215,7 @@ when a later module PR moves files.
 | api_tokens | 3 (+4 shims) | 0 | follows this layout |
 | audit_log | 3 | 0 | follows this layout |
 | auth | 3 (+4 shims) | 0 | follows this layout |
-| background_tasks | 49 | 83 | flat |
+| background_tasks | 4 (+10 shims) | 0 | follows this layout |
 | branding | 2 (+6 shims) | 0 | follows this layout |
 | dashboard | 2 | 0 | follows this layout |
 | feature_flags | 3 (+1 shim) | 0 | follows this layout |

docs/modules/audit_log.md

@@ -31,7 +31,7 @@ A worker never runs `create_app`, so it has no `app.state.sm.db` for `on_startup
 audit_log = "audit_log.core.capture:audit_callback"
 ```
 
-`background_tasks` binds whatever `simple_module_db.audit_sinks.discover_audit_callback()` finds to both of its worker engines — the sync engine task bodies and signal handlers use (`sync_db`) and the async engine durable outbox handlers run on (`outbox_worker`). A task's ORM writes, including `TaskExecution` lifecycle rows, are therefore audited exactly like a request's whenever `audit_log` is installed; neither package imports the other. The worker's audit rows carry no `user_id` (no request principal).
+`background_tasks` binds whatever `simple_module_db.audit_sinks.discover_audit_callback()` finds to both of its worker engines — the sync engine task bodies and signal handlers use (`core/sync_db`) and the async engine durable outbox handlers run on (`outbox/worker`). A task's ORM writes, including `TaskExecution` lifecycle rows, are therefore audited exactly like a request's whenever `audit_log` is installed; neither package imports the other. The worker's audit rows carry no `user_id` (no request principal).
 
 ### Statement DML: bulk, Core and raw writes
 

docs/modules/background_tasks.md

@@ -226,7 +226,7 @@ Entry values are plain Celery [beat entries](https://docs.celeryq.dev/en/stable/
 
 Beat persists its schedule (`celery.beat:PersistentScheduler`) so it survives restarts.
 
-The worker and beat read the same DB-backed settings the web process does: `scripts/run_worker.py` (and the copy `smpy new` scaffolds) calls `background_tasks.runner.build_runner_celery()`, which hydrates `BackgroundTasksSettings` from the database at boot exactly as the web lifespan does. Except for the DB-only `execution_tracking` and `track_publish` fields, precedence is env → DB → default: an `SM_BG_TASKS_*` variable that is set wins over a stored value for the same field, and the database supplies everything the environment leaves unset. So an admin-edited queue or time limit reaches the worker after a restart, unless the deployment pins that field in its environment. Because the web process behaves identically, this adds no new trust boundary; still, treat write access to these settings as control of the workers. If the database cannot be read the worker starts from env and defaults and logs a warning.
+The worker and beat read the same DB-backed settings the web process does: `scripts/run_worker.py` (and the copy `smpy new` scaffolds) calls `background_tasks.worker.runner.build_runner_celery()`, which hydrates `BackgroundTasksSettings` from the database at boot exactly as the web lifespan does. Except for the DB-only `execution_tracking` and `track_publish` fields, precedence is env → DB → default: an `SM_BG_TASKS_*` variable that is set wins over a stored value for the same field, and the database supplies everything the environment leaves unset. So an admin-edited queue or time limit reaches the worker after a restart, unless the deployment pins that field in its environment. Because the web process behaves identically, this adds no new trust boundary; still, treat write access to these settings as control of the workers. If the database cannot be read the worker starts from env and defaults and logs a warning.
 
 ## Routing and queues
 
@@ -260,7 +260,7 @@ A publisher that never imported a task can still enqueue it by name: `enqueue_af
 ## Worker tuning
 
 Task bodies can inspect the host's configured tenancy with the public
-`background_tasks.sync_db.worker_tenancy() -> WorkerTenancy | None` (#685).
+`background_tasks.core.sync_db.worker_tenancy() -> WorkerTenancy | None` (#685).
 `WorkerTenancy` is an immutable SQLModel snapshot with `multi_tenant: bool` and
 `default_tenant: str | None`. It returns `None` until `set_database_url` pins a
 URL, including when a URL is only available from the environment. A configured
@@ -633,7 +633,7 @@ Published (and consumable from any other module):
 
 ## Worker introspection
 
-`WorkerInspector` (`background_tasks/worker_inspector.py`) wraps `celery.control.inspect()`:
+`WorkerInspector` (`background_tasks/worker/inspector.py`) wraps `celery.control.inspect()`:
 
 | Method | Returns |
 |---|---|
@@ -645,7 +645,7 @@ It's synchronous; the view route calls it via `asyncio.to_thread`.
 
 The table below describes full tracking with `track_publish=True`; reduced modes gate these writes as described under [Execution tracking](#execution-tracking).
 
-Synchronous handlers in `signals.py`, all bound through Celery's `signals.<name>.connect`, persisting through a separate **sync** SQLAlchemy engine (`sync_db.py`) so they don't have to spin up an event loop:
+Synchronous handlers in `worker/signals.py`, all bound through Celery's `signals.<name>.connect`, persisting through a separate **sync** SQLAlchemy engine (`core/sync_db.py`) so they don't have to spin up an event loop:
 
 | Signal | What it writes |
 |---|---|

framework/cli/simple_module_cli/templates/host/_optional/background_tasks/run_worker.py

@@ -6,17 +6,17 @@
     uv run celery -A scripts.run_worker:celery beat   -l info
 
 Both the web process and the worker go through the same
-:func:`background_tasks.celery_app.build_celery` factory, so the broker
+:func:`background_tasks.worker.celery_app.build_celery` factory, so the broker
 config, autodiscovered tasks, and signal handlers stay in lockstep. The
 bootstrap around it (``.env``, host settings, DB-hydrated module settings)
-lives in :mod:`background_tasks.runner`, shared with scaffolded apps.
+lives in :mod:`background_tasks.worker.runner`, shared with scaffolded apps.
 """
 
 from __future__ import annotations
 
 from pathlib import Path
 
-from background_tasks.runner import build_runner_celery
+from background_tasks.worker.runner import build_runner_celery
 
 # Module-level name ``celery`` is what ``celery -A scripts.run_worker:celery``
 # looks for. Keep it stable. The fallback is the repo-root ``.env`` (next to

framework/cli/tests/test_cli_recipes.py

@@ -34,7 +34,7 @@ def test_recipe_writes_run_worker_script(tmp_path: Path) -> None:
     text = script.read_text()
     # GH #490: the shared runner, which hydrates DB-backed settings — not a
     # local copy built from the environment alone.
-    assert "from background_tasks.runner import build_runner_celery" in text
+    assert "from background_tasks.worker.runner import build_runner_celery" in text
     assert "celery = build_runner_celery(" in text
     assert "BackgroundTasksSettings()" not in text
 

framework/db/simple_module_db/query_filter.py

@@ -46,7 +46,7 @@
 )
 
 # ``_engine_policy`` (imported above) is re-exported: the per-engine policy map lives in
-# ``_engine_policies``; modules/background_tasks/tests/test_worker_tenancy.py reads it here.
+# ``_engine_policies``; the background_tasks worker tenancy test reads it here.
 
 
 class EngineTenancy:

framework/hosting/tests/test_invalidation_wiring.py

@@ -107,7 +107,7 @@ class _StubState:
 
 class _StubApp:
     def __init__(self, bus: InvalidationBus, settings) -> None:
-        from background_tasks.services import BackgroundTasksServices
+        from background_tasks.core.services import BackgroundTasksServices
 
         self.state = _StubState()
         self.state.sm = _StubSm(bus)
@@ -116,8 +116,8 @@ def __init__(self, bus: InvalidationBus, settings) -> None:
 
 class TestShutdown:
     async def test_shutdown_detaches_and_stops_the_transport(self, monkeypatch):
+        from background_tasks.core.settings import BackgroundTasksSettings
         from background_tasks.module import BackgroundTasksModule
-        from background_tasks.settings import BackgroundTasksSettings
 
         stopped: list[bool] = []
 
@@ -134,7 +134,7 @@ async def publish(self, message: str) -> None:
             async def stop(self) -> None:
                 stopped.append(True)
 
-        import background_tasks.invalidation as bg_invalidation
+        import background_tasks.core.invalidation as bg_invalidation
 
         monkeypatch.setattr(bg_invalidation, "RedisInvalidationTransport", _FakeTransport)
 
@@ -164,8 +164,8 @@ class TestStartupAgainstADeadBroker:
     async def test_an_unreachable_url_does_not_fail_the_boot(self):
         import socket
 
+        from background_tasks.core.settings import BackgroundTasksSettings
         from background_tasks.module import BackgroundTasksModule
-        from background_tasks.settings import BackgroundTasksSettings
 
         with socket.socket() as probe:
             probe.bind(("127.0.0.1", 0))
@@ -193,8 +193,8 @@ async def test_publishing_through_a_dead_broker_still_evicts_locally(self):
         """And the app keeps working afterwards, degraded to per-process."""
         import socket
 
+        from background_tasks.core.settings import BackgroundTasksSettings
         from background_tasks.module import BackgroundTasksModule
-        from background_tasks.settings import BackgroundTasksSettings
 
         with socket.socket() as probe:
             probe.bind(("127.0.0.1", 0))
@@ -223,7 +223,7 @@ async def test_publishing_through_a_dead_broker_still_evicts_locally(self):
 class TestSettingsValidation:
     def test_an_empty_channel_name_is_rejected(self):
         """Subscribing to "" would silently hear nothing."""
-        from background_tasks.settings import BackgroundTasksSettings
+        from background_tasks.core.settings import BackgroundTasksSettings
         from pydantic import ValidationError
 
         with pytest.raises(ValidationError):

framework/testing/simple_module_test/plugin.py

@@ -92,8 +92,8 @@ def _bootstrap_eager_celery() -> None:
     """
     os.environ.setdefault("SM_BG_TASKS_BROADCAST_INVALIDATIONS", "false")
     with contextlib.suppress(ImportError):
-        from background_tasks.celery_app import build_celery
-        from background_tasks.settings import BackgroundTasksSettings
+        from background_tasks.core.settings import BackgroundTasksSettings
+        from background_tasks.worker.celery_app import build_celery
 
         build_celery(BackgroundTasksSettings(task_always_eager=True, task_eager_propagates=True))
 

host/tests/test_module_layout.py

@@ -36,6 +36,7 @@
     "api_tokens": frozenset({"credentials"}),
     "audit_log": frozenset({"entries"}),
     "auth": frozenset({"oidc_common"}),
+    "background_tasks": frozenset({"executions", "logs", "outbox", "worker"}),
     "branding": frozenset({"assets"}),
     "file_storage": frozenset({"backends", "files", "multipart"}),
     "keycloak": frozenset({"identity"}),
@@ -55,20 +56,6 @@
 
 # Frozen inventory, not a computed baseline: moves must remove their old entries.
 LEGACY_FLAT: dict[str, frozenset[str]] = {
-    "background_tasks": frozenset(
-        """
-        _lifecycle.py _signal_bus.py _signal_support.py after_commit.py celery_app.py
-        config_guards.py constants.py db_settings.py deps.py endpoints filters.py health.py
-        invalidation.py log_context.py log_purge.py log_service.py log_sink.py
-        outbox_claim.py outbox_clock.py outbox_dispatch.py outbox_guard.py outbox_loop.py
-        outbox_service.py outbox_support.py outbox_sweep.py outbox_transport.py
-        outbox_worker.py retry_service.py runner.py service.py services.py settings.py
-        settings_celery.py settings_env_checks.py settings_logs.py signals.py sync_db.py
-        task_heartbeat.py task_resource.py task_routing.py tenant_celery.py
-        tenant_context.py tracking.py worker_inspector.py worker_settings.py
-        workers_state.py
-        """.split()
-    ),
     "users": frozenset(
         """
         admin administrators.py audit.py auth_local backend.py bootstrap.py constants.py

modules/background_tasks/README.md

@@ -14,7 +14,7 @@ Requires a Redis broker. The broker / result-backend URLs are module settings (`
 
 - Zero-config task discovery — any installed module that ships a `tasks.py` has its tasks autodiscovered (`celery.autodiscover_tasks` imports `<package>.tasks` for every installed module). No per-module registration hook.
 - Admin UI at `/admin/background-tasks` — list recent runs, retry failed, inspect tracebacks (gated by the `background_tasks.view` permission).
-- `build_celery(settings)` factory in `background_tasks.celery_app`, plus `bind_task_context` / `get_log_context` / `install_log_filter` exported from the package root (import name `background_tasks`, distribution name `simple_module_background_tasks`).
+- `build_celery(settings)` factory in `background_tasks.worker.celery_app`, plus `bind_task_context` / `get_log_context` / `install_log_filter` exported from the package root (import name `background_tasks`, distribution name `simple_module_background_tasks`).
 - `enqueue_after_commit(db, task, *args, **kwargs)` — publish a task only once the session's transaction commits, never on rollback.
 - `heartbeat()` — called from a long task body, keeps the stuck sweep from flagging it; `@shared_task(stuck_after=seconds)` sets a per-task stuck threshold. See the module docs for both.