Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -96,6 +96,7 @@ All notable changes to this project are documented in this file. The format is b
- `audit_log` follows the module layout: `audit_log.core.{constants,settings,services,capture}`, `audit_log.entries.{service,filters,resolve,activity,export}`, `audit_log.web.{deps,endpoints}`, and the page components move from `pages/components/` to `ui/components/`. The `simple_module.audit_sinks` entry point now targets `audit_log.core.capture:audit_callback`. Nothing outside the module imported the old paths, so no aliases remain.
- `auth` follows the module layout: `auth.core.{constants,state}` and `auth.web.{deps,middleware}`; `auth.oidc_common` stays as a declared domain area. `auth.constants`, `auth.state`, `auth.deps` and `auth.middleware` still import, with a `DeprecationWarning`, until 0.0.46.
- `tenants` follows the module layout: `tenants.core.{constants,settings,services,errors,service,user_cleanup}` (with the membership, owner-guard and deletion helpers), `tenants.resolution.{resolver,host_resolver,token_tenant}`, `tenants.invitations.{service,mail}`, `tenants.web.{deps,shared_props,endpoints}`, and the page components and hooks move to `ui/components` and `ui/hooks`. The invitation mail template moves to `invitations/templates/invitation.txt`. `tenants.constants`, `tenants.errors`, `tenants.host_resolver`, `tenants.resolver`, `tenants.service` and `tenants.deps` still import, with a `DeprecationWarning`, until 0.0.46. The durable handler names `tenants.user_cleanup.purge_user_memberships` and `tenants.send_invitation_email` are unchanged.
- `file_storage` follows the module layout: `file_storage.core.{constants,settings,services,errors,scope,format,health,audit_labels,tenant_cache,tenant_deletion,object_reaper,worker,worker_support}`, `file_storage.files.*`, `file_storage.multipart.{sessions,guards,support,checksum}`, `file_storage.backends.guard`, `file_storage.web.{deps,serving,cors,csp,cookieless,body_limits,endpoints}`, and the page components, hook and helpers move to `ui/`. `file_storage.settings`, `.service`, `.deps`, `.scope`, `.downloads`, `.upload_support` and `.tenant_deletion` still import, with a `DeprecationWarning`, until 0.0.46; the other old flat paths (`.constants`, `.errors`, `.services`, `.endpoints.*`, `.upload_guards`, `.uploads`, `.checksum`, `.worker`, `.cors`, `.csp` and so on) have no alias. The durable `TenantStorageCleanupRequested` event key and its handler names, and the Celery task names, are unchanged.


- Coalesce concurrent tenant membership and permission grant cache misses. In-flight invalidation retries against fresh authorization data; failures and cancellation release waiting readers. Cache unknown tenant subdomains for five seconds and drop the new slug's host lookups (on a new `tenants.hosts` invalidation channel, so no user id is read as a slug) and the owner's memberships after committed tenant creation; other users' caches are not cleared. Services built outside request dependencies pass `invalidate_hosts=make_host_invalidator(app)` beside `invalidate=make_invalidator(app)`. Retries after concurrent invalidation are bounded to three reads (2026-11-05 local performance review).
Expand Down
2 changes: 1 addition & 1 deletion docs/module-layout.md
Original file line number Diff line number Diff line change
Expand Up @@ -219,7 +219,7 @@ when a later module PR moves files.
| branding | 2 (+6 shims) | 0 | follows this layout |
| dashboard | 2 | 0 | follows this layout |
| feature_flags | 3 (+1 shim) | 0 | follows this layout |
| file_storage | 39 | 65 | flat |
| file_storage | 4 (+7 shims) | 0 | follows this layout |
| keycloak | 3 (+5 shims) | 0 | follows this layout |
| mail | 2 (+2 shims) | 0 | follows this layout |
| oidc | 3 (+3 shims) | 0 | follows this layout |
Expand Down
2 changes: 1 addition & 1 deletion docs/modules/background_tasks.md
Original file line number Diff line number Diff line change
Expand Up @@ -678,7 +678,7 @@ Workers never run the hosting lifespan, so `FileStorageSettings()` (or any `DbBa

```python
from background_tasks import settings_for
from file_storage.settings import FileStorageSettings
from file_storage.core.settings import FileStorageSettings


@celery.task
Expand Down
4 changes: 2 additions & 2 deletions docs/modules/file_storage.md
Original file line number Diff line number Diff line change
Expand Up @@ -453,7 +453,7 @@ files the caller uploaded (`created_by` is their user id), unless they hold
`file_storage.manage`. Someone else's file answers `404`, exactly like an
unknown id, and the browse screen's type and uploader filters list only the
caller's own rows (the usage figure stays bucket-wide, as the quota is). The
scope is passed explicitly by the endpoints (`deps.get_owner_scope`) to the
scope is passed explicitly by the endpoints (`web.deps.get_owner_scope`) to the
service's `owner=` argument; internal callers such as branding and reapers
pass none and stay unscoped, and anonymous `public` serving is unaffected.

Expand Down Expand Up @@ -486,7 +486,7 @@ class MyModule(ModuleBase):
## Inertia pages

- `FileStorage/Browse.tsx` — file list + upload dropzone; handles the upload progress + delete confirmation flow. Each row shows a "Public" badge and a make public / make private action.
- `FileStorage/components/UploadDropzone.tsx` — drag-drop upload child component.
- `ui/components/UploadDropzone.tsx` — drag-drop upload child component.

## Locales

Expand Down
11 changes: 1 addition & 10 deletions host/tests/test_module_layout.py
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@
"audit_log": frozenset({"entries"}),
"auth": frozenset({"oidc_common"}),
"branding": frozenset({"assets"}),
"file_storage": frozenset({"backends", "files", "multipart"}),
"keycloak": frozenset({"identity"}),
"mail": frozenset({"delivery"}),
"oidc": frozenset({"identity"}),
Expand Down Expand Up @@ -68,16 +69,6 @@
workers_state.py
""".split()
),
"file_storage": frozenset(
"""
aggregates.py audit_labels.py backend_guard.py backends body_limits.py checksum.py
constants.py cookieless.py cors.py csp.py deletes.py deps.py downloads.py endpoints
errors.py format.py health.py media_types.py object_reaper.py queries.py ranges.py
reads.py resolve.py scope.py service.py services.py serving.py settings.py
tenant_cache.py tenant_deletion.py thumbnails.py upload_guards.py upload_support.py
uploads.py visibility.py worker.py worker_support.py
""".split()
),
"users": frozenset(
"""
admin administrators.py audit.py auth_local backend.py bootstrap.py constants.py
Expand Down
4 changes: 2 additions & 2 deletions modules/branding/branding/assets/images.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@

if TYPE_CHECKING:
from file_storage.contracts.schemas import StoredFileOut
from file_storage.service import FileStorageService
from file_storage.files.service import FileStorageService

# Raster + icon only. SVG is excluded on purpose: it is an XML document that can
# carry <script>, so serving one back from our own origin would be stored XSS.
Expand Down Expand Up @@ -111,7 +111,7 @@ async def store_image(
maps the refusal to 413, but branding calls the service directly, so an
unmapped ``QuotaExceededError`` would surface as a 500.
"""
from file_storage.upload_support import QuotaExceededError
from file_storage.multipart.support import QuotaExceededError

try:
return await storage.upload(file, platform=platform)
Expand Down
2 changes: 1 addition & 1 deletion modules/branding/branding/assets/reaper.py
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ def schedule_reap(app: FastAPI, db: AsyncSession, file_id: str, *, tenant_id: st


async def reap(app: FastAPI, file_id: str, *, tenant_id: str | None) -> None:
from file_storage.service import FileStorageService
from file_storage.files.service import FileStorageService

try:
parsed = uuid.UUID(file_id)
Expand Down
2 changes: 1 addition & 1 deletion modules/branding/branding/core/service.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@

if TYPE_CHECKING:
from fastapi import FastAPI
from file_storage.service import FileStorageService
from file_storage.files.service import FileStorageService
from sqlalchemy.ext.asyncio import AsyncSession

from branding.core.settings import BrandingSettings
Expand Down
4 changes: 2 additions & 2 deletions modules/branding/branding/web/deps.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,8 @@
from typing import Annotated

from fastapi import Depends, Request
from file_storage.deps import get_file_storage_service
from file_storage.service import FileStorageService
from file_storage.files.service import FileStorageService
from file_storage.web.deps import get_file_storage_service
from simple_module_db.deps import get_db
from sqlalchemy.ext.asyncio import AsyncSession

Expand Down
6 changes: 3 additions & 3 deletions modules/branding/branding/web/endpoints/api.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@
from __future__ import annotations

from fastapi import APIRouter, Depends, HTTPException, Request, UploadFile
from file_storage.deps import get_file_storage_service
from file_storage.service import FileStorageService
from file_storage.files.service import FileStorageService
from file_storage.web.deps import get_file_storage_service
from simple_module_core.app_state import get_design_packs
from simple_module_hosting.permissions import RequiresPermission

Expand All @@ -18,7 +18,7 @@

# Branding images are uploaded with ``platform=True``: they are the install's,
# not the uploading admin's active organisation's, and the anonymous asset
# routes must find them with no tenant bound (see ``file_storage.scope``).
# routes must find them with no tenant bound (see ``file_storage.core.scope``).

_MANAGE = Depends(RequiresPermission(constants.PERM_MANAGE))

Expand Down
6 changes: 3 additions & 3 deletions modules/branding/branding/web/endpoints/assets.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
``tenant_context(tenant)``, so only a row the tenant owns matches;
* a **system** value is read as a *platform* file (``platform=True``): an
``all_tenants()`` lookup restricted to rows owned by
``file_storage.scope.PLATFORM_TENANT_ID``.
``file_storage.core.scope.PLATFORM_TENANT_ID``.

Either way a setting pointed at someone else's file id serves a 404, never
their file.
Expand All @@ -26,13 +26,13 @@

from fastapi import APIRouter, Depends, HTTPException, Request
from fastapi.responses import RedirectResponse, StreamingResponse
from file_storage.deps import get_file_storage_service
from file_storage.service import (
from file_storage.files.service import (
FileStorageService,
RedirectDownload,
StoredFileNotFoundError,
StreamDownload,
)
from file_storage.web.deps import get_file_storage_service
from simple_module_db import tenant_context

from branding.assets.images import ALLOWED_IMAGE_TYPES, normalize_content_type
Expand Down
4 changes: 2 additions & 2 deletions modules/branding/branding/web/endpoints/tenant_api.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,8 @@
from __future__ import annotations

from fastapi import APIRouter, Depends, HTTPException, Request, UploadFile
from file_storage.deps import get_file_storage_service
from file_storage.service import FileStorageService
from file_storage.files.service import FileStorageService
from file_storage.web.deps import get_file_storage_service
from settings.contracts.schemas import SettingScope, SettingUpsert
from settings.core.constants import PERM_TENANT_EDIT
from settings.core.service import SettingService
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,8 @@
import pytest
from branding.core.settings import BrandingSettings
from branding.web.shared_props import branding_payload
from file_storage.files.service import FileStorageService, StreamDownload
from file_storage.models import StoredFile
from file_storage.service import FileStorageService, StreamDownload

_PNG = b"\x89PNG\r\n\x1a\n" + b"\x00" * 32

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,8 @@

import httpx
import pytest
from file_storage.files.service import FileStorageService, StoredFileNotFoundError
from file_storage.models import StoredFile
from file_storage.service import FileStorageService, StoredFileNotFoundError

_PNG = b"\x89PNG\r\n\x1a\n" + b"\x00" * 32

Expand Down
4 changes: 3 additions & 1 deletion modules/branding/tests/core/module/test_branding.py
Original file line number Diff line number Diff line change
Expand Up @@ -204,7 +204,9 @@ async def fake_upload(self, upload, *, platform: bool = False):
assert platform, "branding must act on platform-owned files"
return SimpleNamespace(id=fake_id)

monkeypatch.setattr("file_storage.service.FileStorageService.upload", fake_upload, raising=True)
monkeypatch.setattr(
"file_storage.files.service.FileStorageService.upload", fake_upload, raising=True
)

resp = await authenticated_client.post(
"/api/branding/logo",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,8 @@
import pytest
from branding.core import tenant_branding
from branding.core.constants import LOGO_URL
from file_storage.core.scope import PLATFORM_TENANT_ID
from file_storage.models import StoredFile
from file_storage.scope import PLATFORM_TENANT_ID
from settings.contracts.schemas import SettingScope, SettingUpsert
from settings.core.service import SettingService
from simple_module_db import all_tenants
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,8 @@
import httpx
import pytest
from branding.core.constants import LOGO_URL
from file_storage.core.scope import PLATFORM_TENANT_ID
from file_storage.models import StoredFile
from file_storage.scope import PLATFORM_TENANT_ID
from simple_module_db import all_tenants
from sqlalchemy import select

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,8 @@

import httpx
import pytest
from file_storage.files.service import FileStorageService, StreamDownload
from file_storage.models import StoredFile
from file_storage.service import FileStorageService, StreamDownload

_PNG = b"\x89PNG\r\n\x1a\n" + b"\x00" * 32

Expand Down
4 changes: 2 additions & 2 deletions modules/file_storage/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,8 +52,8 @@ From another module, inject the service via its dependency:
```python
from fastapi import Depends, File, UploadFile

from file_storage.deps import get_file_storage_service
from file_storage.service import FileStorageService
from file_storage.web.deps import get_file_storage_service
from file_storage.files.service import FileStorageService


async def attach_receipt(
Expand Down
2 changes: 1 addition & 1 deletion modules/file_storage/file_storage/backends/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ class with extra fields). :func:`build_backend` resolves the active

if TYPE_CHECKING:
from file_storage.contracts.service import StorageBackend
from file_storage.settings import FileStorageSettings
from file_storage.core.settings import FileStorageSettings

BackendFactory = Callable[["FileStorageSettings"], "StorageBackend"]

Expand Down
4 changes: 2 additions & 2 deletions modules/file_storage/file_storage/backends/filesystem.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,15 +12,15 @@

import aiofiles

from file_storage import constants
from file_storage.backends import register_backend
from file_storage.backends.filesystem_multipart import FilesystemMultipartMixin
from file_storage.contracts.service import (
NotSupportedError,
StorageBackendError,
StorageNotFoundError,
)
from file_storage.settings import FileStorageSettings
from file_storage.core import constants
from file_storage.core.settings import FileStorageSettings


class FilesystemBackend(FilesystemMultipartMixin):
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,14 +18,14 @@

import aiofiles

from file_storage import constants
from file_storage.contracts.service import (
NotSupportedError,
ObjectInfo,
Part,
StorageBackendError,
StorageNotFoundError,
)
from file_storage.core import constants

_UPLOAD_ID = re.compile(r"^[0-9a-f]{32}$")
_ETAG_SUFFIX = ".etag"
Expand Down
4 changes: 2 additions & 2 deletions modules/file_storage/file_storage/backends/s3.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,6 @@
from datetime import datetime
from typing import Any

from file_storage import constants
from file_storage.backends import register_backend
from file_storage.backends.s3_multipart import S3MultipartMixin
from file_storage.contracts.service import (
Expand All @@ -20,7 +19,8 @@
StorageError,
StorageNotFoundError,
)
from file_storage.settings import FileStorageSettings
from file_storage.core import constants
from file_storage.core.settings import FileStorageSettings

_MISSING_S3_EXTRA = (
"S3 backend requires the 'aioboto3' package. Install with: uv add --optional s3 aioboto3"
Expand Down
5 changes: 3 additions & 2 deletions modules/file_storage/file_storage/backends/s3_put.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,10 @@
from tempfile import SpooledTemporaryFile
from typing import TYPE_CHECKING, Any

from file_storage import constants, upload_support
from file_storage.contracts.service import Part, StorageBackendError
from file_storage.upload_support import SizeMismatchError
from file_storage.core import constants
from file_storage.multipart import support as upload_support
from file_storage.multipart.support import SizeMismatchError

if TYPE_CHECKING:
from file_storage.backends.s3 import S3Backend
Expand Down
6 changes: 3 additions & 3 deletions modules/file_storage/file_storage/contracts/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -50,20 +50,20 @@ def store_path(
public: bool = False,
) -> StoredFileOut:
"""Store a path or stream in the caller's tenant, in an independent transaction."""
from file_storage.worker import store_path as store
from file_storage.core.worker import store_path as store

return store(source, filename=filename, content_type=content_type, public=public)


def open_local(file_id: UUID) -> AbstractContextManager[Path]:
"""Download to a private local path for the lifetime of this context."""
from file_storage.worker import open_local as local
from file_storage.core.worker import open_local as local

return local(file_id)


def presigned_url(file_id: UUID, *, ttl_seconds: int | None = None) -> str:
"""Return a scoped download URL; unsupported backends raise NotSupportedError."""
from file_storage.worker import presigned_url as sign
from file_storage.core.worker import presigned_url as sign

return sign(file_id, ttl_seconds=ttl_seconds)
6 changes: 6 additions & 0 deletions modules/file_storage/file_storage/core/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
"""What every file_storage feature builds on.

Constants, settings, the app.state container, errors, tenant scope, byte
formatting, health, audit labels, the public-file tenant cache, durable tenant
teardown, the object reaper and synchronous worker access.
"""
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,8 @@
from fastapi.responses import JSONResponse
from simple_module_hosting.i18n_deps import get_translator

from file_storage import constants
from file_storage.backend_guard import BackendUnavailableError
from file_storage.backends.guard import BackendUnavailableError
from file_storage.core import constants


async def _backend_unavailable(request: Request, exc: Exception) -> JSONResponse:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,8 @@
from sqlalchemy import select
from sqlmodel import col

from file_storage import constants, thumbnails
from file_storage.core import constants
from file_storage.files import thumbnails
from file_storage.models import StoredFile, UploadSession

logger = logging.getLogger(__name__)
Expand Down
Loading
Loading