Replace settings_override regex-scrape with format AST parsing

README.md

@@ -97,7 +97,7 @@ flowchart TB
 
 Weblate discovers formats from the `WEBLATE_FORMATS` setting (see `FileFormatLoader` in upstream `weblate.formats.models`). The official Docker image evaluates a single optional file after base settings: if `/app/data/settings-override.py` exists, it is compiled and executed with `exec()` in the **same namespace** as the rest of `weblate.settings_docker`.
 
-Stock `weblate.settings_docker` does **not** always bind `WEBLATE_FORMATS` in that namespace before the hook runs, so a bare `WEBLATE_FORMATS += (...)` in the override can raise `NameError`. This repository ships `src/boost_weblate/settings_override.py` as the Docker `exec()` fragment: it assigns `WEBLATE_FORMATS` by **reading** upstream `weblate/formats/models.py` and regex-slicing `FormatsConf.FORMATS` (aligned with the installed Weblate version, without importing `weblate.formats.models` during settings load, which can raise `AppRegistryNotReady`). It also appends the endpoint Django app to `INSTALLED_APPS` — see [`WEBLATE_ADD_APPS`](#weblate_add_apps) below.
+Stock `weblate.settings_docker` does **not** always bind `WEBLATE_FORMATS` in that namespace before the hook runs, so a bare `WEBLATE_FORMATS += (...)` in the override can raise `NameError`. This repository ships `src/boost_weblate/settings_override.py` as the Docker `exec()` fragment: it assigns `WEBLATE_FORMATS` by **reading** upstream `weblate/formats/models.py` and AST-parsing `FormatsConf.FORMATS` (aligned with the installed Weblate version, without importing `weblate.formats.models` during settings load, which can raise `AppRegistryNotReady`). It also appends the endpoint Django app to `INSTALLED_APPS` — see [`WEBLATE_ADD_APPS`](#weblate_add_apps) below.
 
 **Operators:** ensure the plugin package is installed in the Weblate environment (`pip` / image layer), then install the override file where Weblate expects it. For the stock Docker layout:
 
@@ -107,7 +107,7 @@ COPY settings-override.py /app/data/settings-override.py
 
 That path is fixed; Weblate does not scan `DATA_DIR` for arbitrary override files. The override file is **not** the same as `WEBLATE_PY_PATH` / `python/customize` (importable customization on `sys.path`); for format registration, use this exec hook unless your image explicitly imports another settings module. See the comments in `settings_override.py` for the full distinction.
 
-**Adding another format:** implement the class under `boost_weblate/formats/`, append its dotted class path in `weblate_formats_with_quickbook()` (or extend the tuple built there), redeploy, and restart Weblate. If upstream changes the layout of `FormatsConf` in `models.py`, update the regex in `settings_override.py` accordingly.
+**Adding another format:** implement the class under `boost_weblate/formats/`, append its dotted class path in `weblate_formats_with_quickbook()` (or extend the tuple built there), redeploy, and restart Weblate. If upstream restructures `FormatsConf` in `models.py` (e.g. renames the class or moves `FORMATS` off a simple tuple assignment), update the AST helpers in `settings_override.py` accordingly.
 
 ## WEBLATE_ADD_APPS
 

docs/deployment-runbook.md

@@ -77,7 +77,7 @@ Do not duplicate pass-through vars in `environment:`; configure them once in `.e
 Build-time wiring (no env vars):
 
 1. **`settings_override.py`** is copied to `/app/data/settings-override.py` by the Dockerfile. Weblate's Docker entrypoint `exec()`s this file during settings load.
-2. **`WEBLATE_FORMATS`** — the override reads upstream `FormatsConf.FORMATS` via regex, appends `boost_weblate.formats.quickbook.QuickBookFormat`, and writes the result back to `WEBLATE_FORMATS`. No env var needed.
+2. **`WEBLATE_FORMATS`** — the override reads upstream `FormatsConf.FORMATS` via AST parse of `models.py`, appends `boost_weblate.formats.quickbook.QuickBookFormat`, and writes the result back to `WEBLATE_FORMATS`. No env var needed.
 3. **`INSTALLED_APPS`** — the override appends `boost_weblate.endpoint.apps.BoostEndpointConfig`. The app's `ready()` hook then registers `/boost-endpoint/` routes on `weblate.urls.real_patterns`.
 
 Runtime plugin env vars (set in `.env`, read by `settings_override.py` at boot):
@@ -420,7 +420,7 @@ Common causes:
 
 | Symptom | Likely cause | Fix |
 |---------|-------------|-----|
-| `AppRegistryNotReady` | Upstream Weblate reformatted `FormatsConf.FORMATS` | Update the `_FORMATS_BLOCK` regex in `settings_override.py` |
+| `RuntimeError` during settings `exec()` (e.g. `could not parse FormatsConf.FORMATS`) | Upstream Weblate restructured `FormatsConf.FORMATS` | Update the AST helpers in `settings_override.py` |
 | `connection refused` on Postgres | `pg_hba.conf` or firewall blocking Docker bridge | Allow `172.17.0.0/16` in `pg_hba.conf`; reload Postgres |
 | `WEBLATE_ADMIN_PASSWORD … set in .env` | `.env` missing or variable unset | Ensure `.env` exists at repo root with both required secrets |
 | `${WEBLATE_URL_PREFIX}/healthz/` 404 | `WEBLATE_URL_PREFIX` mismatch | Ensure `.env` has `WEBLATE_URL_PREFIX` matching nginx config |

src/boost_weblate/settings_override.py

@@ -9,13 +9,12 @@
 ``/app/data/settings-override.py`` (hyphen on disk) or keep it on ``PYTHONPATH`` and
 point your image at the same content.
 
-``WEBLATE_FORMATS`` is built by **reading** ``weblate/formats/models.py`` as text and
-regex-slicing ``FormatsConf.FORMATS``. That avoids ``import weblate.formats.models``,
+``WEBLATE_FORMATS`` is built by **reading** ``weblate/formats/models.py`` and
+AST-parsing ``FormatsConf.FORMATS``. That avoids ``import weblate.formats.models``,
 which pulls in Django ORM classes during settings import and raises
-``AppRegistryNotReady``. The slice is **layout-sensitive**: it assumes ``FORMATS = (``
-inside ``FormatsConf`` (same file) is followed by ``class Meta:`` at the same indent;
-if upstream reformats ``FormatsConf`` or moves ``FORMATS`` / ``Meta``, update
-``_FORMATS_BLOCK`` below.
+``AppRegistryNotReady``. If upstream restructures ``FormatsConf`` (e.g. renames the
+class or moves ``FORMATS`` off a simple tuple assignment), update the AST helpers
+below.
 
 When this file is ``exec``'d into Weblate's settings namespace (Docker),
 ``INSTALLED_APPS`` is taken from ``globals()`` and extended. Upstream
@@ -28,8 +27,8 @@
 
 from __future__ import annotations
 
+import ast
 import os
-import re
 from pathlib import Path
 from typing import Any
 
@@ -39,11 +38,39 @@
 _QUICKBOOK_FORMAT = "boost_weblate.formats.quickbook.QuickBookFormat"
 _ENDPOINT_APP_CONFIG = "boost_weblate.endpoint.apps.BoostEndpointConfig"
 
-_FORMATS_BLOCK = re.compile(
-    r"^\s{4}FORMATS\s*=\s*\(([\s\S]*?)\)\s*\n\s{4}class Meta:",
-    re.MULTILINE,
-)
-_STRING_LITERAL = re.compile(r'"([^"\\]*)"(?:\s*,|\s*$)', re.MULTILINE)
+
+def _parse_formatsconf_formats_ast(models_text: str) -> list[str]:
+    tree = ast.parse(models_text)
+    for node in tree.body:
+        if isinstance(node, ast.ClassDef) and node.name == "FormatsConf":
+            return _formats_assignment_to_strings(node.body)
+    msg = "Class FormatsConf not found in weblate formats models source"
+    raise RuntimeError(msg)
+
+
+def _formats_assignment_to_strings(class_body: list[ast.stmt]) -> list[str]:
+    for node in class_body:
+        if not isinstance(node, ast.Assign):
+            continue
+        for target in node.targets:
+            if isinstance(target, ast.Name) and target.id == "FORMATS":
+                return _string_tuple_or_list(node.value)
+    msg = "FORMATS assignment not found on FormatsConf"
+    raise RuntimeError(msg)
+
+
+def _string_tuple_or_list(node: ast.expr) -> list[str]:
+    if isinstance(node, (ast.Tuple, ast.List)):
+        out: list[str] = []
+        for elt in node.elts:
+            if isinstance(elt, ast.Constant) and isinstance(elt.value, str):
+                out.append(elt.value)
+            else:
+                msg = f"Unexpected literal in FormatsConf.FORMATS: {ast.dump(elt)}"
+                raise RuntimeError(msg)
+        return out
+    msg = f"Unexpected FormatsConf.FORMATS value: {ast.dump(node)}"
+    raise RuntimeError(msg)
 
 
 def weblate_formats_with_quickbook() -> tuple[str, ...]:
@@ -53,14 +80,13 @@ def weblate_formats_with_quickbook() -> tuple[str, ...]:
     """
     models_py = Path(weblate.formats.__file__).resolve().parent / "models.py"
     src = models_py.read_text(encoding="utf-8")
-    m = _FORMATS_BLOCK.search(src)
-    if not m:
+    try:
+        core = tuple(_parse_formatsconf_formats_ast(src))
+    except RuntimeError:
+        raise
+    except (SyntaxError, ValueError) as exc:
         msg = f"boost_weblate: could not parse FormatsConf.FORMATS from {models_py}"
-        raise RuntimeError(msg)
-    body = m.group(1)
-    core = tuple(
-        p for p in _STRING_LITERAL.findall(body) if p.startswith("weblate.formats.")
-    )
+        raise RuntimeError(msg) from exc
     if not core:
         msg = f"boost_weblate: no format paths parsed from {models_py}"
         raise RuntimeError(msg)

tests/test_settings_override.py

@@ -6,7 +6,6 @@
 
 from __future__ import annotations
 
-import ast
 import importlib.util
 from pathlib import Path
 
@@ -22,42 +21,11 @@ def _load_weblate_formats_models_source() -> str:
     return path.read_text(encoding="utf-8")
 
 
-def _parse_formatsconf_formats_ast(models_text: str) -> list[str]:
-    tree = ast.parse(models_text)
-    for node in tree.body:
-        if isinstance(node, ast.ClassDef) and node.name == "FormatsConf":
-            return _formats_assignment_to_strings(node.body)
-    msg = "Class FormatsConf not found in weblate formats models source"
-    raise AssertionError(msg)
-
-
-def _formats_assignment_to_strings(class_body: list[ast.stmt]) -> list[str]:
-    for node in class_body:
-        if not isinstance(node, ast.Assign):
-            continue
-        for target in node.targets:
-            if isinstance(target, ast.Name) and target.id == "FORMATS":
-                return _string_tuple_or_list(node.value)
-    msg = "FORMATS assignment not found on FormatsConf"
-    raise AssertionError(msg)
-
-
-def _string_tuple_or_list(node: ast.expr) -> list[str]:
-    if isinstance(node, (ast.Tuple, ast.List)):
-        out: list[str] = []
-        for elt in node.elts:
-            if isinstance(elt, ast.Constant) and isinstance(elt.value, str):
-                out.append(elt.value)
-            else:
-                msg = f"Unexpected literal in FormatsConf.FORMATS: {ast.dump(elt)}"
-                raise AssertionError(msg)
-        return out
-    msg = f"Unexpected FormatsConf.FORMATS value: {ast.dump(node)}"
-    raise AssertionError(msg)
-
-
 def test_settings_override_formats_match_ast_parse_of_upstream() -> None:
-    from boost_weblate.settings_override import weblate_formats_with_quickbook
+    from boost_weblate.settings_override import (
+        _parse_formatsconf_formats_ast,
+        weblate_formats_with_quickbook,
+    )
 
     stock = _parse_formatsconf_formats_ast(_load_weblate_formats_models_source())
     got = weblate_formats_with_quickbook()