Skip to content

Commit fffc0c1

Browse files
sync: fence-aware source expansion + the every-page single-h1 pin
Ported from dash-documentation-boilerplate 1.6.11 (30075d0). _expand_source_directives was a whole-document regex sub, so a `.. source::` written INSIDE a fence to teach the directive got expanded like a real one — the injected ```python fence closes the open one early, and from there the inlined file renders as markdown with every `# comment` line becoming an <h1>. Machine lane only; markdown2dash parses fences properly, so nothing looked wrong to a human. Now a line walker that tracks the open marker (``` and ~~~) and expands only at fence depth zero. Prevention here: this fork's one directive (docs/reference_geo/geo.md, inlining lib/policy_store.py) is at top level and no page teaches the syntax inside a fence. The pin is what catches the day one does. The sweep ran over all 11 non-admin pages and found no drift — one h1 each, distinct footer llms.txt links, "/" carrying the root link once. It caught real content drift on leaflet and muicharts, so that is a measurement rather than an assumption. Adapted: the template's inline /admin skip is dead code here, the `pages` fixture already drops it. Also refreshed the prerender-lane test's floor wording — it still said 2.6.1, which has been >=2.7.1 since Phase B. 602 passed / 1 skipped (flask), 599 passed / 4 skipped (fastapi). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
1 parent b6df84e commit fffc0c1

3 files changed

Lines changed: 144 additions & 4 deletions

File tree

DEVELOPMENT-LOG.md

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -483,3 +483,45 @@ dash-documentation-boilerplate `ab22fd7` (a commit this fork's first push
483483
prompted). Security updates are unaffected — separate channel. PRs #1 (base
484484
image 3.11.8 → 3.14.7) and #2 (actions group) are different ecosystems, are
485485
real decisions, and stay open.
486+
487+
---
488+
489+
## 11. Template sync — fence-aware source expansion (2026-08-23)
490+
491+
Ported from `dash-documentation-boilerplate` 1.6.11 (`30075d0`): the
492+
fence-aware `_expand_source_directives` in `pages/markdown.py`, the
493+
every-page single-h1 / deduped-footer pin, and the fence unit test.
494+
495+
**The upstream bug.** `_expand_source_directives` was a plain regex `sub`
496+
over the whole document, so a `.. source::` written *inside* a fenced block
497+
to TEACH the directive was expanded like a real one. The expansion injects
498+
its own ```` ```python ```` fence, which closes the already-open fence early
499+
— everything after it renders as markdown, and every `# comment` line in the
500+
inlined file becomes an `<h1>`. Five h1s on the template's tutorial page,
501+
machine lane only: markdown2dash parses fences properly, so the browser lane
502+
was always correct and nothing looked wrong to a human. The expander is now
503+
a line walker that tracks the open fence marker (``` and ~~~ both) and
504+
expands only at fence depth zero.
505+
506+
**Prevention here, not a fix.** This fork has exactly one `.. source::`
507+
`docs/reference_geo/geo.md:117`, inlining `lib/policy_store.py`, at top
508+
level. No page here teaches the directive inside a fence today. The port is
509+
for the day one does; the pin is what would catch it.
510+
511+
**The sweep found no drift.** It ran over all 11 non-admin pages — every one
512+
serves exactly one `<h1>` to a generic client and a footer whose llms.txt
513+
links are distinct, with `/` carrying the root link once. The same sweep
514+
caught real content drift on two other forks (leaflet, muicharts), so the
515+
clean result here is a measurement, not an assumption.
516+
517+
One adaptation: the template's pin skips `/admin/*` inline. This repo's
518+
`pages` fixture already drops it — the control board fails closed to
519+
anonymous renders and `tests/test_control_board.py` owns its assertions — so
520+
the guard would be dead code, and the docstring says where the exclusion
521+
actually lives.
522+
523+
Also refreshed the prerender-lane test's floor wording: it still named 2.6.1
524+
as "the floor", which has been >=2.7.1 since Phase B.
525+
526+
**602 passed / 1 skipped (flask), 599 passed / 4 skipped (fastapi)**,
527+
flake8 clean.

pages/markdown.py

Lines changed: 27 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -90,9 +90,21 @@ def _expand_source_directives(markdown_content: str) -> str:
9090
`/<page>/llms.txt`. Replacing the directive with the real file content
9191
is what makes the LLM output self-contained for the "paste into a chat
9292
window" audience.
93+
94+
FENCE-AWARE, and it has to be: a directive INSIDE a fenced code block
95+
is documentation showing the syntax, not a directive. Expanding it
96+
injects a ```python fence inside the already-open fence, which CLOSES
97+
it early — from there the inlined file renders as markdown, every
98+
`# comment` line becomes an <h1>, and the machine lane of the page
99+
serves broken structure. Found upstream 2026-08-23 by the single-h1
100+
pin (template 1.6.11) on the boilerplate's docs/example and
101+
docs/directives, which teach `.. source::` inside ```markdown fences;
102+
no page here teaches it that way today, so this is prevention — but
103+
the failure is silent on the browser lane (markdown2dash parses
104+
fences properly), which is exactly why it went unnoticed there.
93105
"""
94-
def replace(match: re.Match) -> str:
95-
file_path = match.group(1).strip()
106+
def expansion(directive_line: str) -> str:
107+
file_path = _SOURCE_DIRECTIVE.match(directive_line).group(1).strip()
96108
try:
97109
full = Path(file_path)
98110
content = full.read_text()
@@ -105,7 +117,19 @@ def replace(match: re.Match) -> str:
105117
except Exception as exc:
106118
return f'\n<!-- Error reading {file_path}: {exc} -->\n'
107119

108-
return _SOURCE_DIRECTIVE.sub(replace, markdown_content)
120+
out: List[str] = []
121+
fence = None # the marker that opened the block we are inside, if any
122+
for line in markdown_content.split('\n'):
123+
head = line.lstrip()[:3]
124+
if fence is None and head in ('```', '~~~'):
125+
fence = head
126+
elif fence is not None and head == fence:
127+
fence = None
128+
elif fence is None and _SOURCE_DIRECTIVE.match(line):
129+
out.append(expansion(line))
130+
continue
131+
out.append(line)
132+
return '\n'.join(out)
109133

110134

111135
def _build_llms_doc(name: str, description: str, expanded_markdown: str, path: str) -> str:

tests/test_pages.py

Lines changed: 75 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -136,10 +136,84 @@ def test_prerender_rides_the_generic_lane_not_a_ua_gate(client):
136136
assert "hidden" not in div.group(0), (
137137
f"{path}: the prerender div carries `hidden` again — "
138138
"visibility-respecting consumers are back to reading "
139-
"'Loading...'; the dimll floor is >=2.6.1 for exactly this"
139+
"'Loading...'; the floor first moved (to 2.6.1) for exactly "
140+
"this, and sits at >=2.7.1 now"
140141
)
141142
assert 'data-dimll-prerender="1">document.getElementById' in html, (
142143
f"{path}: the marked synchronous hide script is missing — "
143144
"JS browsers would flash the prose before React mounts"
144145
)
145146
assert "<main>" in html, f"{path}: prerender block carries no <main> prose"
147+
148+
149+
def test_prerender_single_h1_and_deduped_footer_llms_links(client, page_paths):
150+
"""What the >=2.7.1 floor buys, pinned from the app's side, EVERY page.
151+
152+
Below dimll 2.7.0 every page served TWO h1s to a generic client — the
153+
injected prerender header plus the doc body's own markdown H1, a
154+
duplicate-H1 page in every crawler's eyes (2026-08-22 SEO-audit
155+
finding) — and the home footer printed its /llms.txt link twice (on
156+
"/" the per-page link equals the root's; subpages legitimately carry
157+
both, DISTINCT). The sweep also catches app-side H1 pollution: on the
158+
template its first run found docs/example's machine lane serving FIVE
159+
h1s because _expand_source_directives expanded a `.. source::` example
160+
inside a ```markdown teaching fence (fixed fence-aware, 1.6.11 —
161+
ported here, with tests below).
162+
163+
HTML comments are stripped before counting: templates/index.html
164+
legitimately SAYS "<h1>" inside the comment explaining its noscript
165+
block. /admin/* never reaches this sweep — the `pages` fixture drops
166+
it, because the control board fails closed to anonymous renders and
167+
tests/test_control_board.py owns its assertions.
168+
"""
169+
for path in page_paths:
170+
html = client.get(path).text # default UA — the universal lane
171+
stripped = re.sub(r"<!--.*?-->", "", html, flags=re.S)
172+
173+
h1s = re.findall(r"<h1[\s>]", stripped)
174+
assert len(h1s) == 1, (
175+
f"{path}: {len(h1s)} h1 elements in the generic-lane document — "
176+
"either the pre-2.7.0 prerender-header duplicate or app-side "
177+
"markdown leaking headings (the fence-expansion class)"
178+
)
179+
180+
footer = re.search(r"<footer.*?</footer>", stripped, re.S)
181+
assert footer, f"{path}: no prerender footer in the generic-lane document"
182+
llms_links = re.findall(r'href="([^"]*llms\.txt)"', footer.group(0))
183+
assert len(llms_links) == len(set(llms_links)), (
184+
f"{path}: duplicate llms.txt links in the prerender footer "
185+
f"({llms_links}) — 2.7.0 dedups the per-page link when it "
186+
"equals the root"
187+
)
188+
if path == "/":
189+
assert llms_links == ["/llms.txt"], (
190+
f"home footer llms links {llms_links} — expected exactly the "
191+
"root link once"
192+
)
193+
194+
195+
def test_source_expansion_is_fence_aware(app):
196+
"""A `.. source::` inside a fenced block is documentation, not a directive.
197+
198+
The template's docs/example and docs/directives TEACH the directive
199+
inside ```markdown fences. Expanding those injects a ```python fence
200+
inside the already-open fence, which closes it early — from there the
201+
inlined file renders as markdown on the machine lane and every
202+
`# comment` line becomes an <h1> (the five-h1 finding, 2026-08-23).
203+
No page here teaches it that way today, so this pin guards the day
204+
one does. The app fixture is requested only so pages/markdown.py is
205+
already imported with the repo root as CWD.
206+
"""
207+
import sys
208+
209+
expand = sys.modules["pages.markdown"]._expand_source_directives
210+
211+
expanded = expand(".. source::requirements.txt")
212+
assert "# File: requirements.txt" in expanded, "real directive not expanded"
213+
assert "```" in expanded, "expansion lost its fence"
214+
215+
taught = "```markdown\n.. source::requirements.txt\n```"
216+
assert expand(taught) == taught, "a fenced example was expanded"
217+
218+
tilde = "~~~\n.. source::requirements.txt\n~~~"
219+
assert expand(tilde) == tilde, "a tilde-fenced example was expanded"

0 commit comments

Comments
 (0)