✨ NEW: Add section_ref syntax extension (§1.1 heading references)

.pre-commit-config.yaml

@@ -36,7 +36,7 @@ repos:
       - types-urllib3
       - sphinx~=8.2
       - markdown-it-py~=4.2
-      - mdit-py-plugins~=0.6.0
+      - mdit-py-plugins~=0.7.0
       - sphinx-syntax-example~=0.2.0
       files: >
         (?x)^(

docs/configuration.md

@@ -179,6 +179,9 @@ linkify
 replacements
 : Automatically convert some common typographic texts
 
+section_ref
+: Resolve section-sign references such as `§1.1` to internal links, [see here](syntax/section-ref) for details
+
 smartquotes
 : Automatically convert standard quotations to their opening/closing variants
 

docs/syntax/optional.md

@@ -40,6 +40,7 @@ myst_enable_extensions = [
     "html_image",
     "linkify",
     "replacements",
+    "section_ref",
     "smartquotes",
     "strikethrough",
     "substitution",
@@ -325,6 +326,128 @@ Key differences:
 - **`gfm_autolink`** only matches `www.`-prefixed URLs, `http(s)://`/`mailto:`/`xmpp:` URLs, and email addresses. It has no extra dependency and follows GFM URL boundary rules exactly.
 :::
 
+(syntax/section-ref)=
+## Section references
+
+```{versionadded} 5.2.0
+```
+
+Adding `"section_ref"` to `myst_enable_extensions` (in the {{ confpy }}) will recognise section-sign references such as `§1`, `§1.1` and `§2.3.4`, and turn each into an internal link to the correspondingly numbered heading in the current document.
+
+For example, with the following document:
+
+```md
+# Title
+
+See §1 and §1.1, and also §2.
+
+## Section One
+
+### Sub One One
+
+## Section Two
+```
+
+`§1` links to *Section One*, `§1.1` to *Sub One One*, and `§2` to *Section Two*, each link also carrying the target heading's title as a hover tooltip in Sphinx HTML output.
+
+The numbering is **document-local** and purely **structural**, computed from the heading hierarchy as written in the source:
+
+- If the document has a single top-level heading (the common `# Title` layout), its subsections are numbered `§1`, `§2`, … in order.
+- Otherwise the top-level headings themselves are numbered `§1`, `§2`, ….
+- Deeper levels are appended with dots, so the second subsection of the first section is `§1.2`, and so on.
+
+The numbering is the same in Sphinx and in single-page docutils output.
+docutils may promote a lone top-level heading to the document title, and a heading below it to the subtitle,
+but references are resolved before that, against the headings as written, so `§1` can link to a heading that docutils displays as the subtitle.
+
+A `§` must be immediately followed by digits (e.g. `§1.2`); no spaces are allowed, and a trailing dot is not consumed (so `see §1.` references `§1`).
+A `§` number directly followed by an ASCII letter, such as `§1a`, is not a reference.
+
+References that appear **inside a section heading or inside link text** are intentionally left as text marked with the `section-ref` class, rather than converted to links, since a link there would nest inside another link (a heading reference is also copied into navigation and table-of-contents entries).
+References in other titles, such as the title of an admonition or a table, are resolved as usual.
+
+:::{important}
+§-numbers are **not** the section numbers that Sphinx displays.
+In a `toctree` with the `:numbered:` option, Sphinx numbers sections across the documents it lists, so *Section One* above could be displayed as "2.1.", while `§1` still targets it, as the first section of its own document.
+To cite the number that a `:numbered:` toctree displays, put a [`(label)=` target](syntax/targets) on the heading and use the [`{numref}` role](inv:sphinx:rst:role#numref), which by default prints "Section 2.1".
+:::
+
+:::{caution}
+With the extension enabled, every `§` followed by a number is read as a reference into the **current** document:
+
+- A legal citation such as `§242 BGB` gives a `myst.section_ref` warning, or links to section 242 if the document has one.
+- A reference to another specification, such as `RFC 9110 §8.3`, links to section 8.3 *of the current document* if it has one, with no warning.
+
+To write a `§` that is not a reference, use the `§` entity (e.g. `§242 BGB`).
+A backslash before the `§` (`\§242`) also stops the reference, but stays in the output, since `§` is not ASCII punctuation.
+
+The warnings can be silenced with [`suppress_warnings`](sphinx/config-options), e.g. `suppress_warnings = ["myst.section_ref"]`,
+but that also hides references that are broken.
+:::
+
+:::{note}
+References resolve to positions **in the current document only**.
+A reference written in a file that is pulled in with the `{include}` directive resolves against the numbering of the document that includes it.
+
+Numbers are positional: inserting or moving a heading changes what every later reference points to, without a warning.
+Use a [`(label)=` target](syntax/targets) for references that must survive restructuring.
+
+If a reference cannot be resolved to a numbered heading (for example `§9.9` in a document with fewer sections), the reference is left as text marked with the `section-ref` class, and a `myst.section_ref` warning is emitted.
+
+Cross-document section references may be supported in a future release.
+:::
+
+### Numbering rules
+
+These rules say exactly which section heading a reference points to.
+Sphinx and docutils follow them in the same way (except that docutils has no `{only}` or `ifconfig` directive, so it drops such blocks), and MyST's test suite pins them.
+
+**What is a reference**
+
+- A reference is `§` followed by one or more levels separated by dots, each level being 1 to 9 ASCII digits (`§1`, `§2.10`, `§01.02`).
+  The longest such run is taken, but a trailing dot is not part of it, so `§1.` is `§1` followed by `.`.
+- There is no reference, and the whole run stays as written, if the number is directly followed by an ASCII letter (`§1a`),
+  or if any level has more than nine digits, leading zeros included (`§0000000001`, `§1.1234567890.2`).
+  A non-ASCII letter does not stop a reference: `见§3章` contains `§3`.
+- References are recognised wherever inline text is parsed, but not in code spans, autolinks, raw HTML or math.
+- `\§` is not an escape: the backslash stays visible, and no reference is formed.
+  `\\§1` is a literal backslash followed by the reference `§1`.
+  To write a literal sign, use `§` or `§`.
+- The link text is the reference as written (`§01.02`), and its number is the value of each level (1.2).
+
+**Which headings count**
+
+- Every heading that starts a section counts, in the order of the source, including:
+  - the headings of a file pulled in with `{include}`, at the point of inclusion and at the level they are written at, shifted by the directive's `:heading-offset:`;
+  - the sections of an `{eval-rst}` block, where the document places them;
+  - the headings inside an `{only}` block, whatever its condition, so that the numbers are the same for every builder;
+  - any other section that a directive places in the document, such as one inside `.. ifconfig::` in an `{eval-rst}` block, under the section that contains it;
+  - a front-matter `title` that [`myst_title_to_header`](#sphinx/config-options) turns into a heading: it is the first heading of the document.
+- A Markdown heading inside the body of a directive, such as `{note}`, is rendered as a rubric, and does not count.
+  It is not a section heading, so a reference written in it resolves as usual.
+
+**How headings are numbered**
+
+- A heading is nested under the nearest heading before it that has a smaller level, so a skipped level adds no number:
+  under `# Title`, a `####` heading that directly follows `## A` is `§1.1`.
+- If the document has exactly one top-level heading, its child headings are `§1`, `§2`, …; otherwise the top-level headings are.
+- Numbers start at 1, so `§0` never resolves. The k-th heading at the top is `§k`, and the j-th child of `§p` is `§p.j`.
+- The numbers do not depend on `:numbered:` toctrees, `{sectnum}`, the builder, docutils' title and subtitle promotion, or any other document.
+
+**How references resolve**
+
+- `§1.2` links to the section heading numbered exactly 1.2; there is no fallback to `§1` or to the nearest heading.
+- The link goes to the section's primary anchor, so a `(label)=` target on the heading is used.
+  A reference to a heading in an `{only}` or `ifconfig` branch that the build excludes points to an anchor that is not in the output.
+- A reference in a section heading or in link text is never linked, and never warns.
+  A heading that docutils promotes to the subtitle is still a section heading, but the `:subtitle:` of a `{sidebar}` is not, so a reference there resolves.
+  Everywhere else (paragraphs, emphasis, lists, tables, figure captions, admonitions) it resolves.
+- In image alt text, the reference is kept as written, without a link.
+- A reference that does not resolve stays as text marked with the `section-ref` class, and gives one `myst.section_ref` warning.
+- A reference resolves against the document it is written in, when that document is read
+  (for an `{include}`d file, that is the document that includes it).
+  Content that is copied into another document, for example by an extension, is not covered.
+
 (syntax/substitutions)=
 
 ## Substitutions (with Jinja2)

myst_parser/config/main.py

@@ -42,6 +42,7 @@ def check_extensions(inst: "MdParserConfig", field: dc.Field, value: Any) -> Non
             "html_image",
             "linkify",
             "replacements",
+            "section_ref",
             "smartquotes",
             "strikethrough",
             "substitution",

myst_parser/mdit_to_docutils/base.py

@@ -508,10 +508,12 @@ def renderInlineAsText(self, tokens: list[SyntaxTreeNode]) -> str:  # noqa: N802
         result = ""
 
         for token in tokens or []:
-            if token.type in {"text", "text_special"}:
+            if token.type in {"text", "text_special", "section_ref"}:
                 # An escape or entity is a text_special token with no children.
                 # Its content is the character the alt text must keep
                 # (executablebooks/MyST-Parser#1210).
+                # A section reference is also a childless token,
+                # whose content is the reference as written (e.g. "§1.1").
                 result += token.content
             else:
                 result += self.renderInlineAsText(token.children or [])
@@ -1327,6 +1329,13 @@ def render_span(self, token: SyntaxTreeNode) -> None:
         with self.current_node_context(node, append=True):
             self.render_children(token)
 
+    def render_section_ref(self, token: SyntaxTreeNode) -> None:
+        """Render a section reference (§1.1), for later resolution by ``ResolveSectionRefs``."""
+        node = nodes.inline(token.content, token.content, classes=["section-ref"])
+        node["section_numbers"] = token.meta["numbers"]
+        self.add_line_and_source_path(node, token)
+        self.current_node.append(node)
+
     def render_front_matter(self, token: SyntaxTreeNode) -> None:
         """Pass document front matter data."""
         position = token_line(token, default=0)

myst_parser/mdit_to_docutils/transforms.py

@@ -381,3 +381,166 @@ def apply(self, **kwargs: t.Any) -> None:
                 refnode += nodes.inline(
                     "#" + target, "#" + target, classes=["std", "std-ref"]
                 )
+
+
+def _child_sections(node: nodes.Element) -> list[nodes.section]:
+    """Return the child sections of an element, in document order.
+
+    A section counts wherever the document tree places it, under its nearest
+    section ancestor: the search descends through any other container element,
+    such as a sphinx ``only`` node (whatever its condition, so the numbering
+    does not depend on the builder) or an ``ifconfig`` node, but not into
+    sections or text elements.
+    """
+    sections: list[nodes.section] = []
+    for child in node.children:
+        if isinstance(child, nodes.section):
+            sections.append(child)
+        elif isinstance(child, nodes.Element) and not isinstance(
+            child, nodes.TextElement
+        ):
+            sections.extend(_child_sections(child))
+    return sections
+
+
+def _number_sections(
+    sections: list[nodes.section],
+    prefix: tuple[int, ...],
+    number_map: dict[tuple[int, ...], nodes.section],
+) -> None:
+    """Recursively assign structural numbers to sections, populating ``number_map``.
+
+    The i-th (1-based) section at a level is numbered ``prefix + (i,)``, and the
+    numbering recurses into each section's child sections.
+    """
+    for i, section in enumerate(sections, start=1):
+        number = prefix + (i,)
+        number_map[number] = section
+        _number_sections(_child_sections(section), number, number_map)
+
+
+def _section_number_map(
+    document: nodes.document,
+) -> dict[tuple[int, ...], nodes.section]:
+    """Map structural section numbers to their sections.
+
+    The numbering is that of the document's heading hierarchy as written:
+    when there is a single top-level heading (the common ``# Title`` layout),
+    its child sections are numbered ``1``, ``2``, ...; otherwise the
+    top-level sections themselves are.
+    """
+    number_map: dict[tuple[int, ...], nodes.section] = {}
+    roots = _child_sections(document)
+    if len(roots) == 1:
+        _number_sections(_child_sections(roots[0]), (), number_map)
+    else:
+        _number_sections(roots, (), number_map)
+    return number_map
+
+
+def _leave_section_ref_inert(node: nodes.Element) -> bool:
+    """Whether a section-reference marker (see ``render_section_ref``) should be left as inert styled text.
+
+    A marker is left untouched (no link, no warning) when it sits inside:
+
+    - a ``reference`` or ``pending_xref`` — converting it would nest an ``<a>``
+      inside another ``<a>``, which is invalid HTML; or
+    - a section heading — a reference there would be copied into navigation
+      and toc entries (sphinx toctree entry links, docutils contents entries),
+      again nesting ``<a>`` in those navs.
+
+    Other titles (admonition, table, topic or sidebar titles) are not copied
+    into navigation, so references in them are resolved as usual.
+    """
+    parent = node.parent
+    while parent is not None:
+        if (
+            isinstance(parent, nodes.reference)
+            or parent.tagname == "pending_xref"
+            or (
+                isinstance(parent, nodes.title)
+                and isinstance(parent.parent, nodes.section)
+            )
+        ):
+            return True
+        parent = parent.parent
+    return False
+
+
+class ResolveSectionRefs(Transform):
+    """Resolve ``§1.1`` section references to the target section's anchor.
+
+    Numbering is document-local and purely structural (independent of any
+    ``:numbered:`` toctree), see ``_section_number_map``.
+
+    The transform runs on the headings as written, so the same references
+    resolve identically in docutils and sphinx: after the id transforms, so
+    each section's first id is final, and before docutils' doctitle and
+    section-subtitle promotion, which sphinx never applies.
+    A link to a heading that docutils later promotes to the subtitle still
+    lands, because the promotion copies the section's ids onto the subtitle.
+    """
+
+    # strictly after docutils' PropagateTargets (260), sphinx's SortIds (261)
+    # and PrioritiseExplicitIds (262), which settle each section's first id
+    # (AddSlugIds (700) only appends), and strictly before docutils' DocTitle
+    # (320) and SectSubTitle (350), which reshape the section tree;
+    # this is also before sphinx's DoctreeReadEvent (880), so anything that
+    # reads the body at doctree-read sees the resolved references
+    default_priority = 300
+
+    def apply(self, **kwargs: t.Any) -> None:
+        """Apply the transform."""
+        # gather the section-reference markers emitted by ``render_section_ref``;
+        # these can only exist when the ``section_ref`` extension was enabled at
+        # parse time, so their presence self-gates the transform
+        markers = [
+            node
+            for node in findall(self.document)(nodes.inline)
+            if "section_numbers" in node
+        ]
+        if not markers:
+            return
+
+        number_map = _section_number_map(self.document)
+
+        for node in markers:
+            # a reference nested inside a link or heading would produce invalid
+            # nested anchors, so such markers are left as inert styled text
+            if _leave_section_ref_inert(node):
+                continue
+            content = node.astext()
+            number = tuple(node["section_numbers"])
+            target = number_map.get(number)
+            if target is not None and target["ids"]:
+                ref = nodes.reference(
+                    "",
+                    "",
+                    internal=True,
+                    refid=target["ids"][0],
+                    classes=["section-ref"],
+                    section_numbers=node["section_numbers"],
+                )
+                ref += node.children
+                # add the section title as a hover tooltip (``title="..."`` in
+                # sphinx HTML)
+                title_node = next(
+                    (
+                        child
+                        for child in target.children
+                        if isinstance(child, nodes.title)
+                    ),
+                    None,
+                )
+                if title_node is not None:
+                    ref["reftitle"] = clean_astext(title_node)
+                node.parent.replace(node, ref)
+            else:
+                # ``node=`` (not ``line=``) so the location is the file the
+                # reference was written in, which differs inside an ``{include}``
+                create_warning(
+                    self.document,
+                    f"Section reference target not found in this document: {content!r}",
+                    MystWarnings.SECTION_REF,
+                    node=node,
+                )

myst_parser/parsers/docutils_.py

@@ -28,6 +28,7 @@
     CollectFootnotes,
     PrioritiseExplicitIds,
     ResolveAnchorIds,
+    ResolveSectionRefs,
     SortFootnotes,
     UnreferencedFootnotesDetector,
 )
@@ -260,6 +261,7 @@ def get_transforms(self):
             AddSlugIds,
             PrioritiseExplicitIds,
             ResolveAnchorIds,
+            ResolveSectionRefs,
         ]
 
     def parse(self, inputstring: str, document: nodes.document) -> None:

myst_parser/parsers/mdit.py

@@ -20,6 +20,7 @@
 from mdit_py_plugins.gfm_autolink import gfm_autolink_plugin
 from mdit_py_plugins.myst_blocks import myst_block_plugin
 from mdit_py_plugins.myst_role import myst_role_plugin
+from mdit_py_plugins.section_ref import section_ref_plugin
 from mdit_py_plugins.substitution import substitution_plugin
 from mdit_py_plugins.wordcount import wordcount_plugin
 
@@ -84,6 +85,8 @@ def create_md_parser(
         linkify_enabled = True
     if "gfm_autolink" in config.enable_extensions:
         md.use(gfm_autolink_plugin)
+    if "section_ref" in config.enable_extensions:
+        md.use(section_ref_plugin)
     if "strikethrough" in config.enable_extensions:
         md.enable("strikethrough")
         md.options["strikethrough_single_tilde"] = config.strikethrough_single_tilde