Skip to content

Let Rust own shortcode expansion: fix escaped shortcodes crashing pandoc, and expand them in footnote definitions - #789

Merged
gordonwoodhull merged 2 commits into
mainfrom
bugfix/bd-2uva9urq-disable-lua-shortcodes
Oct 6, 2026
Merged

gordonwoodhull merged 2 commits into
mainfrom
bugfix/bd-2uva9urq-disable-lua-shortcodes

Conversation

@gordonwoodhull

Copy link
Copy Markdown
Member

Summary

Escaped shortcodes such as `{{{< brand >}}}` crashed pandoc with exit 83 whenever the page was rendered through the pandoc path. Rust unescapes {{{< x >}}} once, and the vendored Lua shortcodes pass then expanded the result as a live shortcode. For handlers that Rust doesn't know (brand), the Lua side hit a nil brandCommand at shortcodes-handlers.lua:111.

The fix makes Rust the only place shortcodes are expanded. Two commits:

  1. Expand shortcodes in footnote definitions and lst-cap (bd-xjg7vl6c). shortcode_resolve skipped NoteDefinitionPara, NoteDefinitionFencedBlock and the lst-cap attribute of listing code blocks, which the Lua pass had been covering. Quarto 1 expands all three, so Rust now does too. This has to land first. Without it, turning Lua off would leave [^1]: Note {{< meta author >}}. unexpanded.
  2. Disable the vendored Lua shortcodes pass (bd-2uva9urq). The pre-shortcodes-filter entries in main.lua and crossref/crossref.lua now require q2_lua_shortcodes_disabled, a flag that is never set, so the runner skips them. This is a flag-gate rather than a removal. shortcodes_filter() must still be called because foldcode.lua's process_shortcodes uses _shortcodes_filter. Both edits carry a QUARTO2-PATCH comment, and the README's "Ours vs. pinned" section documents them.

Tests

  • shortcode_all_contexts: a parameterized test over every context where a shortcode can appear. Contexts that Quarto 1 also leaves literal are pinned as literal.
  • shortcode_pandoc_escapes: renders to native and checks that escaped brand shortcodes stay literal in a code span, a code span with arguments and a fenced block. A control confirms that a live {{< brand >}} still only warns. Three of the four cases failed with exit 83 before the flag-gate.
  • pandoc_filters::test_shim_group_shape_and_patch_markers pinned exactly two QUARTO2-PATCH markers in main.lua. It now ignores the unrelated shortcodes marker.

Not in this PR

  • Brand support. A live {{< brand >}} still doesn't render. That stays with bd-qnylgu69.
  • pandoc-wasm. pandoc-request.golden.json stores a hash of the embedded filter tree (share_tree_version), so editing the Lua changes it. That file only exists on feature/pandoc-wasm. Whichever branch lands second needs one Q2_REGENERATE_GOLDEN=1 regeneration. The pandoc recordings are self-contained and don't need re-recording.

Test plan

  • cargo nextest run --workspace (running locally; result will be posted as a comment)
  • CI green

Rust shortcode_resolve skipped NoteDefinitionPara, NoteDefinitionFencedBlock
and the lst-cap attribute of listing code blocks; Quarto 1 expands all three.
Add a parameterized all-contexts test; contexts Q1 also leaves literal are
pinned as such.
…on (bd-2uva9urq)

Escaped shortcodes in code spans/blocks were unescaped once by Rust and then
expanded as live by the Lua pass, crashing pandoc (exit 83) on handlers Rust
does not know (brand). Flag-gate the pre-shortcodes-filter entry in main.lua
and crossref.lua off (q2_lua_shortcodes_disabled); shortcodes_filter() is
still called because foldcode.lua's process_shortcodes needs
_shortcodes_filter.

- New shortcode_pandoc_escapes integration test (native writer); it failed
  with exit 83 before the flag-gate.
- pandoc_filters marker test now ignores the unrelated shortcodes
  QUARTO2-PATCH marker.
- README 'Ours vs. pinned' entries for main.lua and crossref/crossref.lua.
@posit-snyk-bot

posit-snyk-bot commented Oct 4, 2026 •

Copy link
Copy Markdown
Contributor

✅ Snyk checks have passed. No issues have been found so far.

Status Scan Engine Critical High Medium Low Total (0)
✅ Open Source Security 0 0 0 0 0 issues
✅ Licenses 0 0 0 0 0 issues

💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse.

@gordonwoodhull
gordonwoodhull merged commit 4753b6a into main Oct 6, 2026
11 checks passed
@gordonwoodhull
gordonwoodhull deleted the bugfix/bd-2uva9urq-disable-lua-shortcodes branch October 6, 2026 00:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants