Skip to content

parsing.ts uses YAML.parse() which fails on multi-document YAML (frontmatter) #159

Description

@cioinside

Summary

packages/ragmir-core/src/parsing.ts (compiled to dist/parsing.js:86-88) uses YAML.parse() to handle .yaml/.yml files. This rejects any file containing multiple YAML documents — including the widely-used pattern of YAML frontmatter (Hugo, Jekyll, Eleventy, Hexo, Pelican, etc.).

Repro

---
project_context: "test"
quality_score: 5
---
# Title

This is the body of the YAML document.
import YAML from "yaml";
import fs from "node:fs";
const text = fs.readFileSync("test.yaml", "utf8");
YAML.parse(text);
// throws:
//   Source contains multiple documents; please use YAML.parseAllDocuments()
//   at line 5, column 1

Run rgr ingest against a project that contains such a file → the file is rejected at parse time and is not indexed. There is no warning to the user beyond the bare error message.

Expected

Multi-document YAML files parse successfully. All documents are concatenated into indexable text.

Root cause

packages/ragmir-core/src/parsing.ts (compiled dist/parsing.js):

case ".yaml":
case ".yml": {
    text = YAML.stringify(YAML.parse(await readFile(file.absolutePath, "utf8")));
    break;
}

YAML.parse() from the yaml npm package (v2.9.0) refuses multi-document sources. The package exposes YAML.parseAllDocuments() specifically for this case.

Proposed fix

 case ".yaml":
 case ".yml": {
-    text = YAML.stringify(YAML.parse(await readFile(file.absolutePath, "utf8")));
+    const docs = YAML.parseAllDocuments(await readFile(file.absolutePath, "utf8"));
+    text = docs.map((d) => YAML.stringify(d)).join("\n");
     break;
 }

Concatenate all documents as indexable text. Empty / frontmatter-only documents stringify to null; the downstream pipeline already treats null text as a no-op.

Environment

  • @jcode.labs/ragmir 4.4.0
  • yaml 2.9.0 (matches package.json dependency)
  • Node 22.23.1

Tested workaround

Applied a one-line patch to the installed dist/parsing.js:

-    text = YAML.stringify(YAML.parse(await readFile(file.absolutePath, "utf8")));
+    const docs = YAML.parseAllDocuments(await readFile(file.absolutePath, "utf8"));
+    text = docs.map((d) => YAML.stringify(d)).join("\n");

After the patch, rgr ingest processes multi-document YAML files cleanly.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions