Router seeding from the server: routers module, announceRoutes, drop filesystem-routing

.changeset/http-transport-cli-redirects.md

@@ -1,5 +1,6 @@
 ---
 "prerender-crawler": minor
+"@solidjs/prerender": minor
 ---
 
 Prerender anything over HTTP, from the command line, with redirects the host can serve.
@@ -18,6 +19,10 @@ Prerender anything over HTTP, from the command line, with redirects the host can
 
 **Query-string pages.** `keepQuery: true` renders `/posts?page=2` apart from `/posts` (parameters sorted for dedupe), following its links and capturing its data, but writes it only when its seed entry names a `filename` — a static host serves a path the same for every query. Off by default; CLI `--keep-query`.
 
+**Router seeding, from the server.** New `prerender-crawler/routers` (no Node imports — for application server code): `tanstackRouterPages(router)` and `solidRouterPages(router | routes, { base? })` list a router's static pages from the instance the app built for the request (leaves and indexes, no params or splats), and `announcePages(request, headers, paths)` puts them on the response's hint header when the request is the crawler's. `@solidjs/prerender` gains `announceRoutes(Router)` — one line in the app root, reading the ambient request event; a no-op in the browser. The Vite plugin, the CLI against a module, and the CLI against a running server all seed from the same header.
+
+**Removed:** `fileRoutePages`, `staticRoutePaths`, and the Vite plugin's `fileRoutes` option — build-time seeding that walked a `filesystem-routing` directory and re-derived its path rules. The server's own router is the source of truth for which pages exist, and the header works for crawls that never see the project's disk (the CLI against a running server). `filesystem-routing` is no longer a peer dependency.
+
 **Seeds are normalized like links.** A seed spelled `about/`, `/a#top`, or `/posts?page=2` now meets the crawled link to the same page in one queue entry. Previously a seed with a query was fetched verbatim and written to a literal `posts?page=2/` directory.
 
 **`RenderedPage.duration`.** Milliseconds from request start to body read, pacing excluded.

examples/ssg/src/App.tsx

@@ -1,10 +1,14 @@
 import { Title } from "@solidjs/meta";
+import { announceRoutes } from "@solidjs/prerender";
 import { Loading } from "solid-js";
 import { Router } from "./router";
 
 // The app root: the router and the site-wide layout. Pages live under
-// src/routes; the build crawls them into static HTML starting from "/".
+// src/routes; the build crawls them into static HTML starting from "/",
+// and the router announces its static pages to the crawl so one nothing
+// links to still builds. (A no-op for visitors and in the browser.)
 export default function App() {
+  announceRoutes(Router);
   return (
     <Router>
       {props => (

packages/crawler/README.md

@@ -56,22 +56,20 @@ export default defineConfig({
 });
 ```
 
-The plugin is build-only. It assumes three things about the app:
+The plugin is build-only. It assumes two things about the app:
 
 1. `vite build` produces a client output directory (the `client` environment's `outDir`, default `dist/client`).
 2. Some environment's output includes a module exporting a request handler — `handleRequest`, `fetch`, or `default.fetch`. Default: `server.js` in the `ssr` environment's `outDir`; override with `serverEntry`.
-3. Optionally, a [`filesystem-routing`](https://www.npmjs.com/package/filesystem-routing) route directory names the static pages.
 
-After the other environments build, it imports the server handler and crawls it in-process. Pages and integration-emitted files land in the client output.
+After the other environments build, it imports the server handler and crawls it in-process. Pages and integration-emitted files land in the client output. Which pages exist is the server's to say — see [Seeding from the router](#seeding-from-the-router).
 
 ### Options
 
 Everything from [`PrerenderOptions`](#engine-options) plus:
 
-| Option        | Default                  |                                                                                                                                                                                                                                                                                                                                                             |
-| ------------- | ------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| `serverEntry` | `<ssr outDir>/server.js` | Built module exporting the handler.                                                                                                                                                                                                                                                                                                                         |
-| `fileRoutes`  | `true`                   | Seed the crawl with the static pages of the project's `filesystem-routing` directory. `true` applies when the package and `src/routes` exist and is skipped silently otherwise; pass `{ dir, extensions }` to mirror a customized `fileRoutes()` (then a missing package is an error); `false` disables. Dynamic routes are still found by following links. |
+| Option        | Default                  |                                     |
+| ------------- | ------------------------ | ----------------------------------- |
+| `serverEntry` | `<ssr outDir>/server.js` | Built module exporting the handler. |
 
 ### `import.meta.env.PRERENDER_MODE`
 
@@ -108,6 +106,24 @@ result.skipped; // SkippedPage[]    — failures left out (failOnError: false)
 
 `httpTransport` sends the crawl's requests to the target's origin (path and query kept) and hands redirects back as the 3xx responses the server sent. Pass `{ headers }` for an auth token or `{ fetch }` for a custom implementation. `moduleTransport` imports a module exporting `handleRequest`, `fetch`, or `default.fetch` and calls it directly.
 
+### Seeding from the router
+
+The crawl finds pages by following links and by reading the hint header (`x-prerender`, comma-separated paths) off responses. A page nothing links to is invisible to the first; the second is how a server that knows its routes declares them — and the thing that knows the routes is the router the app built for the request. `prerender-crawler/routers` has the helpers, free of Node imports so application server code can use them:
+
+```ts
+import { announcePages, tanstackRouterPages, solidRouterPages } from "prerender-crawler/routers";
+
+// TanStack Router (any flavor — the helper reads the instance's `routesByPath`)
+announcePages(request, response.headers, tanstackRouterPages(router));
+
+// Solid Router (a `createRouter` instance, or a route-definition tree with `{ base }`)
+announcePages(request, response.headers, solidRouterPages(Router));
+```
+
+`announcePages` writes the header only when the request is the crawler's (it carries the hint header) — a visitor's response is untouched. Each `*Pages` helper returns the paths that address a static page: no parameters or splats, a leaf or an index (a layout with children but no index has no page of its own). Dynamic routes are still found by their links; only a render knows their values. The Vite plugin, the CLI against a module, and the CLI against a running server all send the hint header, so one line in the app seeds all three.
+
+[`@solidjs/prerender`](../solid) wraps this as `announceRoutes(Router)` for Solid apps, reading the request from the ambient request event.
+
 ### Redirects
 
 A path that answers 3xx is recorded (`result.redirects`, one record per hop — `/a → /b → /c` is two records, the way host rules spell it) and its same-origin target is crawled as a page in its own right, so the destination renders once at its own URL. The redirected path itself gets a **meta-refresh stub** pointing at the chain's final destination, so the old URL keeps working on hosts with no redirect support. A redirect to another spelling of the same page (`/posts → /posts/`) is followed in place, not recorded.
@@ -209,12 +225,12 @@ interface PrerenderContext {
 - `redirects(options?)`, `formatRedirectsFile(records, force?)` — the redirects integration and its `_redirects` formatter.
 - `sitemap(options)`, `indexable(page)`, `formatSitemap(entries)` — the sitemap integration and its parts.
 - `report(options?)` — the crawl report integration.
-- `fileRoutePages({ root, dir, extensions })` / `staticRoutePaths(entries)` — the static page paths of a `filesystem-routing` manifest, as a `pages` source.
+- `prerender-crawler/routers`: `announcePages(request, headers, paths)`, `tanstackRouterPages(router)`, `solidRouterPages(router | routes, { base? })`, `HINT_HEADER`.
 - `extractLinks(html, pageUrl, { keepQuery? })`, `normalizeLink(href, base, origin)`, `normalizeRoute(url)`, `normalizePath(pathname)`, `splitRoute(route)`, `outputFilename(path, autoSubfolderIndex)` — the crawl's own primitives.
 
 ## Requirements
 
-Node 20+. Vite 7 or 8 for the plugin (optional peer). `filesystem-routing` ≥ 0.2 for route seeding (optional peer).
+Node 20+. Vite 7 or 8 for the plugin (optional peer).
 
 ## License
 

packages/crawler/package.json

@@ -30,6 +30,10 @@
     "./vite": {
       "types": "./dist/vite.d.ts",
       "default": "./dist/vite.js"
+    },
+    "./routers": {
+      "types": "./dist/routers.d.ts",
+      "default": "./dist/routers.js"
     }
   },
   "bin": {
@@ -49,20 +53,19 @@
     "test:watch": "vitest"
   },
   "peerDependencies": {
-    "filesystem-routing": ">=0.2.0",
     "vite": "^7.0.0 || ^8.0.0"
   },
   "peerDependenciesMeta": {
-    "filesystem-routing": {
-      "optional": true
-    },
     "vite": {
       "optional": true
     }
   },
   "devDependencies": {
+    "@solidjs/router": "2.0.0-next.21",
+    "@solidjs/web": "2.0.0-rc.6",
+    "@tanstack/router-core": "^1.171.27",
     "@types/node": "^22.0.0",
-    "filesystem-routing": "0.2.1",
+    "solid-js": "2.0.0-rc.6",
     "typescript": "^5.8.0",
     "vite": "^8.0.0",
     "vitest": "^4.0.0"

packages/crawler/src/index.ts

@@ -6,6 +6,15 @@ export { outputFilename } from "./output.ts";
 export { formatRedirectsFile, redirects } from "./redirects.ts";
 export type { RedirectsIntegrationOptions } from "./redirects.ts";
 export { report } from "./report.ts";
+export { HINT_HEADER, announcePages, solidRouterPages, tanstackRouterPages } from "./routers.ts";
+export type {
+  AnnounceOptions,
+  SolidRouteLike,
+  SolidRouterLike,
+  SolidRouterPagesOptions,
+  TanStackRouteLike,
+  TanStackRouterLike
+} from "./routers.ts";
 export type {
   PrerenderReport,
   ReportIntegrationOptions,