From 49de010fbb49d34ccffaffa5b1642f52779d29ed Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Thomas=20M=C3=BCller?= <1005065+DeepDiver1975@users.noreply.github.com> Date: Mon, 15 Jun 2026 16:48:58 +0200 Subject: [PATCH 1/2] feat: replace Elasticsearch search with Pagefind MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Migrate documentation search from a self-hosted Elasticsearch index to Pagefind, a static search library that crawls the built HTML in public/ and writes a chunked index into public/pagefind/. This removes the need for an Elasticsearch server, local docker container, CORS workarounds, and the ELASTICSEARCH_* CI secrets. - package.json: drop @elastic/elasticsearch, cheerio, html-entities and lodash; add pagefind dev dependency and a `pagefind` npm script - site.yml / extensions.md: remove the generate-index.js Antora extension - delete ext-antora/generate-index.js and es-docker-compose.yml - ci.yml: drop ELASTICSEARCH_* env/secrets, run `npm run pagefind` after the Antora build so the deployed site always ships a fresh index - build-the-docs.md: rewrite the Search section for the Pagefind workflow Verified locally: `npm run antora` builds cleanly with no reference to the removed extension, and `npm run pagefind` indexes 1359 pages. Co-Authored-By: Claude Opus 4.8 Signed-off-by: Thomas Müller <1005065+DeepDiver1975@users.noreply.github.com> --- .github/workflows/ci.yml | 7 +- docs/build-the-docs.md | 78 +----- docs/extensions.md | 1 - es-docker-compose.yml | 9 - ext-antora/generate-index.js | 192 -------------- package-lock.json | 502 +++++++++-------------------------- package.json | 12 +- site.yml | 1 - 8 files changed, 141 insertions(+), 661 deletions(-) delete mode 100644 es-docker-compose.yml delete mode 100644 ext-antora/generate-index.js diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index e780f2959..b2d6f1873 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -32,14 +32,9 @@ jobs: run: npm install - name: Build docs - env: - UPDATE_SEARCH_INDEX: ${{ github.ref == 'refs/heads/master' }} - ELASTICSEARCH_NODE: ${{ secrets.ELASTICSEARCH_NODE }} - ELASTICSEARCH_INDEX: ${{ secrets.ELASTICSEARCH_INDEX }} - ELASTICSEARCH_READ_AUTH: ${{ secrets.ELASTICSEARCH_READ_AUTH }} - ELASTICSEARCH_WRITE_AUTH: ${{ secrets.ELASTICSEARCH_WRITE_AUTH }} run: | npm run antora + npm run pagefind echo 'doc.owncloud.com' > public/CNAME - name: Save cache diff --git a/docs/build-the-docs.md b/docs/build-the-docs.md index 72555dfa2..c947c8599 100644 --- a/docs/build-the-docs.md +++ b/docs/build-the-docs.md @@ -392,88 +392,30 @@ Though components get sourced locally when running `npm run antora-dev-xxx`, som The search bar is the component on the top right of the documentation where one can enter a term and get matches. If something is found, the matches are displayed as suggestions that can be clicked. -For "normal" changes, search is not necessary and may only complicate building commands and delay building times. To use the search functionality during production or development, see [Prepared npm Commands](#prepared-npm-commands) for details, some prerequisites apply. This is not only true for changes in the documentation, but also for UI changes. +Search is powered by [Pagefind](https://pagefind.app), a static search library. There is no server, container, or secret to set up: Pagefind crawls the already-built HTML in `public/` and writes a static, chunked index into `public/pagefind/`. Because it runs over the build output, the index automatically covers every component and version included in that build. -Note that ownCloud currently uses Elasticsearch version 7.x. All internal scripts and builds are therefore aligned to it and *must not* be changed, though you can use any latest minor/patch release. +Follow this procedure to build and use search locally: -Follow this procedure to show and use search and populate an index: - -1. Create an `es-docker-compose.yml` file with the following content. Note that no security or passwords is needed to be set up as it is only used locally: - - ``` - services: - elasticsearch: - image: elasticsearch:7.17.15 - ports: - - 9200:9200 - - 9300:9300 - environment: - - discovery.type=single-node - - xpack.security.enabled=false - ``` - -2. Start the container with the `up -d` command, use `down` to stop it. - - ``` - docker compose -f es-docker-compose.yml up -d - ``` - -3. To avoid a CORS Policy error, the browser must be prepared to allow access to the local Elasticsearch container. If this is not prepared, no search is possible and the browser console will return the following error: +1. Build the site as usual with any `npm run antora-xxx` command, for example: ``` - Access to XMLHttpRequest at 'http://localhost:9200/' from origin 'http://localhost:8080' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource. - ``` - There are several ways to fix this, only one is shown when using Chrome browsers. Install the plugin named [Allow CORS: Access-Control-Allow-Origin](https://chrome.google.com/webstore/detail/lhobafahddgcelffkeicbaginigeejlf). - -4. When building docs, the following environment variables must be added to the build process. Note that you can use any `npm run antora-xxx` command: - ``` - UPDATE_SEARCH_INDEX=true \ - ELASTICSEARCH_NODE=http://localhost:9200 \ - ELASTICSEARCH_INDEX=docs \ - ELASTICSEARCH_WRITE_AUTH=x:y \ npm run antora-local ``` - Note that `ELASTICSEARCH_WRITE_AUTH` is necessary for building though it does not do any authentication. A value for that envvar must not be omitted but can be any dummy value you like in the format of at minimum two characters separated by a colon. - Running the build now also returns on the console: +2. Index the built site. This reads `public/` and writes `public/pagefind/`: ``` - elastic: generate search index - elastic: rebuild search index - elastic: remove old search index - elastic: create empty search index - elastic: upload search index + npm run pagefind ``` -5. Optionally, the status of Elasticsearch can be monitored: - `http://localhost:9200/_cat/indices?v=` - ``` - green open .geoip_databases Ygj4WI-STGmJrSeCfep7Tg 1 0 41 0 38.3mb 38.3mb - yellow open docs oag2dCMnS4CiSXX0Ul8plA 1 1 163 0 1.4mb 1.4mb - ``` - To make a dummy query after the index has been created, type the following as URL in the browser and replace `term` with what you want to search for: - `http://localhost:9200/_search?q=term` - ``` - {"took":45,"timed_out":false,"_shards":{"total":1,"successful":1,"skipped":0,"failed":0}, ... - ``` - -6. To view the build result either with `npm run serve` (Antora build) or `npm run preview` (UI build) run: + The command prints a short summary of how many pages and words were indexed. + +3. Serve the build and open it in the browser: ``` - ELASTICSEARCH_NODE=http://localhost:9200 \ - ELASTICSEARCH_INDEX=docs \ - ELASTICSEARCH_READ_AUTH=x:y \ npm run serve ``` - Note that for `ELASTICSEARCH_READ_AUTH`, the same applies as for `ELASTICSEARCH_WRITE_AUTH`. +4. Enter any search term into the search field to see matches returned. -7. Open the build via the browser and enter any search term as required into the search field to see matches returned. - -8. Note that building against ownCloud's hosted Elasticsearch is not possible locally though you can use it for previewing the build. To do so, type the following: - ``` - ELASTICSEARCH_NODE=https://search.owncloud.com \ - ELASTICSEARCH_INDEX=docs \ - ELASTICSEARCH_READ_AUTH=docs:cADL6DDAKEBrkFMrvfxXEtYm \ - npm run serve - ``` +In CI, the production build runs `npm run antora` followed by `npm run pagefind` before deploying `public/`, so the deployed site always ships a fresh index. Re-run `npm run pagefind` whenever the HTML changes; serving a build without indexing it first yields an empty search. ## Resolving Edit-this-page in Development diff --git a/docs/extensions.md b/docs/extensions.md index 62ac06958..e6e15493d 100644 --- a/docs/extensions.md +++ b/docs/extensions.md @@ -12,7 +12,6 @@ The documentation uses extensions that are added via the `site.yml` file. Some e **Antora** -* Create an Elastic Search Index: `./ext-antora/generate-index.js` * Load Global Site Attributes: `./ext-antora/load-global-site-attributes.js` ## Optional Extensions diff --git a/es-docker-compose.yml b/es-docker-compose.yml deleted file mode 100644 index 0b4068bda..000000000 --- a/es-docker-compose.yml +++ /dev/null @@ -1,9 +0,0 @@ -services: - elasticsearch: - image: elasticsearch:7.17.15 - ports: - - 9200:9200 - - 9300:9300 - environment: - - discovery.type=single-node - - xpack.security.enabled=false diff --git a/ext-antora/generate-index.js b/ext-antora/generate-index.js deleted file mode 100644 index 02ae7fbe7..000000000 --- a/ext-antora/generate-index.js +++ /dev/null @@ -1,192 +0,0 @@ -'use strict' - -// Antora extension to create a new and clean search index based on ElasticSearch - -const _ = require('lodash') -const cheerio = require('cheerio') -const Entities = require('html-entities') -const { Client } = require('@elastic/elasticsearch') - -module.exports.register = function () { - // run when the Antora site publishing event has been fired = all done - // get all the pages created and index the result - this.on('sitePublished', ({ playbook, contentCatalog }) => { - const pages = contentCatalog.getPages() - generateIndex(playbook, pages) - }) -} - -async function generateIndex(playbook, pages) { - // this env is usually not set when building local - skip indexing - // but when set, we index, also when building local - // only set in .drone.star in the docs repo when doing a full build - if (process.env.UPDATE_SEARCH_INDEX !== 'true') { - console.log('ElasticSearch: index generation skipped') - return - } else { - console.log('ElasticSearch: indexing requested') - } - - // note that the following envvars are required to build the index - // see the documentation at docs/README.md and docs-ui/src/partials/header-content.hbs for more information - if (!process.env.ELASTICSEARCH_NODE) { - console.log('ElasticSearch: ELASTICSEARCH_NODE envvar is missing, indexing skipped') - return - } - if (!process.env.ELASTICSEARCH_INDEX) { - console.log('ElasticSearch: ELASTICSEARCH_INDEX envvar is missing, indexing skipped') - return - } - if (!process.env.ELASTICSEARCH_WRITE_AUTH) { - console.log('ElasticSearch: ELASTICSEARCH_WRITE_AUTH envvar is missing, indexing skipped') - return - } - - let siteUrl = playbook.site.url - if (!siteUrl) { - siteUrl = '' - } - - const index_start = Date.now() - - // index the documents available - const documents = pages.map((page) => { - // a document like '_email-config.adoc' gets excluded from the catalog - // because it has a leading '_' which will result in an undefined - // 'page.pub.url' variable breaking generating the index - if (page.pub === undefined) { - return - } - - const titles = [] - - const html = page.contents.toString() - // for performance reasons, we use parameter xml which then uses the htmlparser2 engine. - // for more details see: - // https://cheerio.js.org/docs/api/interfaces/CheerioOptions - // https://github.com/taoqf/node-html-parser#performance - const $ = cheerio.load(html, { xml: true }) - - const $article = $('article') - const title = $article.find('h1').text() - - $article.find('h1,h2,h3,h4,h5,h6').each(function () { - let $title = $(this) - let id = $title.attr('id') - - titles.push({ - text: $title.text(), - id: $title.attr('id'), - url: siteUrl + page.pub.url + '#' + $title.attr('id') - }) - - $title.remove() - }) - - let text = Entities.decode($('article').text()) - .replace(/(<([^>]+)>)/gi, '') - .replace(/\n/g, ' ') - .replace(/\r/g, ' ') - .replace(/\s+/g, ' ') - .trim() - - // todo, we want to do more fancy stuff with results - // also see: docs-ui/src/js/vendor/elastic.js - return { - component: page.src.component, - version: page.src.version, - name: page.src.stem, - title: title, - text: text, - url: siteUrl + page.pub.url, - titles: titles - } - }) - - // prepare index for uploading - let result = [] - documents.forEach((document, index) => { - result.push({ - index: { - _index: process.env.ELASTICSEARCH_INDEX, - _type: 'page', - _id: index - } - }) - - result.push(document) - }) - - // create a new client to connect - const client = new Client({ - node: process.env.ELASTICSEARCH_NODE, - auth: { - username: process.env.ELASTICSEARCH_WRITE_AUTH.split(':')[0], - password: process.env.ELASTICSEARCH_WRITE_AUTH.split(':')[1] - } - }) - - // remove the old index and create/upload the new one - try { - console.log('ElasticSearch: remove old search index') - await indexDelete(client) - console.log('ElasticSearch: create empty search index') - await indexCreate(client) - console.log('ElasticSearch: upload search index') - await indexBulk(client, result) - } catch (err) { - const errObj = JSON.parse(err) - console.log('ElasticSearch: ERROR: ' + errObj.status + ' - ' + errObj.error.reason) - process.exit(1) - } - const index_end = Date.now() - const elapsedTime = (index_end - index_start) / 1000 - console.log(`ElasticSearch: indexing time: ${elapsedTime}s`) -} - -function indexDelete(client) { - return new Promise((resolve, reject) => { - client.indices - .delete({ - index: process.env.ELASTICSEARCH_INDEX, - ignore_unavailable: true - }) - .then((resp) => { - resolve(resp) - }) - .catch((err) => { - console.warn(err) - resolve(true) - }) - }) -} - -function indexCreate(client) { - return new Promise((resolve, reject) => { - client.indices - .create({ - index: process.env.ELASTICSEARCH_INDEX - }) - .then((resp) => { - resolve(resp) - }) - .catch((err) => { - reject(err) - }) - }) -} - -function indexBulk(client, result) { - return new Promise((resolve, reject) => { - client - .bulk({ - body: result - }) - .then((resp) => { - resolve(resp) - }) - .catch((err) => { - reject(err) - }) - }) -} diff --git a/package-lock.json b/package-lock.json index a6f397858..c336babe5 100644 --- a/package-lock.json +++ b/package-lock.json @@ -1,23 +1,20 @@ { - "name": "docs", + "name": "orchestrator", "lockfileVersion": 3, "requires": true, "packages": { "": { "dependencies": { - "@elastic/elasticsearch": "^7.17.14", "antora": "^3.1.15", "asciidoctor": "^3.0.4", "asciidoctor-kroki": "^0.18.1", - "cheerio": "^1.2.0", - "html-entities": "2.6.0", - "js-yaml": "^4.1.1", - "lodash": "^4.18.1" + "js-yaml": "^4.1.1" }, "devDependencies": { "@sntke/antora-mermaid-extension": "^0.0.13", "broken-link-checker": "^0.7.8", - "http-server": "^14.1.1" + "http-server": "^14.1.1", + "pagefind": "^1.3.0" } }, "node_modules/@antora/asciidoc-loader": { @@ -443,21 +440,6 @@ "node": ">=6.9.0" } }, - "node_modules/@elastic/elasticsearch": { - "version": "7.17.14", - "resolved": "https://registry.npmjs.org/@elastic/elasticsearch/-/elasticsearch-7.17.14.tgz", - "integrity": "sha512-6uQ1pVXutwz1Krwooo67W+3K8BwH1ASMh1WoHTpomUzw8EXecXN5lHIJ9EPqTHuv1WqR2LKkSJyagcq0HYUJpg==", - "license": "Apache-2.0", - "dependencies": { - "debug": "^4.3.1", - "hpagent": "^0.1.1", - "ms": "^2.1.3", - "secure-json-parse": "^2.4.0" - }, - "engines": { - "node": ">=12" - } - }, "node_modules/@iarna/toml": { "version": "2.2.5", "resolved": "https://registry.npmjs.org/@iarna/toml/-/toml-2.2.5.tgz", @@ -499,6 +481,104 @@ "node": ">= 8" } }, + "node_modules/@pagefind/darwin-arm64": { + "version": "1.5.2", + "resolved": "https://registry.npmjs.org/@pagefind/darwin-arm64/-/darwin-arm64-1.5.2.tgz", + "integrity": "sha512-MXpI+7HsAdPkvJ0gk9xj9g541BCqBZOBbdwj9g6lB5LCj6kSV6nqDSjzcAJwvOsfu0fjwvC8hQU+ecfhp+MpiQ==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "darwin" + ] + }, + "node_modules/@pagefind/darwin-x64": { + "version": "1.5.2", + "resolved": "https://registry.npmjs.org/@pagefind/darwin-x64/-/darwin-x64-1.5.2.tgz", + "integrity": "sha512-IojxFWMEJe0RQ7PQ3KXQsPIImNsbpPYpoZ+QUDrL8fAl/O27IX+LVLs74/UzEZy5uA2LD8Nz1AiwKr72vrkZQw==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "darwin" + ] + }, + "node_modules/@pagefind/freebsd-x64": { + "version": "1.5.2", + "resolved": "https://registry.npmjs.org/@pagefind/freebsd-x64/-/freebsd-x64-1.5.2.tgz", + "integrity": "sha512-7EVzo9+0w+2cbe671BtMj10UlNo83I+HrLVLfRxO731svHRJKUfJ/mo05gU14pe9PCfpKNQT8FS3Xc/oDN6pOA==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "freebsd" + ] + }, + "node_modules/@pagefind/linux-arm64": { + "version": "1.5.2", + "resolved": "https://registry.npmjs.org/@pagefind/linux-arm64/-/linux-arm64-1.5.2.tgz", + "integrity": "sha512-Ovt9+K35sqzn8H3ZMXGwls4TD/wMJuvRtShHIsmUQREmaxjrDEX7gHckRCrwYJ4XE1H1p6HkLz3wukrAnsfXQw==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ] + }, + "node_modules/@pagefind/linux-x64": { + "version": "1.5.2", + "resolved": "https://registry.npmjs.org/@pagefind/linux-x64/-/linux-x64-1.5.2.tgz", + "integrity": "sha512-V+tFqHKXhQKq/WqPBD67AFy7scn1/aZID00ws4fSDd+1daSi5UHR9VVlRrOUYKxn3VuFQYRD7lYXdZK1WED1YA==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ] + }, + "node_modules/@pagefind/windows-arm64": { + "version": "1.5.2", + "resolved": "https://registry.npmjs.org/@pagefind/windows-arm64/-/windows-arm64-1.5.2.tgz", + "integrity": "sha512-hN9Nh90fNW61nNRCW9ZyQrAj/mD0eRvmJ8NlTUzkbuW8kIzGJUi3cxjFkEcMZ5h/8FsKWD/VcouZl4yo1F7B6g==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "win32" + ] + }, + "node_modules/@pagefind/windows-x64": { + "version": "1.5.2", + "resolved": "https://registry.npmjs.org/@pagefind/windows-x64/-/windows-x64-1.5.2.tgz", + "integrity": "sha512-Fa2Iyw7kaDRzGMfNYNUXNW2zbL5FQVDgSOcbDHdzBrDEdpqOqg8TcZ68F22ol6NJ9IGzvUdmeyZypLW5dyhqsg==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "win32" + ] + }, "node_modules/@sntke/antora-mermaid-extension": { "version": "0.0.13", "resolved": "https://registry.npmjs.org/@sntke/antora-mermaid-extension/-/antora-mermaid-extension-0.0.13.tgz", @@ -830,12 +910,6 @@ "dev": true, "license": "MIT" }, - "node_modules/boolbase": { - "version": "1.0.0", - "resolved": "https://registry.npmjs.org/boolbase/-/boolbase-1.0.0.tgz", - "integrity": "sha512-JZOSA7Mo9sNGB8+UjSgzdLtokWAky1zbztM3WRLCbZ70/3cTANmQmOdR7y2g+J0e2WXywy1yS468tY+IruqEww==", - "license": "ISC" - }, "node_modules/brace-expansion": { "version": "2.1.1", "resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-2.1.1.tgz", @@ -1068,72 +1142,6 @@ "is-regex": "^1.0.3" } }, - "node_modules/cheerio": { - "version": "1.2.0", - "resolved": "https://registry.npmjs.org/cheerio/-/cheerio-1.2.0.tgz", - "integrity": "sha512-WDrybc/gKFpTYQutKIK6UvfcuxijIZfMfXaYm8NMsPQxSYvf+13fXUJ4rztGGbJcBQ/GF55gvrZ0Bc0bj/mqvg==", - "license": "MIT", - "dependencies": { - "cheerio-select": "^2.1.0", - "dom-serializer": "^2.0.0", - "domhandler": "^5.0.3", - "domutils": "^3.2.2", - "encoding-sniffer": "^0.2.1", - "htmlparser2": "^10.1.0", - "parse5": "^7.3.0", - "parse5-htmlparser2-tree-adapter": "^7.1.0", - "parse5-parser-stream": "^7.1.2", - "undici": "^7.19.0", - "whatwg-mimetype": "^4.0.0" - }, - "engines": { - "node": ">=20.18.1" - }, - "funding": { - "url": "https://github.com/cheeriojs/cheerio?sponsor=1" - } - }, - "node_modules/cheerio-select": { - "version": "2.1.0", - "resolved": "https://registry.npmjs.org/cheerio-select/-/cheerio-select-2.1.0.tgz", - "integrity": "sha512-9v9kG0LvzrlcungtnJtpGNxY+fzECQKhK4EGJX2vByejiMX84MFNQw4UxPJl3bFbTMw+Dfs37XaIkCwTZfLh4g==", - "license": "BSD-2-Clause", - "dependencies": { - "boolbase": "^1.0.0", - "css-select": "^5.1.0", - "css-what": "^6.1.0", - "domelementtype": "^2.3.0", - "domhandler": "^5.0.3", - "domutils": "^3.0.1" - }, - "funding": { - "url": "https://github.com/sponsors/fb55" - } - }, - "node_modules/cheerio/node_modules/entities": { - "version": "6.0.1", - "resolved": "https://registry.npmjs.org/entities/-/entities-6.0.1.tgz", - "integrity": "sha512-aN97NXWF6AWBTahfVOIrB/NShkzi5H7F9r1s9mD3cDj4Ko5f2qhhVoYMibXF7GlLveb/D2ioWay8lxI97Ven3g==", - "license": "BSD-2-Clause", - "engines": { - "node": ">=0.12" - }, - "funding": { - "url": "https://github.com/fb55/entities?sponsor=1" - } - }, - "node_modules/cheerio/node_modules/parse5": { - "version": "7.3.0", - "resolved": "https://registry.npmjs.org/parse5/-/parse5-7.3.0.tgz", - "integrity": "sha512-IInvU7fabl34qmi9gY8XOVxhYyMyuH2xUNpb2q8/Y+7552KlejkRvqvD19nMoUW/uQGGbqNpA6Tufu5FL5BZgw==", - "license": "MIT", - "dependencies": { - "entities": "^6.0.0" - }, - "funding": { - "url": "https://github.com/inikulin/parse5?sponsor=1" - } - }, "node_modules/clean-git-ref": { "version": "2.0.1", "resolved": "https://registry.npmjs.org/clean-git-ref/-/clean-git-ref-2.0.1.tgz", @@ -1343,34 +1351,6 @@ "node": ">=0.8" } }, - "node_modules/css-select": { - "version": "5.2.2", - "resolved": "https://registry.npmjs.org/css-select/-/css-select-5.2.2.tgz", - "integrity": "sha512-TizTzUddG/xYLA3NXodFM0fSbNizXjOKhqiQQwvhlspadZokn1KDy0NZFS0wuEubIYAV5/c1/lAr0TaaFXEXzw==", - "license": "BSD-2-Clause", - "dependencies": { - "boolbase": "^1.0.0", - "css-what": "^6.1.0", - "domhandler": "^5.0.2", - "domutils": "^3.0.1", - "nth-check": "^2.0.1" - }, - "funding": { - "url": "https://github.com/sponsors/fb55" - } - }, - "node_modules/css-what": { - "version": "6.2.2", - "resolved": "https://registry.npmjs.org/css-what/-/css-what-6.2.2.tgz", - "integrity": "sha512-u/O3vwbptzhMs3L1fQE82ZSLHQQfto5gyZzwteVIEyeaY5Fc7R4dapF/BvRoSYFeqfBk4m0V1Vafq5Pjv25wvA==", - "license": "BSD-2-Clause", - "engines": { - "node": ">= 6" - }, - "funding": { - "url": "https://github.com/sponsors/fb55" - } - }, "node_modules/dateformat": { "version": "4.6.3", "resolved": "https://registry.npmjs.org/dateformat/-/dateformat-4.6.3.tgz", @@ -1384,6 +1364,7 @@ "version": "4.4.3", "resolved": "https://registry.npmjs.org/debug/-/debug-4.4.3.tgz", "integrity": "sha512-RGwwWnwQvkVfavKVt22FGLw+xYSdzARwm0ru6DhTVA3umU5hZc28V3kO4stgYryrTlLpuvgI9GiijltAjNbcqA==", + "dev": true, "license": "MIT", "dependencies": { "ms": "^2.1.3" @@ -1461,61 +1442,6 @@ "integrity": "sha512-LLBi6pEqS6Do3EKQ3J0NqHWV5hhb78Pi8vvESYwyOy2c31ZEZVdtitdzsQsKb7878PEERhzUk0ftqGhG6Mz+pQ==", "license": "MIT" }, - "node_modules/dom-serializer": { - "version": "2.0.0", - "resolved": "https://registry.npmjs.org/dom-serializer/-/dom-serializer-2.0.0.tgz", - "integrity": "sha512-wIkAryiqt/nV5EQKqQpo3SToSOV9J0DnbJqwK7Wv/Trc92zIAYZ4FlMu+JPFW1DfGFt81ZTCGgDEabffXeLyJg==", - "license": "MIT", - "dependencies": { - "domelementtype": "^2.3.0", - "domhandler": "^5.0.2", - "entities": "^4.2.0" - }, - "funding": { - "url": "https://github.com/cheeriojs/dom-serializer?sponsor=1" - } - }, - "node_modules/domelementtype": { - "version": "2.3.0", - "resolved": "https://registry.npmjs.org/domelementtype/-/domelementtype-2.3.0.tgz", - "integrity": "sha512-OLETBj6w0OsagBwdXnPdN0cnMfF9opN69co+7ZrbfPGrdpPVNBUj02spi6B1N7wChLQiPn4CSH/zJvXw56gmHw==", - "funding": [ - { - "type": "github", - "url": "https://github.com/sponsors/fb55" - } - ], - "license": "BSD-2-Clause" - }, - "node_modules/domhandler": { - "version": "5.0.3", - "resolved": "https://registry.npmjs.org/domhandler/-/domhandler-5.0.3.tgz", - "integrity": "sha512-cgwlv/1iFQiFnU96XXgROh8xTeetsnJiDsTc7TYCLFd9+/WNkIqPTxiM/8pSd8VIrhXGTf1Ny1q1hquVqDJB5w==", - "license": "BSD-2-Clause", - "dependencies": { - "domelementtype": "^2.3.0" - }, - "engines": { - "node": ">= 4" - }, - "funding": { - "url": "https://github.com/fb55/domhandler?sponsor=1" - } - }, - "node_modules/domutils": { - "version": "3.2.2", - "resolved": "https://registry.npmjs.org/domutils/-/domutils-3.2.2.tgz", - "integrity": "sha512-6kZKyUajlDuqlHKVX1w7gyslj9MPIXzIFiz/rGu35uC1wMi+kMhQwGhl4lt9unC9Vb9INnY9Z3/ZA3+FhASLaw==", - "license": "BSD-2-Clause", - "dependencies": { - "dom-serializer": "^2.0.0", - "domelementtype": "^2.3.0", - "domhandler": "^5.0.3" - }, - "funding": { - "url": "https://github.com/fb55/domutils?sponsor=1" - } - }, "node_modules/dunder-proto": { "version": "1.0.1", "resolved": "https://registry.npmjs.org/dunder-proto/-/dunder-proto-1.0.1.tgz", @@ -1558,19 +1484,6 @@ "integrity": "sha512-MSjYzcWNOA0ewAHpz0MxpYFvwg6yjy1NG3xteoqz644VCo/RPgnr1/GGt+ic3iJTzQ8Eu3TdM14SawnVUmGE6A==", "license": "MIT" }, - "node_modules/encoding-sniffer": { - "version": "0.2.1", - "resolved": "https://registry.npmjs.org/encoding-sniffer/-/encoding-sniffer-0.2.1.tgz", - "integrity": "sha512-5gvq20T6vfpekVtqrYQsSCFZ1wEg5+wW0/QaZMWkFr6BqD3NfKs0rLCx4rrVlSWJeZb5NBJgVLswK/w2MWU+Gw==", - "license": "MIT", - "dependencies": { - "iconv-lite": "^0.6.3", - "whatwg-encoding": "^3.1.1" - }, - "funding": { - "url": "https://github.com/fb55/encoding-sniffer?sponsor=1" - } - }, "node_modules/end-of-stream": { "version": "1.4.5", "resolved": "https://registry.npmjs.org/end-of-stream/-/end-of-stream-1.4.5.tgz", @@ -1580,18 +1493,6 @@ "once": "^1.4.0" } }, - "node_modules/entities": { - "version": "4.5.0", - "resolved": "https://registry.npmjs.org/entities/-/entities-4.5.0.tgz", - "integrity": "sha512-V0hjH4dGPh9Ao5p0MoRY6BVqtwCjhz6vI5LT8AJ55H+4g9/4vbHx1I54fS0XuclLhDHArPQCiMjDxjaL8fPxhw==", - "license": "BSD-2-Clause", - "engines": { - "node": ">=0.12" - }, - "funding": { - "url": "https://github.com/fb55/entities?sponsor=1" - } - }, "node_modules/eol": { "version": "0.2.0", "resolved": "https://registry.npmjs.org/eol/-/eol-0.2.0.tgz", @@ -2075,12 +1976,6 @@ "integrity": "sha512-7xgomUX6ADmcYzFik0HzAxh/73YlKR9bmFzf51CZwR+b6YtzU2m0u49hQCqV6SvlqIqsaxovfwdvbnsw3b/zpg==", "license": "MIT" }, - "node_modules/hpagent": { - "version": "0.1.2", - "resolved": "https://registry.npmjs.org/hpagent/-/hpagent-0.1.2.tgz", - "integrity": "sha512-ePqFXHtSQWAFXYmj+JtOTHr84iNrII4/QRlAAPPE+zqnKy4xJo7Ie1Y4kC7AdB+LxLxSTTzBMASsEcy0q8YyvQ==", - "license": "MIT" - }, "node_modules/html-encoding-sniffer": { "version": "3.0.0", "resolved": "https://registry.npmjs.org/html-encoding-sniffer/-/html-encoding-sniffer-3.0.0.tgz", @@ -2108,53 +2003,6 @@ "node": ">=12" } }, - "node_modules/html-entities": { - "version": "2.6.0", - "resolved": "https://registry.npmjs.org/html-entities/-/html-entities-2.6.0.tgz", - "integrity": "sha512-kig+rMn/QOVRvr7c86gQ8lWXq+Hkv6CbAH1hLu+RG338StTpE8Z0b44SDVaqVu7HGKf27frdmUYEs9hTUX/cLQ==", - "funding": [ - { - "type": "github", - "url": "https://github.com/sponsors/mdevils" - }, - { - "type": "patreon", - "url": "https://patreon.com/mdevils" - } - ], - "license": "MIT" - }, - "node_modules/htmlparser2": { - "version": "10.1.0", - "resolved": "https://registry.npmjs.org/htmlparser2/-/htmlparser2-10.1.0.tgz", - "integrity": "sha512-VTZkM9GWRAtEpveh7MSF6SjjrpNVNNVJfFup7xTY3UpFtm67foy9HDVXneLtFVt4pMz5kZtgNcvCniNFb1hlEQ==", - "funding": [ - "https://github.com/fb55/htmlparser2?sponsor=1", - { - "type": "github", - "url": "https://github.com/sponsors/fb55" - } - ], - "license": "MIT", - "dependencies": { - "domelementtype": "^2.3.0", - "domhandler": "^5.0.3", - "domutils": "^3.2.2", - "entities": "^7.0.1" - } - }, - "node_modules/htmlparser2/node_modules/entities": { - "version": "7.0.1", - "resolved": "https://registry.npmjs.org/entities/-/entities-7.0.1.tgz", - "integrity": "sha512-TWrgLOFUQTH994YUyl1yT4uyavY5nNB5muff+RtWaqNVCAK408b5ZnnbNAUEWLTCpum9w6arT70i1XdQ4UeOPA==", - "license": "BSD-2-Clause", - "engines": { - "node": ">=0.12" - }, - "funding": { - "url": "https://github.com/fb55/entities?sponsor=1" - } - }, "node_modules/http-equiv-refresh": { "version": "1.0.0", "resolved": "https://registry.npmjs.org/http-equiv-refresh/-/http-equiv-refresh-1.0.0.tgz", @@ -2268,6 +2116,7 @@ "version": "0.6.3", "resolved": "https://registry.npmjs.org/iconv-lite/-/iconv-lite-0.6.3.tgz", "integrity": "sha512-4fCk79wshMdzMp2rH06qWrJE4iolqLhCUH+OiuIgU++RB0+94NlDL81atO7GX55uUKueo0txHNtvEyI6D7WdMw==", + "dev": true, "license": "MIT", "dependencies": { "safer-buffer": ">= 2.1.2 < 3.0.0" @@ -2652,12 +2501,6 @@ "node": ">= 0.10" } }, - "node_modules/lodash": { - "version": "4.18.1", - "resolved": "https://registry.npmjs.org/lodash/-/lodash-4.18.1.tgz", - "integrity": "sha512-dMInicTPVE8d1e5otfwmmjlxkZoUpiVLwyeTdUsi/Caj/gfzzblBcCE5sRHV/AsjuCmxWrte2TNGSYuCeCq+0Q==", - "license": "MIT" - }, "node_modules/lodash.clonedeep": { "version": "4.5.0", "resolved": "https://registry.npmjs.org/lodash.clonedeep/-/lodash.clonedeep-4.5.0.tgz", @@ -2827,6 +2670,7 @@ "version": "2.1.3", "resolved": "https://registry.npmjs.org/ms/-/ms-2.1.3.tgz", "integrity": "sha512-6FlzubTLZG3J2a/NVCAleEhjzq5oxgHyaCU9yYXvcLsvoVaHJq/s5xXI6/XXP6tz7R9xAOtHnSO/tXtF3WRTlA==", + "dev": true, "license": "MIT" }, "node_modules/multi-progress": { @@ -2959,18 +2803,6 @@ "node": ">=0.10.0" } }, - "node_modules/nth-check": { - "version": "2.1.1", - "resolved": "https://registry.npmjs.org/nth-check/-/nth-check-2.1.1.tgz", - "integrity": "sha512-lqjrjmaOoAnWfMmBPL+XNnynZh2+swxiX3WUE0s4yEHI6m+AwrK2UZOimIRl3X/4QctVqS8AiZjFqyOGrMXb/w==", - "license": "BSD-2-Clause", - "dependencies": { - "boolbase": "^1.0.0" - }, - "funding": { - "url": "https://github.com/fb55/nth-check?sponsor=1" - } - }, "node_modules/nunjucks": { "version": "3.2.4", "resolved": "https://registry.npmjs.org/nunjucks/-/nunjucks-3.2.4.tgz", @@ -3099,6 +2931,25 @@ "node": ">=0.10.0" } }, + "node_modules/pagefind": { + "version": "1.5.2", + "resolved": "https://registry.npmjs.org/pagefind/-/pagefind-1.5.2.tgz", + "integrity": "sha512-XTUaK0hXMCu2jszWE584JGQT7y284TmMV9l/HX3rnG5uo3rHI/uHU56XTyyyPFjeWEBxECbAi0CaFDJOONtG0Q==", + "dev": true, + "license": "MIT", + "bin": { + "pagefind": "lib/runner/bin.cjs" + }, + "optionalDependencies": { + "@pagefind/darwin-arm64": "1.5.2", + "@pagefind/darwin-x64": "1.5.2", + "@pagefind/freebsd-x64": "1.5.2", + "@pagefind/linux-arm64": "1.5.2", + "@pagefind/linux-x64": "1.5.2", + "@pagefind/windows-arm64": "1.5.2", + "@pagefind/windows-x64": "1.5.2" + } + }, "node_modules/pako": { "version": "2.1.0", "resolved": "https://registry.npmjs.org/pako/-/pako-2.1.0.tgz", @@ -3122,79 +2973,6 @@ "@types/node": "*" } }, - "node_modules/parse5-htmlparser2-tree-adapter": { - "version": "7.1.0", - "resolved": "https://registry.npmjs.org/parse5-htmlparser2-tree-adapter/-/parse5-htmlparser2-tree-adapter-7.1.0.tgz", - "integrity": "sha512-ruw5xyKs6lrpo9x9rCZqZZnIUntICjQAd0Wsmp396Ul9lN/h+ifgVV1x1gZHi8euej6wTfpqX8j+BFQxF0NS/g==", - "license": "MIT", - "dependencies": { - "domhandler": "^5.0.3", - "parse5": "^7.0.0" - }, - "funding": { - "url": "https://github.com/inikulin/parse5?sponsor=1" - } - }, - "node_modules/parse5-htmlparser2-tree-adapter/node_modules/entities": { - "version": "6.0.1", - "resolved": "https://registry.npmjs.org/entities/-/entities-6.0.1.tgz", - "integrity": "sha512-aN97NXWF6AWBTahfVOIrB/NShkzi5H7F9r1s9mD3cDj4Ko5f2qhhVoYMibXF7GlLveb/D2ioWay8lxI97Ven3g==", - "license": "BSD-2-Clause", - "engines": { - "node": ">=0.12" - }, - "funding": { - "url": "https://github.com/fb55/entities?sponsor=1" - } - }, - "node_modules/parse5-htmlparser2-tree-adapter/node_modules/parse5": { - "version": "7.3.0", - "resolved": "https://registry.npmjs.org/parse5/-/parse5-7.3.0.tgz", - "integrity": "sha512-IInvU7fabl34qmi9gY8XOVxhYyMyuH2xUNpb2q8/Y+7552KlejkRvqvD19nMoUW/uQGGbqNpA6Tufu5FL5BZgw==", - "license": "MIT", - "dependencies": { - "entities": "^6.0.0" - }, - "funding": { - "url": "https://github.com/inikulin/parse5?sponsor=1" - } - }, - "node_modules/parse5-parser-stream": { - "version": "7.1.2", - "resolved": "https://registry.npmjs.org/parse5-parser-stream/-/parse5-parser-stream-7.1.2.tgz", - "integrity": "sha512-JyeQc9iwFLn5TbvvqACIF/VXG6abODeB3Fwmv/TGdLk2LfbWkaySGY72at4+Ty7EkPZj854u4CrICqNk2qIbow==", - "license": "MIT", - "dependencies": { - "parse5": "^7.0.0" - }, - "funding": { - "url": "https://github.com/inikulin/parse5?sponsor=1" - } - }, - "node_modules/parse5-parser-stream/node_modules/entities": { - "version": "6.0.1", - "resolved": "https://registry.npmjs.org/entities/-/entities-6.0.1.tgz", - "integrity": "sha512-aN97NXWF6AWBTahfVOIrB/NShkzi5H7F9r1s9mD3cDj4Ko5f2qhhVoYMibXF7GlLveb/D2ioWay8lxI97Ven3g==", - "license": "BSD-2-Clause", - "engines": { - "node": ">=0.12" - }, - "funding": { - "url": "https://github.com/fb55/entities?sponsor=1" - } - }, - "node_modules/parse5-parser-stream/node_modules/parse5": { - "version": "7.3.0", - "resolved": "https://registry.npmjs.org/parse5/-/parse5-7.3.0.tgz", - "integrity": "sha512-IInvU7fabl34qmi9gY8XOVxhYyMyuH2xUNpb2q8/Y+7552KlejkRvqvD19nMoUW/uQGGbqNpA6Tufu5FL5BZgw==", - "license": "MIT", - "dependencies": { - "entities": "^6.0.0" - }, - "funding": { - "url": "https://github.com/inikulin/parse5?sponsor=1" - } - }, "node_modules/path-parse": { "version": "1.0.7", "resolved": "https://registry.npmjs.org/path-parse/-/path-parse-1.0.7.tgz", @@ -3850,6 +3628,7 @@ "version": "2.1.2", "resolved": "https://registry.npmjs.org/safer-buffer/-/safer-buffer-2.1.2.tgz", "integrity": "sha512-YZo3K82SD7Riyi0E1EQPojLz7kpepnSQI9IyPbHHg1XXXevb5dJI7tpyN2ADxGcQbHG7vcyRHk0cbwqcQriUtg==", + "dev": true, "license": "MIT" }, "node_modules/secure-compare": { @@ -4442,15 +4221,6 @@ "node": ">=0.8.0" } }, - "node_modules/undici": { - "version": "7.27.0", - "resolved": "https://registry.npmjs.org/undici/-/undici-7.27.0.tgz", - "integrity": "sha512-+t2Z/GwkZQDtu00813aP66ygViGtPHKhhoFZpQKpKrE+9jIgES+Zw+mFNaDWOVRKiuJjuqKHzD3B1sfGg8+ZOQ==", - "license": "MIT", - "engines": { - "node": ">=20.18.1" - } - }, "node_modules/undici-types": { "version": "7.24.6", "resolved": "https://registry.npmjs.org/undici-types/-/undici-types-7.24.6.tgz", @@ -4574,28 +4344,6 @@ "node": ">=0.10.0" } }, - "node_modules/whatwg-encoding": { - "version": "3.1.1", - "resolved": "https://registry.npmjs.org/whatwg-encoding/-/whatwg-encoding-3.1.1.tgz", - "integrity": "sha512-6qN4hJdMwfYBtE3YBTTHhoeuUrDBPZmbQaxWAqSALV/MeEnR5z1xd8UKud2RAkFoPkmB+hli1TZSnyi84xz1vQ==", - "deprecated": "Use @exodus/bytes instead for a more spec-conformant and faster implementation", - "license": "MIT", - "dependencies": { - "iconv-lite": "0.6.3" - }, - "engines": { - "node": ">=18" - } - }, - "node_modules/whatwg-mimetype": { - "version": "4.0.0", - "resolved": "https://registry.npmjs.org/whatwg-mimetype/-/whatwg-mimetype-4.0.0.tgz", - "integrity": "sha512-QaKxh0eNIi2mE9p2vEdzfagOKHCcj1pJ56EEHGQOVxp8r9/iszLUUV7v89x9O1p/T+NlTM5W7jW6+cz4Fq1YVg==", - "license": "MIT", - "engines": { - "node": ">=18" - } - }, "node_modules/which-typed-array": { "version": "1.1.21", "resolved": "https://registry.npmjs.org/which-typed-array/-/which-typed-array-1.1.21.tgz", diff --git a/package.json b/package.json index b78ba9c20..a8020f66e 100644 --- a/package.json +++ b/package.json @@ -10,22 +10,20 @@ "antora-dev-local": "antora --stacktrace --url http://localhost:8080 site-dev.yml", "antora-dev-bundle": "antora --stacktrace --ui-bundle-url ../docs-ui/build/ui-bundle.zip --url http://localhost:8080 site-dev.yml", "serve": "http-server public/ -d -i", - "linkcheck": "broken-link-checker --filter-level 3 --recursive --verbose" + "linkcheck": "broken-link-checker --filter-level 3 --recursive --verbose", + "pagefind": "pagefind --site public" }, "dependencies": { - "@elastic/elasticsearch": "^7.17.14", "antora": "^3.1.15", "asciidoctor": "^3.0.4", "asciidoctor-kroki": "^0.18.1", - "cheerio": "^1.2.0", - "html-entities": "2.6.0", - "js-yaml": "^4.1.1", - "lodash": "^4.18.1" + "js-yaml": "^4.1.1" }, "devDependencies": { "@sntke/antora-mermaid-extension": "^0.0.13", "broken-link-checker": "^0.7.8", - "http-server": "^14.1.1" + "http-server": "^14.1.1", + "pagefind": "^1.3.0" }, "overrides": { "asciidoctor-opal-runtime": { diff --git a/site.yml b/site.yml index b80e1b58e..ed43a8c1d 100644 --- a/site.yml +++ b/site.yml @@ -79,7 +79,6 @@ asciidoc: antora: extensions: - - ./ext-antora/generate-index.js - ./ext-antora/comp-version.js - require: ./ext-antora/load-global-site-attributes.js #attributefile: https://raw.githubusercontent.com/owncloud/docs/refs/heads/master/global-attributes.yml From c2fb874aadae87bc828ab4420a668d47446bd865 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Thomas=20M=C3=BCller?= <1005065+DeepDiver1975@users.noreply.github.com> Date: Tue, 16 Jun 2026 11:56:51 +0200 Subject: [PATCH 2/2] ci: fail build on empty Pagefind index; require pagefind >= 1.5.0 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Address review feedback on the Elasticsearch -> Pagefind migration: - CI ran `npm run pagefind` unconditionally. pagefind exits 0 even when it indexes nothing (e.g. the output dir moves or no indexable pages are emitted), so a broken index would deploy with silently empty search. Tee the output and fail the build unless at least 10 pages were indexed. - Bump the pagefind floor from ^1.3.0 to ^1.5.0. The docs-ui search markup uses the Component UI web components (pagefind-modal, pagefind-modal-trigger), which were introduced in Pagefind 1.5.0; ^1.3.0 declared compatibility with releases that lack them. The lockfile already resolved 1.5.2, so this only corrects the declared range (no dependency change). Signed-off-by: Thomas Müller <1005065+DeepDiver1975@users.noreply.github.com> --- .github/workflows/ci.yml | 11 ++++++++++- package-lock.json | 2 +- package.json | 2 +- 3 files changed, 12 insertions(+), 3 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index b2d6f1873..d48a2a6b1 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -34,7 +34,16 @@ jobs: - name: Build docs run: | npm run antora - npm run pagefind + # Build the Pagefind search index and fail the build if it is empty. + # pagefind exits 0 even when it indexes nothing (e.g. the output dir + # moved or no indexable pages were emitted), which would deploy a site + # with silently broken search. Require a non-trivial page count. + npm run pagefind | tee pagefind.log + grep -qE 'Indexed [1-9][0-9]+ pages' pagefind.log || { + echo '::error::Pagefind indexed fewer than 10 pages - search index looks broken' + exit 1 + } + rm -f pagefind.log echo 'doc.owncloud.com' > public/CNAME - name: Save cache diff --git a/package-lock.json b/package-lock.json index c336babe5..91fc58569 100644 --- a/package-lock.json +++ b/package-lock.json @@ -14,7 +14,7 @@ "@sntke/antora-mermaid-extension": "^0.0.13", "broken-link-checker": "^0.7.8", "http-server": "^14.1.1", - "pagefind": "^1.3.0" + "pagefind": "^1.5.0" } }, "node_modules/@antora/asciidoc-loader": { diff --git a/package.json b/package.json index a8020f66e..96e6c590b 100644 --- a/package.json +++ b/package.json @@ -23,7 +23,7 @@ "@sntke/antora-mermaid-extension": "^0.0.13", "broken-link-checker": "^0.7.8", "http-server": "^14.1.1", - "pagefind": "^1.3.0" + "pagefind": "^1.5.0" }, "overrides": { "asciidoctor-opal-runtime": {