From e0d5c00c7dd974fc6e65908d25dd140161b4fbc2 Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 7 Sep 2026 16:28:08 +0000 Subject: [PATCH 1/2] docs: rewrite plugin docs to match current app behavior Verified docs/docs.md claims against the lnreader main app's actual plugin types and runtime (src/plugins/types/index.ts, pluginManager.ts, helpers/fetch.ts, helpers/storage.ts) and fixed drift: - Split into two explicit contracts: the plugin bundle API vs. the repository manifest/install metadata (icon -> iconUrl, customJS/ customCSS as downloaded manifest URLs, install flow). - Documented pagination (parsePage/SourcePage/totalPages), previously missing entirely. - Fixed NovelItem (missing required `id`), SourceNovel (`chapters` required, `status` is NovelStatus-only, missing `totalPages`), PluginSettings (Select/CheckboxGroup types and per-type value shapes, reload-required-for-cached-settings caveat), storage helpers (full storage API, read-only localStorage/sessionStorage), and the runtime library list (dayjs, urlencode, htmlparser2). - Filled in "Using Cheerio" and "Custom fetching functions" with real examples, fetchApi's default headers, and fetchText's silent-empty- string-on-failure behavior. - Fixed the popularNovels filter example, the Switch setting example, and the SourceNovel::path anchor link. - Added a "Getting started" section. Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_01LoFgTFvjNVdrUbZVjdFuxf --- docs/docs.md | 484 ++++++++++++++++++++++++++++++++++++++++++--------- 1 file changed, 402 insertions(+), 82 deletions(-) diff --git a/docs/docs.md b/docs/docs.md index 52c16d2e6..1627379e4 100644 --- a/docs/docs.md +++ b/docs/docs.md @@ -1,17 +1,53 @@ ## Documentation for LNReader plugins -- [PluginBase](#pluginbase) +- [Getting started](#getting-started) +- [Plugin bundle API](#plugin-bundle-api) + - [PluginBase](#pluginbase) - [NovelItem](#novelitem) - [SourceNovel](#sourcenovel) - [ChapterItem](#chapteritem) - [Filters](#filters) - [PluginSettings](#pluginsettings) - [NovelStatus](#novelstatus) -- [Using Cheerio](#using-cheerio) -- [Custom fetching functions](#custom-fetching-functions) -- [Other libraries](#other-libraries) + - [Pagination](#pagination) + - [Using Cheerio](#using-cheerio) + - [Custom fetching functions](#custom-fetching-functions) + - [Other libraries](#other-libraries) +- [Repository manifest & install metadata](#repository-manifest--install-metadata) -Most of the Plugin/Novel type definitions accessed using the `Plugin` namespace imported via +--- + +### Getting started + +1. Pick the language folder your source belongs to under `plugins//` (full language name, + e.g. `plugins/english/`), and create a `.ts` file there — e.g. `plugins/english/myNovelSite.ts`. +2. Copy [`docs/plugin-template.ts`](./plugin-template.ts) into that file as a starting point. It + already imports the pieces most plugins need (`fetchApi`/`fetchText`, `Plugin` namespace, + `Filters`, `cheerio`, `defaultCover`, `NovelStatus`) and stubs out the required methods. +3. Add a 96x96px icon at `public/static/src///icon.png` (note: this + folder uses the **short** language code, e.g. `en`, not the full folder name from step 1), then + set `icon = 'src///icon.png'` on your class — see + [PluginBase::icon](#pluginbaseicon) and [Repository manifest & install metadata](#repository-manifest--install-metadata) + for what happens to that path at publish time. +4. Fill in `popularNovels`, `parseNovel`, `parseChapter`, and `searchNovels` against the target + site — the reference sections below cover the shape each one returns. [Using Cheerio](#using-cheerio) + and [Custom fetching functions](#custom-fetching-functions) cover the two building blocks most + plugins need for that. +5. Test locally with `npm run dev:start`, which launches a browser playground at + `http://localhost:3000` where you can run your plugin's functions against the real site. Before + opening a PR, run `npm run check:plugin -- plugins//yourPlugin.ts` (see + [`docs/testing.md`](./testing.md)) — this is the same live-site check CI runs. + +For CMS-templated sites (WordPress themes, Madara, etc.) and any other repo-specific workflow +detail (multi-source generators, icon conventions, live-check tooling), see +[`docs/quickstart.md`](./quickstart.md). + +--- + +## Plugin bundle API + +This is the contract your plugin file itself implements: a default-exported instance of a class +satisfying `Plugin.PluginBase`, imported via ```ts import { Plugin } from '@/types/plugin'; @@ -25,24 +61,25 @@ PluginBase is a base class for all plugins. class ExamplePlugin implements Plugin.PluginBase {} ``` -| Field | Required | Description | -| ----------------------------------------------------------- | -------- | ------------------------------------------------------- | -| [id](#pluginbaseid) | yes | Plugin ID | -| [name](#pluginbasename) | yes | Plugin Name | -| [icon](#pluginbaseicon) | yes | Plugin Icon | -| [site](#pluginbasesite) | yes | Plugin site link | -| [version](#pluginbaseversion) | yes | Plugin version | -| [imageRequestInit](#pluginbaseimagerequestinit) | no | Plugin Image Request Init | -| [filters](#pluginbasefilters) | no | [Filter definition](#filter-definition-object) object | -| [pluginSettings](#pluginbasepluginsettings) | no | [Plugin settings](#pluginsettings) object | -| [webStorageUtilized](#pluginbasewebstorageutilized) | no | Flag for plugins that need `localStorage`/`sessionStorage` | -| [customJS](#pluginbasecustomjs) | no | Path to a custom JS file bundled with the plugin | -| [customCSS](#pluginbasecustomcss) | no | Path to a custom CSS file bundled with the plugin | -| [popularNovels(page, options)](#pluginbasepopularnovels) | yes | Novel list getter | -| [parseNovel(path)](#pluginbaseparsenovel) | yes | Novel info and chapter list getter | -| [parseChapter(path)](#pluginbaseparsechapter) | yes | Chapter text getter | -| [searchNovels(searchTerm, page)](#pluginbasesearchnovels) | yes | Novel searching getter | -| [resolveUrl(path, isNovel)](#pluginbaseresolveurl) | no | Helper that turns a novel/chapter path into a full URL | +| Field | Required | Description | +| ------------------------------------------------------------- | -------- | ------------------------------------------------------------- | +| [id](#pluginbaseid) | yes | Plugin ID | +| [name](#pluginbasename) | yes | Plugin Name | +| [icon](#pluginbaseicon) | yes | Path to the plugin's icon, converted to `iconUrl` at publish time — see [Repository manifest](#repository-manifest--install-metadata) | +| [site](#pluginbasesite) | yes | Plugin site link | +| [version](#pluginbaseversion) | yes | Plugin version | +| [imageRequestInit](#pluginbaseimagerequestinit) | no | Plugin Image Request Init | +| [filters](#pluginbasefilters) | no | [Filter definition](#filter-definition-object) object | +| [pluginSettings](#pluginbasepluginsettings) | no | [Plugin settings](#pluginsettings) object | +| [webStorageUtilized](#pluginbasewebstorageutilized) | no | Flag for plugins that need `localStorage`/`sessionStorage` | +| [customJS](#pluginbasecustomjs) | no | Path to a custom JS file, converted to a manifest URL — see [Repository manifest](#repository-manifest--install-metadata) | +| [customCSS](#pluginbasecustomcss) | no | Path to a custom CSS file, converted to a manifest URL — see [Repository manifest](#repository-manifest--install-metadata) | +| [popularNovels(page, options)](#pluginbasepopularnovels) | yes | Novel list getter | +| [parseNovel(path)](#pluginbaseparsenovel) | yes | Novel info and chapter list getter | +| [parseChapter(path)](#pluginbaseparsechapter) | yes | Chapter text getter | +| [searchNovels(searchTerm, page)](#pluginbasesearchnovels) | yes | Novel searching getter | +| [resolveUrl(path, isNovel)](#pluginbaseresolveurl) | no | Helper that turns a novel/chapter path into a full URL | +| [parsePage(novelPath, page)](#pagination) | no | Chapter-list-by-page getter, for novels too large to list in one `parseNovel` call — see [Pagination](#pagination) | #### PluginBase::id @@ -71,8 +108,13 @@ class ExamplePlugin implements Plugin.PluginBase { #### PluginBase::icon The path to your plugin's icon, relative to `public/static` (do **not** include the -`public/static` prefix itself). The file must actually live at -`public/static/` in this repo. +`public/static` prefix itself). The file must actually live at `public/static/` in this +repo. + +This is an authoring-time path only — the app itself never reads `icon` or `public/static` +directly. At publish time, `scripts/build-plugin-manifest.js` reads this field off your compiled +plugin and turns it into an absolute `iconUrl` in the published manifest, which is what the app +actually fetches and displays. See [Repository manifest & install metadata](#repository-manifest--install-metadata). ```ts class ExamplePlugin implements Plugin.PluginBase { @@ -159,6 +201,11 @@ Path to a custom JavaScript file, relative to `public/static` (same convention a [icon](#pluginbaseicon)). Used by some multi-source templates to run extra JS against the parsed page (e.g. stripping a site's injected copyright notice). +Like `icon`, this is an authoring-time path: `scripts/build-plugin-manifest.js` turns it into an +absolute URL in the published manifest, and the app downloads that URL into the plugin's private +storage the first time the plugin is installed or updated — see +[Repository manifest & install metadata](#repository-manifest--install-metadata). + ```ts class ExamplePlugin implements Plugin.PluginBase { ... @@ -172,6 +219,9 @@ class ExamplePlugin implements Plugin.PluginBase { Path to a custom CSS file, relative to `public/static` (same convention as [icon](#pluginbaseicon)), applied when rendering the chapter/novel page in-app. +Same publish-time/install-time handling as [customJS](#pluginbasecustomjs) — see +[Repository manifest & install metadata](#repository-manifest--install-metadata). + ```ts class ExamplePlugin implements Plugin.PluginBase { ... @@ -183,7 +233,14 @@ class ExamplePlugin implements Plugin.PluginBase { #### PluginBase::filters A [Filter definition](#filter-definition-object) object that holds filters used in the -[popularNovels](#pluginbasepopularnovels) function +[popularNovels](#pluginbasepopularnovels) function. `Filters` and `FilterTypes` come from +`@libs/filterInputs`: + +```ts +import { FilterTypes, Filters } from '@libs/filterInputs'; +``` + +See [Filters](#filters) for the full type reference. ###### Example @@ -248,7 +305,7 @@ class ExamplePlugin implements Plugin.PluginBase { options: Plugin.PopularNovelsOptions, ): Promise { const novels: Plugin.NovelItem[] = []; - if (options.filters.example.value === 'test') { + if (options.filters.status.value === 'ongoing') { novels.push({ name: 'Novel1', path: '/novel1', @@ -287,7 +344,7 @@ See [Using cheerio](#using-cheerio) for more information on how to parse HTML do `SourceNovel` Novel information and chapter list as [SourceNovel](#sourcenovel) object > [!CAUTION] -> [SourceNovel::path](#sourcenovel) should be the same value as [NovelItem::path](#novelitempath) provided as parameter! +> [SourceNovel::path](#sourcenovelpath) should be the same value as [NovelItem::path](#novelitempath) provided as parameter! ###### Example @@ -304,16 +361,15 @@ class ExamplePlugin implements Plugin.PluginBase { genres: 'Isekai, Neverland', status: NovelStatus.Completed, summary: '', + chapters: [], }; - const chapters: Plugin.ChapterItem[] = []; const chapter: Plugin.ChapterItem = { name: '', path: '', releaseTime: '', chapterNumber: 0, }; - chapters.push(chapter); - novel.chapters = chapters; + novel.chapters.push(chapter); return novel; } ... @@ -381,7 +437,6 @@ class ExamplePlugin implements Plugin.PluginBase { const novels: Plugin.NovelItem[] = []; return novels; } - ... } ``` @@ -411,11 +466,12 @@ class ExamplePlugin implements Plugin.PluginBase { It is an object representing information on how to store/access the novel -| Field | type | Required | Description | -| -------------------------------- | -------- | -------- | ------------------------------------------ | -|

path

| `string` | yes | The relative path to the novel | -|

name

| `string` | yes | The name of the novel shown in the library | -|

cover

| `string` | no | URL to novel's cover | +| Field | Type | Required | Description | +| ---------------------------------- | ----------- | -------- | --------------------------------------------------------- | +|

id

| `undefined` | yes | Reserved for the app's internal use — always assign the literal value `undefined`, never a real id, from plugin code | +|

path

| `string` | yes | The relative path to the novel | +|

name

| `string` | yes | The name of the novel shown in the library | +|

cover

| `string` | no | URL to novel's cover | #### Default cover @@ -429,21 +485,22 @@ import { defaultCover } from '@libs/defaultCover'; ### SourceNovel -`SourceNovel` extends [NovelItem](#novelitem), so `path`, `name`, and `cover` behave the same way -here as they do there. - -| Field | Type | Required | Description | -| -------- | ---------------------------------- | -------- | --------------------------------------------- | -| path | `string` | yes | Must match the [NovelItem::path](#novelitempath) passed into `parseNovel` | -| name | `string` | yes | The novel's title | -| cover | `string` | no | URL to the novel's cover | -| genres | `string` | no | Comma-separated genre list, e.g. `"Action,Fantasy,Romance"` | -| summary | `string` | no | The novel's synopsis/description | -| author | `string` | no | | -| artist | `string` | no | | -| status | [NovelStatus](#novelstatus) or `string` | no | See [NovelStatus](#novelstatus) for the standard values | -| rating | `number` | no | Rating out of 5, as a float | -| chapters | [ChapterItem](#chapteritem)`[]` | no | The novel's chapter list | +`SourceNovel` extends [NovelItem](#novelitem), so `id`, `path`, `name`, and `cover` behave the same +way here as they do there. + +| Field | Type | Required | Description | +| -------- | ------------------------------------ | -------- | ----------------------------------------------------------------------------- | +|

path

| `string` | yes | Must match the [NovelItem::path](#novelitempath) passed into `parseNovel` | +| name | `string` | yes | The novel's title | +| cover | `string` | no | URL to the novel's cover | +| genres | `string` | no | Comma-separated genre list, e.g. `"Action,Fantasy,Romance"` | +| summary | `string` | no | The novel's synopsis/description | +| author | `string` | no | | +| artist | `string` | no | | +| status | [NovelStatus](#novelstatus) | no | See [NovelStatus](#novelstatus) for the standard values | +| rating | `number` | no | Rating out of 5, as a float | +| chapters | [ChapterItem](#chapteritem)`[]` | yes | The novel's chapter list. If the novel is paginated, return the first page's chapters here and see [Pagination](#pagination) | +| totalPages | `number` | no | Total number of chapter-list pages, for paginated novels — see [Pagination](#pagination) | --- @@ -455,7 +512,7 @@ here as they do there. | path | `string` | yes | | | releaseTime | `string` | no | `"YYYY-MM-DD"` or an ISO date string | | chapterNumber | `number` | no | | -| page | `string` | no | Only used for novels without pages (see `SourcePage`/`PagePlugin`) | +| page | `string` | no | Only used for novels without pages (see [Pagination](#pagination)) | | scanlator | `string` or `string[]` | no | Name(s) of the scanlation/translation group(s) | ### Filters @@ -514,15 +571,16 @@ filters = { ##### Filter types -Types of filters supported +Types of filters supported. The `FilterTypes` enum values shown below are also the strings used to +serialize each filter's `type` (e.g. `FilterTypes.CheckboxGroup === 'Checkbox'`). -| FilterType | Description | `value` | `options` | -| ------------------------- | ------------------------------------------------------------------ | ---------------------------------------------------------------------------- | ----------------------------------------------- | -| `Picker` | A spinner for choosing one of the choices provided in `options` | `string` the picked value | [Picker](#picker-options) options | -| `TextInput` | A filter allowing a free text input | `string` written value | N/A | -| `Switch` | A boolean switch | `boolean` state of the switch | N/A | -| `CheckboxGroup` | A grouping of checkboxes | `string[]` array containing selected values | [CheckboxGroup](#checkboxgroup-options) options | -| `ExcludableCheckboxGroup` | A grouping of checkboxes where each one can be marked as included or excluded (e.g. "must have this genre" vs. "must not have this genre") | [ExcludableCheckboxGroupValue](#excludablecheckboxgroupvalue-object) object | [CheckboxGroup](#checkboxgroup-options) options | +| FilterType | Serialized as | Description | `value` | `options` | +| -------------------------- | -------------- | ------------------------------------------------------------------ | ---------------------------------------------------------------------------- | ----------------------------------------------- | +| `Picker` | `'Picker'` | A spinner for choosing one of the choices provided in `options` | `string` the picked value | [Picker](#picker-options) options | +| `TextInput` | `'Text'` | A filter allowing a free text input | `string` written value | N/A | +| `Switch` | `'Switch'` | A boolean switch | `boolean` state of the switch | N/A | +| `CheckboxGroup` | `'Checkbox'` | A grouping of checkboxes | `string[]` array containing selected values | [CheckboxGroup](#checkboxgroup-options) options | +| `ExcludableCheckboxGroup` | `'XCheckbox'` | A grouping of checkboxes where each one can be marked as included or excluded (e.g. "must have this genre" vs. "must not have this genre") | [ExcludableCheckboxGroupValue](#excludablecheckboxgroupvalue-object) object | [CheckboxGroup](#checkboxgroup-options) options | ###### Picker options @@ -611,20 +669,30 @@ pluginSettings = { ##### Setting Properties -| Name | Type | Required | Description | -| ------- | -------- | -------- | ---------------------------------------------- | -| value | `string` | yes | Default value for this setting | -| label | `string` | yes | Display label shown in the app's settings UI | -| type | `string` | no | Type of the setting UI component (see below) | +The shape of a setting depends on its `type` — see [Setting Types](#setting-types) below for the +`value`/`options` each one requires. + +| Name | Type | Required | Description | +| ------- | ---------------------------------- | -------- | ---------------------------------------------- | +| value | depends on `type`, see below | yes | Default value for this setting | +| label | `string` | yes | Display label shown in the app's settings UI | +| type | `'Text' \| 'Switch' \| 'Select' \| 'CheckboxGroup'` | no | Type of the setting UI component (see below) | +| options | `{ label: string; value: string }[]` | for `Select`/`CheckboxGroup` | The choices shown for that setting | ##### Setting Types -Currently, two setting types are supported: +Four setting types are supported: + +| Type | Description | UI Component | `value` type | +| ---------------- | -------------------------------------------------- | ------------- | ---------------------------------------- | +| `Text` | A text input field (default if type is omitted) | TextInput | `string` | +| `Switch` | A boolean toggle switch | SwitchItem | `boolean` | +| `Select` | A single choice from a dropdown menu | Menu | `string` — must match one option's `value` | +| `CheckboxGroup` | Multiple choices toggled independently | Checkbox list | `string[]` — the selected option values | -| Type | Description | UI Component | Default Value Type | -| -------- | ---------------------------------------------- | ------------ | ------------------ | -| `Switch` | A boolean toggle switch | SwitchItem | `boolean` | -| `Text` | A text input field (default if type is omitted) | TextInput | `string` | +`Select` and `CheckboxGroup` also require an `options: { label: string; value: string }[]` array, +the same shape as [Picker options](#picker-options)/[CheckboxGroup options](#checkboxgroup-options) +for filters. > [!NOTE] > If `type` is not specified, the setting defaults to `Text` type and will be rendered as a TextInput. @@ -643,6 +711,14 @@ const settingValue = storage.get('settingKey'); storage.set('settingKey', 'newValue'); ``` +> [!WARNING] +> The settings screen writes directly to `storage` — it does not reload or re-instantiate your +> plugin. If you read a setting as a **class-field initializer** (as in the examples below), that +> field is only evaluated once, when the plugin is loaded, so it won't reflect a value the user +> changes afterwards until the plugin is reloaded (e.g. on app restart or plugin update). If your +> plugin needs to react to a changed setting immediately, call `storage.get('settingKey')` **inside** +> the method that needs it instead of caching it in a field. + ##### Examples ###### Example 1: Switch Setting @@ -654,7 +730,7 @@ class ExamplePlugin implements Plugin.PluginBase { pluginSettings = { hideLocked: { - value: '', + value: false, label: 'Hide locked chapters', type: 'Switch', }, @@ -710,6 +786,35 @@ class ExamplePlugin implements Plugin.PluginBase { } ``` +###### Example 3: Select and CheckboxGroup Settings + +```ts +class ExamplePlugin implements Plugin.PluginBase { + ... + pluginSettings = { + quality: { + value: 'high', + label: 'Image quality', + type: 'Select', + options: [ + { label: 'Low', value: 'low' }, + { label: 'High', value: 'high' }, + ], + }, + excludedTags: { + value: [], + label: 'Excluded tags', + type: 'CheckboxGroup', + options: [ + { label: 'Mature', value: 'mature' }, + { label: 'Adaptation', value: 'adaptation' }, + ], + }, + }; + ... +} +``` + --- ### NovelStatus @@ -734,9 +839,66 @@ import { NovelStatus } from '@libs/novelStatus'; | `STUB` | `'STUB'` | | `Inactive` | `'Inactive'` | -`status` isn't restricted to these values (it accepts any `string`), but prefer a `NovelStatus` -member whenever the source's status maps onto one — free-text values won't be recognized by the -app's status filter. +`status` is typed as `NovelStatus`, not a free-form string — always assign one of the members +above (there's no fallback for other strings; pick `Unknown` if the source's status doesn't map +onto any of them). + +--- + +### Pagination + +Some sites split a novel's chapter list across multiple pages rather than returning it all from +`parseNovel`. For those, implement `parsePage` in addition to `parseNovel`: + +```ts +parsePage?(novelPath: string, page: string): Promise; +``` + +- `SourceNovel::chapters` should hold the **first page** of chapters, and + `SourceNovel::totalPages` should be set to the total number of pages. +- `parsePage` is called with the same `novelPath` and a `page` string (`ChapterItem::page`, if you + set it) for every subsequent page the app needs, and should return that page's chapters: + +```ts +type SourcePage = { + chapters: Plugin.ChapterItem[]; +}; +``` + +###### Example + +```ts +class ExamplePlugin implements Plugin.PluginBase { + ... + async parseNovel(novelPath: string): Promise { + const novel: Plugin.SourceNovel = { + path: novelPath, + name: 'test', + chapters: await this.parseChapterListPage(novelPath, '1'), + totalPages: 5, + }; + return novel; + } + + async parsePage( + novelPath: string, + page: string, + ): Promise { + return { chapters: await this.parseChapterListPage(novelPath, page) }; + } + + private async parseChapterListPage( + novelPath: string, + page: string, + ): Promise { + // fetch and parse the given page's chapter list + return []; + } +} +``` + +If your plugin doesn't paginate chapter lists, omit `parsePage`/`totalPages` entirely and just +return the full list from `parseNovel`. --- @@ -774,6 +936,44 @@ async popularNovels(page: number): Promise { } ``` +A similar pattern for `parseNovel`, pulling structured fields (author, genres, status) plus a +chapter list off the novel page: + +```ts +async parseNovel(novelPath: string): Promise { + const body = await fetchApi(this.site + novelPath).then(res => res.text()); + const $ = parseHTML(body); + + const novel: Plugin.SourceNovel = { + path: novelPath, + name: $('h1.novel-title').text().trim(), + cover: $('.novel-cover img').attr('src'), + author: $('.novel-author').text().trim(), + genres: $('.novel-genres a') + .map((i, el) => $(el).text().trim()) + .get() + .join(','), + summary: $('.novel-summary').text().trim(), + status: $('.novel-status').text().includes('Ongoing') + ? NovelStatus.Ongoing + : NovelStatus.Completed, + chapters: [], + }; + + $('ul.chapter-list li a').each((i, el) => { + const chapterPath = $(el).attr('href')?.replace(this.site, ''); + if (!chapterPath) return; + novel.chapters.push({ + name: $(el).text().trim(), + path: chapterPath, + chapterNumber: i + 1, + }); + }); + + return novel; +} +``` + Notes: - `$(el)` re-scopes a selector to a single element found by `.each()`; without it you'd search the @@ -781,6 +981,8 @@ Notes: - `path` should be relative (strip `this.site`/the domain) — see [NovelItem::path](#novelitempath). - Prefer `.attr('href')` / `.attr('src')` over `.text()` for links and images, and always guard for `undefined` since a selector can fail to match if the site changes its markup. +- `.map((i, el) => ...).get()` is Cheerio's way of turning a selection into a plain array — `.get()` + is required, a bare `.map()` returns a Cheerio object, not an array. See the [Cheerio API docs](https://cheerio.js.org/docs/api) for the full set of selectors/methods (`.find()`, `.first()`, `.eq()`, `.attr()`, `.text()`, `.html()`, etc.), and look at existing @@ -791,7 +993,7 @@ plugins under `plugins/**` for real examples. ### Custom fetching functions Plugins can't use the browser/Node `fetch` directly — use the wrappers from `@libs/fetch` instead, -which handle plugin-specific request setup (proxying, headers, etc.): +which handle plugin-specific request setup (default headers, etc.): ```ts import { fetchApi, fetchText, fetchProto } from '@libs/fetch'; @@ -806,6 +1008,20 @@ declare function fetchApi(url: string, init?: FetchInit): Promise; The general-purpose fetcher. Returns a standard `Response`, so use `.text()`, `.json()`, etc. on the result, the same way you would with the native `fetch`. +Every request automatically gets these default headers, merged under whatever you pass in +`init.headers` (a header you set yourself takes priority over the default of the same name): + +- `User-Agent` — the app's configured user agent +- `Connection: keep-alive` +- `Accept: */*` +- `Accept-Language: *` +- `Accept-Encoding: gzip, deflate` +- `Sec-Fetch-Mode: cors` +- `Cache-Control: max-age=0` + +You don't need to set these yourself; only pass headers the site actually requires beyond the +defaults (`Referer`, `Authorization`, `Cookie`, etc.). + ```ts const res = await fetchApi(this.resolveUrl(novelPath)); const body = await res.text(); @@ -824,6 +1040,19 @@ declare function fetchText( A shortcut for `fetchApi(...).then(res => res.text())`, with an optional `encoding` for sites that don't serve UTF-8 (e.g. `fetchText(url, undefined, 'gbk')` for some Chinese-language sites). +> [!WARNING] +> `fetchText` never throws. If the request fails (network error) or the response isn't `ok` (e.g. +> a 404/500), it silently returns an **empty string** `''` instead of rejecting. Check for an +> empty result before parsing it if the site's availability can't be assumed: +> +> ```ts +> const body = await fetchText(url); +> if (!body) { +> // request failed or returned no content — bail out instead of parsing '' +> return novels; +> } +> ``` + #### fetchProto ```ts @@ -870,8 +1099,25 @@ It mirrors the standard [`fetch` init object](https://developer.mozilla.org/en-U ### Other libraries -A few smaller helpers are available for less common cases. You generally won't need these unless -your target site requires them. +A few smaller helpers and runtime-provided packages are available for less common cases. You +generally won't need most of these unless your target site requires them. Everything listed here +is what's actually injected into a running plugin's `require(...)` calls — anything else you +`import` won't resolve at runtime even if it type-checks locally. + +| Package | Exposes | Typical use | +| ------------------- | -------------------------------------------- | ---------------------------------------------------------------- | +| `cheerio` | `load` | HTML parsing — see [Using Cheerio](#using-cheerio) | +| `@libs/fetch` | `fetchApi`, `fetchText`, `fetchProto` | Network requests — see [Custom fetching functions](#custom-fetching-functions) | +| `dayjs` | the `dayjs` default export | Parsing/formatting relative or oddly-formatted release dates | +| `urlencode` | `encode`, `decode` | Percent-encoding for URLs/query params, with non-UTF-8 charset support | +| `htmlparser2` | `Parser` | Low-level streaming HTML/XML parsing, for pages too large or malformed for Cheerio | +| `@libs/storage` | `storage`, `localStorage`, `sessionStorage` | Persistent key-value storage — see [storage](#storage-localstorage--sessionstorage) below | +| `@libs/isAbsoluteUrl` | `isUrlAbsolute` | See below | +| `@libs/filterInputs` | `FilterTypes`, `Filters`, ... | See [Filters](#filters) | +| `@libs/novelStatus` | `NovelStatus` | See [NovelStatus](#novelstatus) | +| `@libs/defaultCover` | `defaultCover` | See [Default cover](#default-cover) | +| `@libs/aes` | `gcm` | See [AES decryption](#aes-decryption) below | +| `@libs/utils` | `utf8ToBytes`, `bytesToUtf8` | See [AES decryption](#aes-decryption) below | #### isUrlAbsolute @@ -893,10 +1139,22 @@ import { storage, localStorage, sessionStorage } from '@libs/storage'; `storage` is the same persistent key-value store used for [plugin settings](#pluginsettings) — you can also use it directly for things like caching a session cookie or an auth token between -requests. `localStorage`/`sessionStorage` are separate, lower-level stores for plugins that need -that exact browser-style API (for example, reusing scraping code shared with a web target). If -your plugin uses either of them, set [`webStorageUtilized`](#pluginbasewebstorageutilized) to -`true` on the plugin so the app knows to provide that access. +requests. It supports more than a plain get/set pair: + +| Method | Description | +| ------------------------------------ | ----------------------------------------------------------------------------- | +| `storage.set(key, value, expires?)` | Stores `value` under `key`; `expires` is an optional `Date` or epoch-millisecond timestamp after which the entry is treated as gone | +| `storage.get(key, raw?)` | Returns the stored value, or `undefined` if missing/expired. Pass `raw: true` to get back `{ created, value, expires }` instead of just `value` | +| `storage.delete(key)` | Removes a single key | +| `storage.clearAll()` | Removes every key this plugin has stored | +| `storage.getAllKeys()` | Returns all keys currently set by this plugin | + +`localStorage`/`sessionStorage` are separate, lower-level stores for plugins that need that exact +browser-style API (for example, reusing scraping code shared with a web target) — but unlike +`storage`, they are **read-only** from plugin code: each only exposes `get()`, with no `set()`. +They're populated by the app's own WebView integration, not written by your plugin. If your +plugin uses either of them, set [`webStorageUtilized`](#pluginbasewebstorageutilized) to `true` on +the plugin so the app knows to provide that access. #### AES decryption @@ -916,3 +1174,65 @@ const plaintext = bytesToUtf8(cipher.decrypt(ciphertextBytes)); This is an advanced case — only needed if you've confirmed the site is actually encrypting its payloads, not just minifying/obfuscating them. + +--- + +## Repository manifest & install metadata + +The plugin bundle API above is what your `.ts` file implements. Separately, this repository +publishes a **manifest** (`plugins.json`) listing every plugin, which is what the LNReader app +actually reads to show, install, and update plugins from this repo. The manifest entry for a +plugin is built automatically from your class's fields — you don't write it by hand — but it's +useful to know its shape, since it's what the app sees, not your source file directly. + +```ts +// app-side manifest entry shape +type PluginItem = { + id: string; + name: string; + site: string; + lang: string; // full language name, e.g. "English" + version: string; + url: string; // raw URL to the compiled plugin's JS, which the app downloads and runs + iconUrl: string; // absolute URL to the icon, always present (falls back to a placeholder) + customJS?: string; // absolute URL to the custom JS file, if any + customCSS?: string; // absolute URL to the custom CSS file, if any + hasUpdate?: boolean; + hasSettings?: boolean; +}; +``` + +`lang`, `url`, and `iconUrl` are always required on every manifest entry; the app relies on all +three being present to list and install a plugin. + +### How manifest fields are produced + +`npm run build:manifest` (`scripts/build-plugin-manifest.js`) compiles every plugin under +`plugins/**`, evaluates its default export, and reads off `id`, `name`, `site`, `version`, `icon`, +`customJS`, `customCSS`, and `filters` to build each `plugins.json` entry: + +- `iconUrl` is built from your class's [`icon`](#pluginbaseicon) field: `icon` (or + `siteNotAvailable.png` if unset) is appended to this repo's `public/static` raw-content URL for + the current branch. +- `customJS`/`customCSS` are built the same way from your class's + [`customJS`](#pluginbasecustomjs)/[`customCSS`](#pluginbasecustomcss) fields, when set — omitted + from the manifest entirely if you didn't set them. +- `url` points at the compiled JS for your plugin, not your `.ts` source. + +`scripts/download-plugin-icons.js` separately fetches/validates the icon file that +`iconUrl` will point to (falling back to the site's favicon when you haven't committed one) and +prunes any `public/static` assets no manifest entry references any more. + +### Install flow (app side) + +When the app installs or updates a plugin from a repository's manifest, it: + +1. Downloads the compiled JS from the entry's `url` and evaluates it as the plugin. +2. If the entry has a `customJS` and/or `customCSS` URL, downloads each into the plugin's own + private on-device storage (not `public/static` — that's this repo's hosting location, not + where the installed copy ends up) alongside the plugin's compiled code. +3. Displays the plugin using `iconUrl` directly — the app never resolves an `icon` path itself. + +In other words: `icon`/`customJS`/`customCSS` on your class are source-repo-relative paths that +exist only so the build step above can turn them into the absolute, downloadable URLs +(`iconUrl`/`customJS`/`customCSS`) the manifest — and therefore the app — actually uses. From f1d7476a5ea58cd49b22fc0d44a3fbe7876f2278 Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 7 Sep 2026 15:51:46 +0000 Subject: [PATCH 2/2] docs: fix drift in testing.md and website-tutorial.md Verified against .github/workflows/plugin-live-check.yml, scripts/live-check-plugin.js, and the playground UI (src/pages/home.tsx and friends): - testing.md: the live-check workflow posts a job/run summary (GITHUB_STEP_SUMMARY), not a PR comment. - website-tutorial.md: the playground's five tabs are actually Popular/Search/Parse Novel/Parse Chapter/Settings, not the documented Headers/Popular Novels/Search Novels/Parse Novel/Parse Chapter. There's no per-request "Headers" tab; Settings configures playground-wide fetch mode/User-Agent/cookies. Also documented two previously-unmentioned Parse Novel features: parsePage/totalPages pagination controls and the Export EPUB button. quickstart.md and komga-plugin.md were audited and found accurate; no changes made to those. Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_01LoFgTFvjNVdrUbZVjdFuxf --- docs/testing.md | 7 ++++--- docs/website-tutorial.md | 24 ++++++++++++++++-------- 2 files changed, 20 insertions(+), 11 deletions(-) diff --git a/docs/testing.md b/docs/testing.md index fcac60c38..37136b95c 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -35,9 +35,10 @@ Each step reports one of three outcomes: ## CI Any PR that touches a file under `plugins/**/*.ts` (excluding multisrc-generated files) runs this -same check automatically against just the changed plugins, and posts a summary comment. The check -only fails the PR on a genuine `FAIL` — `INCONCLUSIVE` results (a site being briefly down) never -block a merge. +same check automatically against just the changed plugins, and posts the results as a job summary +on the workflow run (visible from the PR's checks list, under the Actions tab) — not as a PR +comment. The check only fails the PR on a genuine `FAIL` — `INCONCLUSIVE` results (a site being +briefly down) never block a merge. You can also trigger it manually against any plugin path from the Actions tab (`Plugin Live Check` → `Run workflow`), which is useful for re-checking an existing plugin after diff --git a/docs/website-tutorial.md b/docs/website-tutorial.md index f3750f91d..1a59d5e61 100644 --- a/docs/website-tutorial.md +++ b/docs/website-tutorial.md @@ -18,20 +18,28 @@ A comprehensive guide to testing your LNReader plugins using the web interface. ## Features Overview -The testing website provides five main sections to test different plugin functions: - -- **Headers** - Configure custom HTTP headers -- **Popular Novels** - Test `popularNovels()` with pagination and filters -- **Search Novels** - Test `searchNovels()` with search queries -- **Parse Novel** - Test `parseNovel()` with a novel path +The testing website provides five tabs to test different plugin functions: + +- **Popular** - Test `popularNovels()`, including its `Latest`/`Popular` toggle, page-by-page + fetching, and filters (via the `Filters` button, when the plugin declares any) +- **Search** - Test `searchNovels()` with search queries +- **Parse Novel** - Test `parseNovel()` with a novel path. If the plugin sets `totalPages` (see + [Pagination](./docs.md#pagination)), this tab also exercises `parsePage()` through + Previous/Next/Fetch Page controls, and offers an "Export EPUB" button that fetches every + page's chapters (via `parsePage`, when paginated) and each one's content (via `parseChapter`) + into a downloadable EPUB file - **Parse Chapter** - Test `parseChapter()` with a chapter path +- **Settings** - Playground-wide request configuration: the browser User-Agent (and whether to + send it), extra cookies to attach to every request, and the fetch mode (Proxy/Node + Fetch/Curl) used to reach the target site. This is a testing convenience for the playground + itself, separate from a plugin's own [`pluginSettings`](./docs.md#pluginsettings) ## Pre-Submission Testing Before submitting your plugin, verify that: -- All five sections work without errors -- Multiple pages load correctly +- All five tabs work without errors +- Multiple pages load correctly, for both `popularNovels` and, if implemented, `parsePage` - Search returns accurate results - Novel parsing extracts all metadata - Chapter content is clean