Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
aef97d6
refactor!: replace configs gem with registry index
gildesmarais Aug 22, 2026
f103414
feat(registry): sync CLI, seed bundle, and background refresh
gildesmarais Aug 22, 2026
5725ba0
docs: registry sync runbook and catalog API ownership
gildesmarais Aug 22, 2026
a440551
docs: add registry go-live manual
gildesmarais Aug 22, 2026
22da0da
fix(deps): pin html2rss from GitHub for CI
gildesmarais Aug 22, 2026
c61e567
feat(registry): sync governance with staged promote and trust context
gildesmarais Aug 22, 2026
4ba7386
chore(deps): pin html2rss to registry hardening commit
gildesmarais Aug 22, 2026
8ff3bfa
fix(ci): use SimpleCov add_filter and git html2rss pin
gildesmarais Aug 22, 2026
45dc4f5
refactor(registry): fold sync satellites into sync_transport
gildesmarais Aug 22, 2026
0cfeb9c
refactor(registry): fold catalog wire and load guards into index
gildesmarais Aug 22, 2026
043fc7e
refactor(registry): centralize manifest I/O and dedup test signing
gildesmarais Aug 22, 2026
076964e
chore(registry): dedup fixtures and sync docs
gildesmarais Aug 22, 2026
98d6ccb
refactor(ruby4): apply leading operators and it blocks on registry br…
gildesmarais Aug 22, 2026
d82231e
refactor(registry): dedupe transport, status lookup, and deep_dup
gildesmarais Aug 22, 2026
a0810a0
perf(registry): use Set for catalog diff and tighten hot paths
gildesmarais Aug 22, 2026
60adb24
test(registry): drop send pins and table-drive transport specs
gildesmarais Aug 22, 2026
f6756f7
refactor(api): modernize configs and root_metadata helpers
gildesmarais Aug 22, 2026
1261dcf
fix(registry): restore rubocop disables and constant-cache memoization
gildesmarais Aug 22, 2026
c9c8dec
docs: codify Ruby 4.0 style for agents and contributors
gildesmarais Aug 22, 2026
77c9abd
refactor(registry): modernize registry subsystem, unify types, and do…
gildesmarais Aug 23, 2026
468dc5d
refactor(registry): streamline registry modules, unify StructuredData…
gildesmarais Aug 23, 2026
63066d9
refactor(registry): map catalog wire fields at Index edge
gildesmarais Aug 23, 2026
0830b30
chore(deps): pin html2rss Gemfile.lock to registry-v1 tip
gildesmarais Aug 23, 2026
9fcb521
fix(registry): implement cli_exit_code, atomic store swap, index fres…
gildesmarais Aug 24, 2026
f30d1d6
feat(cli): add unified operator CLI and embed 144 curated seed configs
gildesmarais Aug 24, 2026
8c1dc70
docs(compose): add registry-sync service and compose operator workflow
gildesmarais Aug 24, 2026
a8950b6
chore(compose): remove legacy watchtower service
gildesmarais Aug 24, 2026
63c06d0
refactor(registry): implement embedded bundle lookup and eliminate se…
gildesmarais Aug 24, 2026
2a1e9e5
feat(docker): bake official registry artifact at build time and drop …
gildesmarais Aug 24, 2026
b36b10c
docs(registry): align terminology and document embedded registry deli…
gildesmarais Aug 24, 2026
25ff6a1
feat(docker): verify official registry signature and digests at build…
gildesmarais Aug 24, 2026
14d0315
feat(cli): add registry verify subcommand and invoke in Docker build
gildesmarais Aug 24, 2026
7f2e301
chore(bundle): update html2rss git revision in Gemfile.lock
gildesmarais Aug 24, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -60,3 +60,7 @@ frontend/.astro

# Bundler/vendored dependencies
/vendor/

# Local bundle builds
/dist/

5 changes: 5 additions & 0 deletions .rubocop.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ plugins:
- rubocop-thread_safety

AllCops:
TargetRubyVersion: 4.0
DisplayCopNames: true
NewCops: enable
Exclude:
Expand Down Expand Up @@ -38,6 +39,10 @@ Style/Documentation:
AllowedConstants:
- App

Style/ItBlockParameter:
Enabled: true
EnforcedStyle: allow_single_line

RSpec/SpecFilePathFormat:
Exclude:
- 'spec/html2rss/web/app/*_spec.rb'
Expand Down
50 changes: 46 additions & 4 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,21 +54,63 @@ See [docs/design-system.md](docs/design-system.md) for visual rules.
- **No host execution:** All commands MUST run inside the Dev Container via `make` or `bundle exec`.
- **No skipped quality gate:** Opening a PR without a green Dev Container gate is forbidden. If the gate cannot run, do not open the PR; fix the environment or hand off with explicit blocker + next command for the user.

## Ruby 4 Style

**Ruby 4.0+ only** (see `.tool-versions`). No Ruby 3.x backward-compat shims, guards, or dual-path APIs.

### Baseline

- `# frozen_string_literal: true` on every `.rb` file
- Plain Ruby — no ActiveSupport
- Keyword arguments for public multi-arg APIs
- Typed YARD on public methods in `app/` (`@param`, `@return`) — enforced by `make yard-verify-public-docs`

### Modern syntax (prefer consistently)

| Idiom | Use instead of |
| --- | --- |
| Leading `&&` / `\|\|` at line start (Ruby 4) | Trailing operators on long wrapped conditions |
| `it` in single-parameter blocks | `{ \|x\| x.foo }` when block has one arg only |
| Pattern matching (`in`, `case … in`) | Deep `if/elsif` chains on shape |
| `Data.define` | OpenStruct / hand-rolled structs |
| `filter_map`, `index_by`, `then`, `match?` | Verbose `map`/`compact`, nested `if`, `=~` |
| Endless `def` | One-line pure helpers when RuboCop allows |
| Core `Set` (no `require 'set'`) | Array membership/diff on growing collections |

### Performance (agent defaults)

- **Set** for catalog/diff/membership when sizes can grow
- **Memoize** repeated `ENV.fetch` / pure computations on hot paths
- **One owner** for duplicated helpers — dedupe before splitting into new files
- **Functional iterators** over imperative loops
- **No metric-driven micro-methods** whose only purpose is satisfying RuboCop metrics
- Do **not** document ZJIT/Ruby Box/Ractor as defaults

### Web-specific deltas

- Prefer `class << self` + `private` over `module_function` (see docs/README Architectural Constraints)
- Do not use `send(...)` to reach private APIs in app code or specs
- Specs: table-drive matrices; `:aggregate_failures` for discriminating multi-assert examples
- LOC: dedupe/unify before extracting — new files only when they buy a real seam or test surface

## Config catalog API

Public feed-directory metadata for embedded and local configs.
Public feed-directory metadata from verified registry bundles and local `feeds.yml` entries.

| Item | Detail |
| --- | --- |
| Endpoint | `GET /api/v1/configs` |
| Flag | `CONFIG_CATALOG_ENABLED` (default `true`; set `false` to disable) |
| Disabled response | `404` with `{ "error": "catalog_disabled" }` |
| Embedded entries | `Html2rss::Configs::Catalog.entries` — do not re-walk YAML in the handler |
| Local entries | `Catalog::Merge` includes `feeds.yml` feeds only when `directory.title` is set |
| Registry entries | `Registry::Index.current.catalog_rows` — loads signed bundles from `config/registries.yml`; adds `source: registry`, `registry: <id>` |
| Local entries | `Registry::Index` catalog rows include `feeds.yml` feeds only when `directory.title` is set (`source: local`) |
| Per-registry privacy | `catalog: false` in `registries.yml` omits that registry from the API (feeds still served) |
| Starter feeds (UI) | Frontend `selectStarterFeeds` when feed creation is disabled; catalog find uses full catalog when enabled |
| Catalog find | `findCatalogEntries` → multi-hit list under create URL; links via `catalogFeedHref` (path + defaults) |
| CORS | Route-scoped on `/api/v1/configs` only (`GET`, `OPTIONS`) |
| Root metadata | `GET /api/v1/` exposes `instance.catalog: { enabled, url }` |
| Root metadata | `GET /api/v1/` exposes `instance.catalog: { enabled, url }` and `instance.registries` sync status |
| Contract SSOT | Request specs under `spec/html2rss/web/api/v1_spec.rb` and generated `public/openapi.yaml` |

Registry sync: `bin/html2rss-web registry status`; embedded bundle + optional runtime sync via `Registry::Sync.boot!`. See [docs/README.md](docs/README.md#registry-sync-runbook).

After handler or envelope changes: `make openapi` and `make ci-ready`.
9 changes: 9 additions & 0 deletions CONTEXT.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,3 +54,12 @@ Audit channel: snake_case `security_event` with IP / user-agent / token hash. Au

### LogEvent
Shared emit plumbing for both channels (`RequestContext`, `LogSanitizer`, `AppLogger` / Sentry). Not a third public facade.

### Registry Index
Backend merge owner for registry bundles and local `feeds.yml` feeds. Builds catalog wire rows (`Registry::Index::CatalogRow`), enforces load-time trust and channel-domain allowlists, and serves `config_for` / `catalog_rows` / `status`.

### Registry Sync
Backend orchestration for fetch → verify → stage/promote of signed registry bundles. Owns boot initialization, background refresh, CLI exit codes, and catalog-change telemetry after promotion.

### Sync Transport
Backend HTTPS fetch, sync URL resolution (GitHub releases and channel defaults), and manifest version gating used by `Registry::Sync` and parse-time config resolution.
28 changes: 25 additions & 3 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,26 @@ RUN apk add --no-cache \
/usr/local/bundle/bundler/gems/*/.git \
/usr/local/bundle/cache/bundler/git

# Stage 3: Runtime
# Stage 3: Official Registry Artifact Builder & Verifier
FROM ${RUBY_BASE_IMAGE} AS registry-builder

ARG REGISTRY_BUNDLE_URL="https://github.com/html2rss/html2rss-configs/releases/latest/download/registry-bundle.tar.gz"

WORKDIR /app

COPY --from=builder /usr/local/bundle /usr/local/bundle
COPY bin ./bin
COPY app ./app
COPY config ./config

# hadolint ignore=DL3018
RUN apk add --no-cache curl tar ca-certificates \
&& curl -fsSL -o registry-bundle.tar.gz "${REGISTRY_BUNDLE_URL}" \
&& mkdir -p /build/official \
&& tar -xzf registry-bundle.tar.gz -C /build/official \
&& bin/html2rss-web registry verify --registry official --dir /build/official

# Stage 4: Runtime
FROM ${RUBY_BASE_IMAGE}

LABEL maintainer="Gil Desmarais <html2rss-web-docker@desmarais.de>"
Expand All @@ -51,13 +70,14 @@ ARG GIT_SHA
ENV PORT=4000 \
RACK_ENV=production \
RUBY_YJIT_ENABLE=1 \
PATH="/app/bin:$PATH" \
BUILD_TAG=${BUILD_TAG} \
GIT_SHA=${GIT_SHA}

EXPOSE $PORT

HEALTHCHECK --interval=30m --timeout=60s --start-period=5s \
CMD ["/app/bin/docker-healthcheck"]
CMD ["/app/bin/html2rss-web", "healthcheck"]

ARG USER=html2rss
ARG UID=991
Expand All @@ -78,17 +98,19 @@ RUN apk add --no-cache \
&& mkdir -p /app \
&& mkdir -p /app/tmp/rack-cache-body \
&& mkdir -p /app/tmp/rack-cache-meta \
&& mkdir -p /app/data/registries \
&& chown "$USER":"$USER" -R /app

WORKDIR /app

USER 991

COPY --from=builder /usr/local/bundle /usr/local/bundle
COPY --chown=$USER:$USER bin/docker-healthcheck ./bin/docker-healthcheck
COPY --chown=$USER:$USER bin ./bin
COPY --chown=$USER:$USER Gemfile Gemfile.lock app.rb config.ru ./
COPY --chown=$USER:$USER app ./app
COPY --chown=$USER:$USER config ./config
COPY --from=registry-builder --chown=$USER:$USER /build/official ./registries/official
COPY --chown=$USER:$USER public ./public
COPY --from=frontend-builder --chown=$USER:$USER /app/frontend/dist ./frontend/dist

Expand Down
9 changes: 2 additions & 7 deletions Gemfile
Original file line number Diff line number Diff line change
Expand Up @@ -4,13 +4,8 @@ source 'https://rubygems.org'

git_source(:github) { |repo_name| "https://github.com/#{repo_name}" }

gem 'html2rss', '~> 0.27'
# gem 'html2rss', github: 'html2rss/html2rss', branch: 'master'
gem 'html2rss-configs', github: 'html2rss/html2rss-configs'

# Use these instead of the two above (uncomment them) when developing locally:
# gem 'html2rss', path: '../html2rss'
# gem 'html2rss-configs', path: '../html2rss-configs'
# Until rubygems 0.28.0: git branch; local monorepo: BUNDLE_LOCAL__HTML2RSS=/path/to/html2rss
gem 'html2rss', github: 'html2rss/html2rss', branch: 'feat/registry-v1'

gem 'base64'
gem 'rack-cache'
Expand Down
55 changes: 26 additions & 29 deletions Gemfile.lock
Original file line number Diff line number Diff line change
@@ -1,9 +1,29 @@
GIT
remote: https://github.com/html2rss/html2rss-configs
revision: f888a8dc5bb260c998c5ae8f39bb7d45320638ee
remote: https://github.com/html2rss/html2rss
revision: 5044272b9dcf404a1ef550e0496508cd6ce5687e
branch: feat/registry-v1
specs:
html2rss-configs (0.2.0)
html2rss
html2rss (0.28.0)
addressable (~> 2.7)
brotli
dry-validation
faraday (> 2.0.1, < 3.0)
faraday-follow_redirects
faraday-gzip (~> 3)
kramdown
mcp (~> 1.2)
mime-types (> 3.0)
nokogiri (>= 1.10, < 2.0)
rack (~> 3.0)
rackup (~> 2.0)
regexp_parser
reverse_markdown (~> 3.0)
rss
sanitize
thor
tzinfo
webrick (~> 1.9)
zeitwerk

GEM
remote: https://rubygems.org/
Expand Down Expand Up @@ -103,27 +123,6 @@ GEM
net-http (~> 0.5)
hana (1.3.7)
hashdiff (1.2.1)
html2rss (0.27.2)
addressable (~> 2.7)
brotli
dry-validation
faraday (> 2.0.1, < 3.0)
faraday-follow_redirects
faraday-gzip (~> 3)
kramdown
mcp (~> 1.2)
mime-types (> 3.0)
nokogiri (>= 1.10, < 2.0)
rack (~> 3.0)
rackup (~> 2.0)
regexp_parser
reverse_markdown (~> 3.0)
rss
sanitize
thor
tzinfo
webrick (~> 1.9)
zeitwerk
i18n (1.15.2)
concurrent-ruby (~> 1.0)
io-console (0.9.2)
Expand Down Expand Up @@ -319,8 +318,7 @@ PLATFORMS
DEPENDENCIES
base64
climate_control
html2rss (~> 0.27)
html2rss-configs!
html2rss!
irb
puma
rack-cache
Expand Down Expand Up @@ -377,8 +375,7 @@ CHECKSUMS
faraday-net_http (3.4.4) sha256=0e78af151747ed1b00f33e25973b4bc220d7f16c00c39676817c8b12331eb588
hana (1.3.7) sha256=5425db42d651fea08859811c29d20446f16af196308162894db208cac5ce9b0d
hashdiff (1.2.1) sha256=9c079dbc513dfc8833ab59c0c2d8f230fa28499cc5efb4b8dd276cf931457cd1
html2rss (0.27.2) sha256=82b28308c023cb669704bf9b0612aa2a00730be4b53d88f957d456ba5e631ac8
html2rss-configs (0.2.0)
html2rss (0.28.0)
i18n (1.15.2) sha256=00f9eb62412fe593b2a65a97daa75300d37abb8f7202ec748e94b6d46a9dd1b5
io-console (0.9.2) sha256=efa74f891dd03c0939a931dfc6e74c2813d904763d456ea9762b0525e748db08
irb (1.18.0) sha256=de9454a0703a54704b9811a5ef31a60c86949fbf4013fcf244fabc7c775248e3
Expand Down
12 changes: 8 additions & 4 deletions app/web/api/v1/configs.rb
Original file line number Diff line number Diff line change
Expand Up @@ -17,20 +17,24 @@ def index(_router)
entries, duration_ms = build_entries
emit_success(entries.size, duration_ms)
success_payload(entries)
rescue Html2rss::Configs::Catalog::MissingDirectoryTitle => error
rescue Html2rss::Registry::CatalogBuilder::MissingDirectoryTitle => error
emit_failure(error)
raise
end

private

def build_entries
started = Process.clock_gettime(Process::CLOCK_MONOTONIC)
entries = Html2rss::Web::Catalog::Merge.call
duration_ms = ((Process.clock_gettime(Process::CLOCK_MONOTONIC) - started) * 1000).round
started = monotonic_now
entries = Registry::Index.current.catalog_rows
duration_ms = ((monotonic_now - started) * 1000).round
[entries, duration_ms]
end

def monotonic_now
Process.clock_gettime(Process::CLOCK_MONOTONIC)
end

def emit_success(count, duration_ms)
Observability.emit(
event_name: 'catalog.build',
Expand Down
41 changes: 33 additions & 8 deletions app/web/api/v1/root_metadata.rb
Original file line number Diff line number Diff line change
Expand Up @@ -27,16 +27,41 @@ def build(router)
# @return [Hash{Symbol=>Object}]
def instance_payload(router)
{
feed_creation: {
enabled: Flags.auto_source_enabled?,
access_token_required: Flags.auto_source_enabled?
},
catalog: {
enabled: Flags.config_catalog_enabled?,
url: "#{router.base_url}/api/v1/configs"
}
feed_creation: feed_creation_payload,
catalog: catalog_payload(router),
registries: registry_status_rows
}
end

# @return [Hash{Symbol => Object}]
def feed_creation_payload
auto_source = Flags.auto_source_enabled?
{
enabled: auto_source,
access_token_required: auto_source
}
end

# @param router [Roda::RodaRequest]
# @return [Hash{Symbol => Object}]
def catalog_payload(router)
{
enabled: Flags.config_catalog_enabled?,
url: "#{router.base_url}/api/v1/configs"
}
end

# @return [Array<Hash{Symbol => Object}>]
def registry_status_rows
Registry::Index.current.status.map do |entry|
{
id: entry.id,
version: entry.version,
updated_at: entry.updated_at&.utc&.iso8601,
sync_mode: entry.mode.to_s
}
end
end
end
end
end
Expand Down
5 changes: 4 additions & 1 deletion app/web/boot.rb
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,10 @@ def build_loader
def configure_loader(new_loader)
new_loader.push_dir(app_root, namespace: Html2rss)
collapsed_web_dirs.each { |path| new_loader.collapse(path) }
new_loader.inflector.inflect('api_v1' => 'ApiV1')
new_loader.inflector.inflect(
'api_v1' => 'ApiV1',
'cli' => 'CLI'
)
end

# @return [Array<String>]
Expand Down
12 changes: 10 additions & 2 deletions app/web/boot/setup.rb
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ def call!
configure_request_service!
configure_runtime_logging!
configure_gem_defaults!
configure_registry!
log_startup!
end

Expand All @@ -32,9 +33,11 @@ def call!
# @return [void]
def configure_gem_defaults!
global_config = LocalConfig.global
headers = global_config[:headers]
stylesheets = global_config[:stylesheets]
Html2rss.configure do |config|
config.headers = global_config[:headers] if global_config[:headers]
config.stylesheets = global_config[:stylesheets] if global_config[:stylesheets]
config.headers = headers if headers
config.stylesheets = stylesheets if stylesheets
end
end

Expand Down Expand Up @@ -76,6 +79,11 @@ def configure_runtime_logging!
Rack::Timeout::Logger.logger = AppLogger.logger
end

# @return [void]
def configure_registry!
Registry::Sync.boot!
end

# @return [void]
def log_startup!
AppLogger.logger.info(
Expand Down
Loading
Loading