feat(topics): /topics/ concept-discovery front door (#topics-discovery) - #1572
Merged
Conversation
- srv/lib/build-topics-gallery.js: builds gallery cards + per-cluster detail (ordered concept path, peer cluster edges) from TopicClusters, KgCommunity, ConceptEdges; packet-safe unbounded reads; fail-open - srv/server.js: register GET /build/topics-gallery with Cache-Control - approuter/xs-app.json: add topics-gallery to /build/* allowlist regex - unit + hybrid tests
- hugo/layouts/topics/list.html: card grid from site.Data.topics_gallery.gallery, empty guard, concept chips (up to 4), search box → /search/, id="topics-map" progressive-enhancement island mount, guarded topics-map.js script include - hugo/layouts/topics/single.html: cluster detail via .Params.cluster lookup, breadcrumb (Home / Topics / label), orderMode switch (path→<ol>/ranked→<ul>), /concepts/<slug>/ links, peers section, data-focus-cluster mini-map mount - hugo/assets/css/topics.css: .topics-* namespace, Horizon CSS vars, card grid, chips, breadcrumb, concepts list, peers pills - hugo/content/topics/_index.md: gallery stub (type: topics, layout: list) - hugo/content/topics/cap-fundamentals.md + btp-basics.md: seed stubs for Hugo bake verification (replaced at runtime by fetch-topics-gallery) - hugo/data/topics_gallery.json: hand-seeded fixture (2 clusters) for Hugo template verification; overwritten by npm run fetch-topics-gallery - scripts/fetch-topics-gallery.ts: extend to write per-cluster content stubs after JSON write; deletes stale hugo/content/topics/*.md (except _index.md) - test/unit/hugo/topics-layouts.test.ts: 18 template-source assertions - test/unit/scripts/fetch-topics-gallery.test.ts: 5 tests including stub write, stale removal, front matter shape assertions
- srv/lib/kg-clusters-data.js: pure builder; buildClustersDataPayload
emits c:<slug> super-nodes + deduped inter-cluster edges; buildClusterSubgraph
returns t:<slug> concept nodes for expand-in-place
- srv/lib/build-clusters-data.js: cached Express handler with 5-min TTL,
X-Cache HIT/MISS, Cache-Control: public max-age=300; ?cluster=<slug>
subgraph branch with slug validation (400 on bad input); fail-open 500
- srv/server.js: register app.get('/graph/clusters-data', clustersDataHandler)
alongside explore-data (~line 289)
- approuter/xs-app.json: add clusters-data to the anonymous /graph/... allowlist
alongside explore-data so the island can fetch client-side without auth
- test/unit/srv/kg-clusters-data.test.js: mocked unit test (4 assertions) — PASS
- test/hybrid/clusters-data-hybrid.test.js: HTTP hybrid test — self-skips on
no HANA binding (infra-only failure, no test logic fault)
- Add hugo-apps/src/topics-map/{main.ts,App.vue,ClusterMap.vue}: Vue 3
island that fetches /graph/clusters-data, renders a Sigma 3 super-graph
of topic clusters (nodes sized by size, colored by stable slug→hue hash),
and expands a cluster in-place on click by fetching ?cluster=<slug>.
Includes "See full graph →" link to /explore/?focus=<firstConceptSlug>.
Zero-size guard prevents WebGL errors in jsdom.
- Add sigma@3.0.3, graphology@0.26.0, graphology-layout-forceatlas2@0.10.1
to hugo-apps/package.json (same versions as app/explore).
- Add 'topics-map' rollup input and topicsMapBudget() plugin (150KB gzip)
to hugo-apps/vite.config.ts.
- Add hugo/data/topics_map_bundle.json sentinel so layout script guards pass.
- Tests: test/unit/hugo/topics-map-vite-input.test.ts (config assertion) +
hugo-apps/src/topics-map/App.test.ts (fetch + degrade). Both pass.
- Bundle emits at hugo/static/js/topics-map.js: 42.6KB gzip (budget 150KB).
- App.test.ts: strengthen "degrades quietly" test to assert
.topics-map-island is NOT rendered when fetch rejects (was hollow:
wrapper.exists() passes regardless of failed gate).
- list.html + single.html: remove {{ with site.Data.topics_map_bundle }}
guard; include <script src="/js/topics-map.js" defer> unconditionally,
matching the established hugo-apps island convention (concepts-filter,
related-graph, etc.). The bundle is always built before Hugo runs.
- Delete hugo/data/topics_map_bundle.json sentinel (no longer needed).
- topics-layouts.test.ts: update assertion to confirm unconditional
include and absence of the now-removed guard.
- Add focus-param.ts: pure parseFocusParam() validates slug against
/^[a-z0-9][a-z0-9-]{0,80}$/ and returns '' for missing/malformed input
- Add ExploreGraph.focusSingleNode(id) exposed method: centres camera on
a single node via the same camera.animate call as applyPathOverlay
- Wire App.vue: reads parseFocusParam(window.location.search) at setup,
watches hasData once to resolve slug → node id → focusSingleNode()
- Additive only: find-path flow is entirely undisturbed
- 6 unit tests pass; vite build succeeds (72.5 kB gzip)
…h test - ExploreGraph now emits 'graphReady' at the end of buildGraph() (after forceAtlas2 layout), guaranteeing node x/y coordinates exist when any consumer calls focusSingleNode(). Previously App.vue called focusSingleNode() off a watch(hasData,...) which fired before ExploreGraph's own watch([nodes,edges]) had run buildGraph() — so getNodeAttribute returned undefined and the camera silently no-oped. - App.vue: replace watch(hasData) with onGraphReady() handler on the @graphReady event from ExploreGraph. Additive — find-path untouched. - focus-param.test.ts: add 83-char over-length rejection case + matching 81-char (boundary) acceptance case. - ExploreGraph.test.ts: add 'emits graphReady after buildGraph' test (verifies nodes + x/y exist at event time) and 'focusSingleNode calls camera.animate' test (verifies the camera call fires correctly). - 50 unit tests pass; vite build clean (72.5 kB gzip).
Add focusDone ref guard in App.vue's onGraphReady(): on first successful focusSingleNode call, set focusDone.value=true so subsequent graphReady events (fired on every filter-toggle rebuild) are ignored. Without this guard the camera snapped back to the deep-linked node on every filter toggle, overriding the user's manual pan/zoom. Test: 'graphReady fires on each rebuild; a one-shot guard limits focusSingleNode to the first' — mounts ExploreGraph, triggers a second build via nodes-prop change, asserts graphReady fires twice but the guarded camera.animate call fires exactly once. 51 unit tests pass; vite build clean (72.5 kB gzip).
…forward - Add bare `using from '../db/knowledge-graph-topic-clusters';` to srv/admin-service.cds (satisfies #1531 rule for the new split db file) - Extend AdminService with TopicClustersAdmin @readonly projection + virtual effectiveLabel (curatedLabel || label, computed in after('READ') handler) - Add overrideTopicLabel / setTopicClusterHidden actions (@requires:'Tutorial.Author') - Implement all three handlers in srv/admin-service.js - Unit test (5/5 green): effectiveLabel fallback + preference, both action writes - Hybrid test: admin override survives runKgTopicClusters() TRUNCATE+INSERT re-run
…p orphaned fetch-concepts tests Three unit-suite regressions introduced by the topics-discovery branch: 1. admin-shell-pats-nav consistency check flagged `topicClusters` as a manifest-routed nav-key missing from NAV_KEY_TO_ROUTE (and NAV_KEY_TO_TITLE) in Shell.controller.js — sidebar click would silently no-op (same trap as #763 / #1105). Add both entries. 2. scheduler-registry lockstep count was 44; the new `kg-topic-clusters` job (topics-discovery nightly reconciliation) bumps it to 45. Update assertion and add the `kg-topic-clusters` containment check. 3. 9 orphaned `test/unit/scripts/fetch-concepts*.test.ts` files reference `scripts/fetch-concepts.ts` which was deleted in #1327 / commit 779a305. The cleanup commit 92c9fd0 landed on origin/main after this branch forked; apply the same deletion here (port main's fix forward).
Adds test/e2e/topics-discovery.test.js following the #1338 pattern: - self-skips when SMOKE_BASE_URL / PLAYWRIGHT_BASE_URL is absent (verified) - auth via SMOKE_TECH_USER / SMOKE_TECH_PASSWORD Basic header - 5 tests covering the full front-door flow: 1. /topics/ returns 200 with <main> + <h1> + gallery/empty-state section 2. gallery card click navigates to /topics/<slug>/ with .topics-detail 3. concept link href contains /concepts/ 4. peer-cluster link href contains /topics/ 5. (best-effort / swallowed) #topics-map mount point present - gracefully skips navigation assertions when gallery bakes empty (pre-data-load state on fresh DEV deploy before nightly job runs)
C1 (Critical) — stable slug did not survive Louvain membership drift.
- Add `memberSlugsBlob : String(5000)` to TopicClusters to persist each
cluster's tutorial-slug set after every run.
- `_buildCommunitiesInput` now reads that persisted blob for each existing
row and uses it as `existing[i].memberSlugs` (last-night's member set),
instead of deriving from `tutMembersByFp.get(r.fingerprint)` which is
keyed by *this* run's fingerprints and returns undefined on any drift.
- Write step persists the current run's member slugs into MEMBERSLUGSBLOB
for every ACTIVE row so next night's run can Jaccard-match against it.
I1 (Important) — mintSlug ignored retired slugs → duplicate-PK INSERT crash.
- Seed `assignedSlugs` with ALL existing slugs (ACTIVE + RETIRED) before
any minting so minted slugs never collide with a retired slug in the same
INSERT batch.
- Add belt-and-suspenders dedup of the `retired` array: drop any slug that
already appears in `upserts` (ACTIVE wins).
Tests:
- topic-cluster-reconcile.test.js: I1 regression (mint avoids retired slug,
no duplicate PK across upserts+retired).
- kg-topic-clusters-job.test.js: C1 drift regression (_buildCommunitiesInput
feeds persisted [t1,t2,t3] as existing.memberSlugs → Jaccard 0.5 → match,
slug preserved; fails on old code which produced [] → Jaccard 0 → re-mint).
- topic-clusters-job-hybrid.test.js: assert ACTIVE rows have non-empty
memberSlugsBlob after a run (write-half guard for C1 round-trip).
npx cds deploy --to sqlite::memory: ✓ npm run build:sdl ✓ 14 unit tests ✓
…very-spec # Conflicts: # approuter/xs-app.json
Change 1 (header.html): add Topics nav item (icon=tags, data-href=/topics/) immediately above the Knowledge Graph entry in the nav popover. Change 2 (explore/about.html + kg-overview.css): add secondary CTA "Browse topic clusters →" next to "Explore the live graph →" in the bottom CTA strip; add matching .kg-overview__cta-strip a.secondary rule. Change 3 (topics/list.html + topics.css): add "See the full knowledge graph →" graphlink paragraph in the hero after the search form; add .topics-hero__graphlink CSS rule (Horizon CSS vars, understated). Tests: extend header-nav-includes-explore.test.ts to assert /topics/ nav item; extend topics-layouts.test.ts to assert /explore/ cross-link. Hugo build verified: all three links confirmed in built HTML output.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a new
/topics/concept-discovery front door that turns the flat ~5,946-concept/concepts/list into a browsable gallery of labeled topic clusters — each with a suggested learning path — backed by a stable-slug reconciliation pipeline and a Sigma cluster map that hands off to/explore/.Design & plan:
docs/superpowers/specs/2026-08-09-topics-discovery-front-door-design.mdanddocs/superpowers/plans/2026-08-09-topics-discovery-front-door.md.What ships
TopicClusterssidecar (db/knowledge-graph-topic-clusters.cds) maps a stable slug to the current Louvain fingerprint via pure Jaccard matching (srv/lib/topic-cluster-reconcile.js), so/topics/<slug>/URLs survive nightly membership drift. Job atsrv/jobs/kg-topic-clusters-job.js(04:47 UTC). AdmincuratedLabel/hiddenoverrides +rationalecarry forward across the TRUNCATE+INSERT; the member-slug set is persisted so drift matching actually works./build/topics-galleryfeed (srv/lib/build-topics-gallery.js) →scripts/fetch-topics-gallery.ts→hugo/data/topics_gallery.json+ per-cluster content stubs →hugo/layouts/topics/{list,single}.html. Suggested-order path via topo-sort overrequiresedges with PageRank fallback (srv/lib/topic-path-order.js). No approuter route change (catch-all → Hugo static, like/tutorials/)./graph/clusters-dataendpoint (super-graph +?cluster=subgraph) feeds a Sigma island (hugo-apps/src/topics-map/, 42.6 KB gzip); "See full graph →" deep-links/explore/?focus=<slug>(one-shot camera pre-focus).AdminService.TopicClustersAdminprojection +overrideTopicLabel/setTopicClusterHiddenactions; FE appapp/admin/topicClusters/registered in the shell (prefixtc); LR-toolbar selection recovered via Element registry (fix(admin): recover explainer bulk-action selection when FE V4 passes no arg (#1552) #1558 pattern).build-topic-clusters.js, and its data file are untouched.Decisions
/topics/(front door) +/topics/<slug>/(detail);/concepts/demoted to the exhaustive A-Z index;/explore/is the deepest zoom.ContentFilesmodel) — the data is ~18–60 nightly clusters, matching the KG communities: homepage "topic cluster" band (#1126 follow-on 1/4) #1170 band and/explore/.Testing
test/e2e/topics-discovery.test.js) drives gallery → cluster → concept → explore.npx cds deploy --to sqlite::memory:+npm run build:sdlclean.Process notes
Built via spec → plan → subagent-driven execution (14 tasks, fresh implementer + task review per task). The final whole-branch review caught a cross-task Critical the per-task reviews structurally couldn't — the reconciliation matched drifted clusters against a stale fingerprint map (empty set → slug re-minted nightly, defeating the stable-slug purpose) plus a companion duplicate-PK crash; both fixed by persisting the member-slug set and excluding retired slugs from minting.
Deferred (follow-up, non-blocking):
memberCountlabeled "concepts" in the card (overcounts);memberSlugsBlob.slice(0,5000)could truncate for >~100-tutorial clusters (harden later); a handful of test-fidelity tightenings noted in the branch history.DEV-first, consistent with the KG surface rollout posture. Deploy from a fresh
origin/mainafter merge (never from this worktree), fullmbt build.🤖 Generated with subagent-driven development.