Problem
goalsRouter.bulkAnalytics (packages/rpc/src/routers/goals.ts) fires two ClickHouse queries per goal — one for the conversion count (processGoalConversionCount) and one for the total-website-users denominator (getTotalWebsiteUsers) — inside a Promise.all over every requested goal.
For a website with N goals, loading the goals dashboard issues up to 2×N concurrent ClickHouse round-trips on every request. A website with 20 goals fires ~40 queries where the actual underlying data (unique visitor counts scoped by date range) could largely be answered by a small, fixed number of queries.
Root cause
Each goal independently computes:
getTotalWebsiteUsers(websiteId, effectiveStartDate, endDate, combinedFilters) — the entrant denominator
processGoalConversionCount(step, combinedFilters, params) — the completion count for that goal's single step
Most goals on a given website share the same date range and carry no goal-specific filters, so steps 1 and 2 above are frequently computing the same denominator and issuing structurally identical completion queries that differ only in which single event/pageview they're matching — exactly the shape ClickHouse can answer in one query via GROUP BY.
Proposed fix
Group goals by (a) whether they carry any filters (request-level or goal-level) and (b) their effective start date:
- Goals with no filters at all in the same date bucket can be counted in a single batched query — one shared
getTotalWebsiteUsers call plus one query that matches all their step conditions and returns per-goal completion counts via GROUP BY, instead of N×2 queries.
- Goals with filters keep the existing one-query-per-goal path unchanged, since the shared event-stream query builder (
buildIdentifiedEventStream) only threads filter conditions through the step at array index 0 (correct for its actual funnel-entry-filter use case) — batching filtered goals together would silently apply another goal's filter to the wrong goal.
This bounds the fix to the case where it's provably correct without touching the shared funnel query builder, while still collapsing the common case (dashboards with mostly unfiltered goals) from O(N) queries to O(distinct date ranges).
Will open a PR shortly with the batched implementation, a regression test asserting the query count, and no change in output for existing behavior.
Problem
goalsRouter.bulkAnalytics(packages/rpc/src/routers/goals.ts) fires two ClickHouse queries per goal — one for the conversion count (processGoalConversionCount) and one for the total-website-users denominator (getTotalWebsiteUsers) — inside aPromise.allover every requested goal.For a website with N goals, loading the goals dashboard issues up to 2×N concurrent ClickHouse round-trips on every request. A website with 20 goals fires ~40 queries where the actual underlying data (unique visitor counts scoped by date range) could largely be answered by a small, fixed number of queries.
Root cause
Each goal independently computes:
getTotalWebsiteUsers(websiteId, effectiveStartDate, endDate, combinedFilters)— the entrant denominatorprocessGoalConversionCount(step, combinedFilters, params)— the completion count for that goal's single stepMost goals on a given website share the same date range and carry no goal-specific filters, so steps 1 and 2 above are frequently computing the same denominator and issuing structurally identical completion queries that differ only in which single event/pageview they're matching — exactly the shape ClickHouse can answer in one query via
GROUP BY.Proposed fix
Group goals by (a) whether they carry any filters (request-level or goal-level) and (b) their effective start date:
getTotalWebsiteUserscall plus one query that matches all their step conditions and returns per-goal completion counts viaGROUP BY, instead of N×2 queries.buildIdentifiedEventStream) only threads filter conditions through the step at array index 0 (correct for its actual funnel-entry-filter use case) — batching filtered goals together would silently apply another goal's filter to the wrong goal.This bounds the fix to the case where it's provably correct without touching the shared funnel query builder, while still collapsing the common case (dashboards with mostly unfiltered goals) from O(N) queries to O(distinct date ranges).
Will open a PR shortly with the batched implementation, a regression test asserting the query count, and no change in output for existing behavior.