-
Notifications
You must be signed in to change notification settings - Fork 16
feat(power): add query energy and TDP dashboard axes / 新增每请求能耗与 TDP 占比图表轴 #735
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Changes from all commits
4880c69
db4422f
308fbdd
bbd7f58
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,15 @@ | ||
| import { describe, expect, it } from 'vitest'; | ||
|
|
||
| import { validateSpec } from './types'; | ||
|
|
||
| describe('validateSpec measured power axes', () => { | ||
| it.each([ | ||
| 'y_measuredJPerSuccessfulQuery', | ||
| 'y_measuredWhPerSuccessfulQuery', | ||
| 'y_measuredPowerPercentTdp', | ||
| ])('preserves %s as a benchmark Y-axis metric', (yAxisMetric) => { | ||
| const spec = validateSpec({ dataSource: 'benchmarks', yAxisMetric }); | ||
|
|
||
| expect(spec.yAxisMetric).toBe(yAxisMetric); | ||
| }); | ||
| }); |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -164,6 +164,24 @@ | |
| "y_measuredJPerTotalToken_title": "Measured Joules per Token (incl. prompt)", | ||
| "y_measuredJPerTotalToken_titleZh": "每 token 实测焦耳能耗(含提示词)", | ||
| "y_measuredJPerTotalToken_roofline": "lower_right", | ||
| "y_measuredJPerSuccessfulQuery": "measuredJPerSuccessfulQuery.y", | ||
| "y_measuredJPerSuccessfulQuery_label": "Measured J per Successful Query (J/query)", | ||
| "y_measuredJPerSuccessfulQuery_labelZh": "每次成功请求实测能耗(J/query)", | ||
| "y_measuredJPerSuccessfulQuery_title": "Measured Joules per Successful Query", | ||
| "y_measuredJPerSuccessfulQuery_titleZh": "每次成功请求实测焦耳能耗", | ||
| "y_measuredJPerSuccessfulQuery_roofline": "lower_right", | ||
| "y_measuredWhPerSuccessfulQuery": "measuredWhPerSuccessfulQuery.y", | ||
| "y_measuredWhPerSuccessfulQuery_label": "Measured Wh per Successful Query (Wh/query)", | ||
| "y_measuredWhPerSuccessfulQuery_labelZh": "每次成功请求实测能耗(Wh/query)", | ||
| "y_measuredWhPerSuccessfulQuery_title": "Measured Watt-hours per Successful Query", | ||
| "y_measuredWhPerSuccessfulQuery_titleZh": "每次成功请求实测瓦时能耗", | ||
| "y_measuredWhPerSuccessfulQuery_roofline": "lower_right", | ||
| "y_measuredPowerPercentTdp": "measuredPowerPercentTdp.y", | ||
| "y_measuredPowerPercentTdp_label": "Measured Average Power (% TDP)", | ||
| "y_measuredPowerPercentTdp_labelZh": "实测平均功耗(TDP 占比)", | ||
| "y_measuredPowerPercentTdp_title": "Measured Average Power as Percent of TDP", | ||
| "y_measuredPowerPercentTdp_titleZh": "实测平均功耗占 TDP 百分比", | ||
| "y_measuredPowerPercentTdp_roofline": "lower_right", | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🟡 WARNING (behavior vs. description mismatch, non-blocking): The PR description states "%TDP has no Pareto direction — it is a utilization gauge, not an efficiency frontier." But this config assigns I confirmed this at runtime: with the %TDP axis selected and Optimal Only on, the chart shows only the single frontier config (B200), not every deployment. Why it matters: Reviewers trusting the PR body would expect all configs to remain visible under Optimal Only on this axis. The behavior is internally consistent (config + test + code agree) and defensible — %TDP is monotonic in Fix: Either drop the |
||
| "y_cost_limit": 5, | ||
| "y_latency_limit": 60 | ||
| }, | ||
|
|
@@ -331,6 +349,24 @@ | |
| "y_measuredJPerTotalToken_title": "Measured Joules per Token (incl. prompt)", | ||
| "y_measuredJPerTotalToken_titleZh": "每 token 实测焦耳能耗(含提示词)", | ||
| "y_measuredJPerTotalToken_roofline": "lower_left", | ||
| "y_measuredJPerSuccessfulQuery": "measuredJPerSuccessfulQuery.y", | ||
| "y_measuredJPerSuccessfulQuery_label": "Measured J per Successful Query (J/query)", | ||
| "y_measuredJPerSuccessfulQuery_labelZh": "每次成功请求实测能耗(J/query)", | ||
| "y_measuredJPerSuccessfulQuery_title": "Measured Joules per Successful Query", | ||
| "y_measuredJPerSuccessfulQuery_titleZh": "每次成功请求实测焦耳能耗", | ||
| "y_measuredJPerSuccessfulQuery_roofline": "lower_left", | ||
| "y_measuredWhPerSuccessfulQuery": "measuredWhPerSuccessfulQuery.y", | ||
| "y_measuredWhPerSuccessfulQuery_label": "Measured Wh per Successful Query (Wh/query)", | ||
| "y_measuredWhPerSuccessfulQuery_labelZh": "每次成功请求实测能耗(Wh/query)", | ||
| "y_measuredWhPerSuccessfulQuery_title": "Measured Watt-hours per Successful Query", | ||
| "y_measuredWhPerSuccessfulQuery_titleZh": "每次成功请求实测瓦时能耗", | ||
| "y_measuredWhPerSuccessfulQuery_roofline": "lower_left", | ||
| "y_measuredPowerPercentTdp": "measuredPowerPercentTdp.y", | ||
| "y_measuredPowerPercentTdp_label": "Measured Average Power (% TDP)", | ||
| "y_measuredPowerPercentTdp_labelZh": "实测平均功耗(TDP 占比)", | ||
| "y_measuredPowerPercentTdp_title": "Measured Average Power as Percent of TDP", | ||
| "y_measuredPowerPercentTdp_titleZh": "实测平均功耗占 TDP 百分比", | ||
| "y_measuredPowerPercentTdp_roofline": "lower_left", | ||
| "y_cost_limit": 5, | ||
| "y_latency_limit": 60 | ||
| } | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -157,6 +157,9 @@ const METRIC_GROUPS: { | |
| 'y_measuredJPerInputToken', | ||
| 'y_measuredJPerOutputToken', | ||
| 'y_measuredJPerTotalToken', | ||
| 'y_measuredJPerSuccessfulQuery', | ||
| 'y_measuredWhPerSuccessfulQuery', | ||
| 'y_measuredPowerPercentTdp', | ||
|
Comment on lines
+160
to
+162
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
These lines expose three new dashboard selections, but the commit adds only a Cypress component test and unit tests; no AGENTS.md reference: AGENTS.md:L132-L139 Useful? React with 👍 / 👎. |
||
| ], | ||
| }, | ||
| { label: 'Custom User Values', labelZh: '自定义值', metrics: ['y_costUser', 'y_powerUser'] }, | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
TDP axis has Pareto direction
Medium Severity
y_measuredPowerPercentTdpdeclareslower_right/lower_leftrooflines even though this axis is a utilization gauge, not an efficiency frontier. That turns on Pareto / Optimal Only / best-per-SKU logic that treats lower% TDPas better, which misranks configs for this metric.Additional Locations (2)
packages/app/src/components/inference/inference-chart-config.json#L368-L369packages/app/src/components/inference/measured-power-direction.test.ts#L142-L150Reviewed by Cursor Bugbot for commit bbd7f58. Configure here.