INFOPLAT-13349: feat(beholder): track log export bytes per node via gRPC stats handler#2251
INFOPLAT-13349: feat(beholder): track log export bytes per node via gRPC stats handler#2251kirqz23 wants to merge 1 commit into
Conversation
|
👋 kirqz23, thanks for creating this pull request! To help reviewers, please consider creating future PRs as drafts first. This allows you to self-review and make any final changes before notifying the team. Once you're ready, you can mark it as "Ready for review" to request feedback. Thanks! |
📊 API Diff Results
|
There was a problem hiding this comment.
Pull request overview
Adds a custom metric to restore per-node visibility into OTLP log export volume by capturing the uncompressed outbound gRPC payload size and emitting it as beholder.logs.export.bytes (labelled by csa_public_key).
Changes:
- Introduces a gRPC
stats.Handler(sizeCaptureHandler) to capturestats.OutPayload.Length. - Wraps the OTLP logs exporter with
meteredLogsExporterto increment a bytes counter on successful exports (to avoid retry inflation). - Wires the stats handler + metered exporter into
NewGRPCClient’s shared log exporter connection.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.
| File | Description |
|---|---|
| pkg/beholder/metered_exporter.go | Adds size-capture stats handler and metered exporter wrapper emitting beholder.logs.export.bytes. |
| pkg/beholder/client.go | Wires the shared capture + exporter wrapper and attaches the gRPC stats handler(s) to the log exporter dial options. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| dialOpts := []grpc.DialOption{ | ||
| grpc.WithStatsHandler(otelgrpc.NewClientHandler(otelOpts...)), | ||
| grpc.WithStatsHandler(&sizeCaptureHandler{capture: capture}), | ||
| } |
| op, ok := rs.(*stats.OutPayload) | ||
| if !ok { | ||
| return | ||
| } | ||
| h.capture.val.Store(int64(op.Length)) |
| func (e *meteredLogsExporter) Export(ctx context.Context, records []sdklog.Record) error { | ||
| err := e.inner.Export(ctx, records) | ||
| if err == nil { | ||
| e.counter.Add(ctx, e.capture.val.Load(), e.attrs) | ||
| } | ||
| return err | ||
| } |
| func newMeteredLogsExporter( | ||
| inner sdklog.Exporter, | ||
| meter otelmetric.Meter, | ||
| csaPublicKeyHex string, | ||
| capture *sizeCapture, |
What
Adds a
beholder.logs.export.bytescounter to the beholder gRPC client, restoringper-node log volume visibility (labelled by
csa_public_key) that was lost whenotelgrpc semconv v1.40.0 dropped
rpc.client.request.size.According to these docs, metrics are totally deprecated without any successors.
RPC semantic convention stability migration guide
Semantic conventions for RPC metrics
How
stats.Handler(sizeCaptureHandler) readsOutPayload.Lengththe exact same source the old otelgrpc metric used and stores it in a shared atomic.sdklog.Exporterwrapper (meteredLogsExporter) sits above the OTLP retryloop and records the captured byte count once per batch, only on success so retries
don't inflate the count.
NewGRPCClient.Requires
Supports