Skip to content

Add k8s pod/cluster attributes to PENDING status span#254

Merged
morgan-wowk merged 1 commit into
masterfrom
execution-tracing-launcher-data
Jun 5, 2026
Merged

Add k8s pod/cluster attributes to PENDING status span#254
morgan-wowk merged 1 commit into
masterfrom
execution-tracing-launcher-data

Conversation

@morgan-wowk

@morgan-wowk morgan-wowk commented May 22, 2026

Copy link
Copy Markdown
Collaborator

Add Kubernetes pod/cluster attributes to PENDING execution spans

k8s.pod.name, k8s.namespace.name, k8s.cluster.url sourced from
ContainerExecution.launcher_data. Handles kubernetes, kubernetes_job,
and non-k8s launchers (SkyPilot) gracefully via early-return guards.

The new _launcher_pod_attrs helper only emits attributes when the span
status is PENDING and launcher_data is present, so spans for other
statuses or non-Kubernetes launchers are unaffected.

Screenshots

Screenshot 2026-05-22 at 8.13.33 PM.png

@yuechao-qin yuechao-qin left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

image.png

Curious

  • tangle-orchestrator-local - What does "local" mean and what other values can be possible there?
  • what is otel.scope.name and where is it set in the code?
  • span.kind what is "internal" and what other values are possible?

Comment thread tests/instrumentation/test_execution_tracing.py
@morgan-wowk morgan-wowk force-pushed the execution-tracing-error-data branch from 2025a32 to d9fcf6b Compare May 29, 2026 18:52
@morgan-wowk morgan-wowk force-pushed the execution-tracing-launcher-data branch from 374f9cf to 22df75e Compare May 29, 2026 18:52
@morgan-wowk

morgan-wowk commented May 29, 2026

Copy link
Copy Markdown
Collaborator Author

🤖 Re: review questions —

  • tangle-orchestrator-local — the suffix comes from APP_ENV (or TANGLE_ENV in the OSS layer), passed as service_name=f"tangle-orchestrator-{tangle_env}" in orchestrator_main_oasis.py. In production it becomes tangle-orchestrator-production. Other services: tangle-api-{env}, tangle-metrics-poller-{env}.
  • otel.scope.name — set by trace.get_tracer("tangle.orchestrator") at module level. OTel SDK populates it automatically.
  • span.kind: INTERNAL — OTel default when no explicit kind is set. Other values: SERVER, CLIENT, PRODUCER, CONSUMER. Internal is correct here since these are in-process reconstructed spans, not representing network boundaries.

@morgan-wowk morgan-wowk force-pushed the execution-tracing-error-data branch from d9fcf6b to b9f8147 Compare May 29, 2026 19:03
@morgan-wowk morgan-wowk force-pushed the execution-tracing-launcher-data branch from 22df75e to b7aa164 Compare May 29, 2026 19:04
Comment thread tests/instrumentation/test_execution_tracing.py
@morgan-wowk morgan-wowk force-pushed the execution-tracing-launcher-data branch from b7aa164 to eee983b Compare June 1, 2026 20:02
@morgan-wowk morgan-wowk changed the base branch from execution-tracing-error-data to graphite-base/254 June 1, 2026 20:14
@morgan-wowk morgan-wowk requested a review from yuechao-qin June 1, 2026 20:16

morgan-wowk commented Jun 5, 2026

Copy link
Copy Markdown
Collaborator Author

Merge activity

  • Jun 5, 5:07 PM UTC: A user started a stack merge that includes this pull request via Graphite.
  • Jun 5, 5:09 PM UTC: Graphite rebased this pull request as part of a merge.
  • Jun 5, 5:09 PM UTC: @morgan-wowk merged this pull request with Graphite.

@morgan-wowk morgan-wowk changed the base branch from graphite-base/254 to master June 5, 2026 17:07
k8s.pod.name, k8s.namespace.name, k8s.cluster.url sourced from
ContainerExecution.launcher_data. Handles kubernetes, kubernetes_job,
and non-k8s launchers (SkyPilot) gracefully via early-return guards.
@morgan-wowk morgan-wowk force-pushed the execution-tracing-launcher-data branch from eee983b to 012975e Compare June 5, 2026 17:08
@morgan-wowk morgan-wowk merged commit 3db02f1 into master Jun 5, 2026
5 of 7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants