Repository navigation
Trace listener #7
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Closed
Closed
Changes from all commits
Commits
Show all changes
4 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1 +1,64 @@ | ||
| A keyword library for Robot Framework intended to be used in combination with Robotmk. | ||
|
|
||
| ## Keywords | ||
|
|
||
| ```robotframework | ||
| *** Settings *** | ||
| Library RobotmkLibrary | ||
| ``` | ||
|
|
||
| | Keyword | Effect | | ||
| | --- | --- | | ||
| | `Monitor Subsequent Keyword Runtime discover_as=` | logs the marker that makes Checkmk discover the runtime of the next keyword, under its own name or the given one | | ||
|
|
||
| ## Trace listener | ||
|
|
||
| `RobotmkLibrary.instrumentation.listener.RobotmkListener` exports a robot run as one OpenTelemetry | ||
| trace: the root span is the top level suite, child suites, tests and keywords nest | ||
| below it, and every log message becomes a log record of the innermost open span. | ||
| The export goes over OTLP/HTTP to the collector of a Checkmk site, which stores the | ||
| data in its synthetic monitoring database. | ||
|
|
||
| ### Usage | ||
|
|
||
| ```sh | ||
| pip install robotframework-robotmklibrary | ||
| export OTEL_EXPORTER_OTLP_ENDPOINT=https://site.example.com:4318 | ||
| export CHECKMK_RESOURCE_ATTRIBUTES=cmk.application=otel_demo,cmk.suite=otel_demo_http,cmk.variant=,cmk.machine_id=426f61d315ae408796589657a55d53a1,cmk.plan_run_id=run-1,cmk.attempt=1 | ||
| robot --listener RobotmkLibrary.instrumentation.listener.RobotmkListener suite.robot | ||
| ``` | ||
|
|
||
| In an RCC environment, add the package to the `pip:` section of `conda.yaml`. | ||
| The Robotmk scheduler sets the environment variables and the listener option for | ||
| every attempt of a plan. | ||
|
|
||
| ### Listener arguments | ||
|
|
||
| | Argument | Fallback | Meaning | | ||
| | --- | --- | --- | | ||
| | `endpoint` | `OTEL_EXPORTER_OTLP_ENDPOINT` | base URL of the collector; `/v1/traces` and `/v1/logs` are appended | | ||
| | `headers` | `OTEL_EXPORTER_OTLP_HEADERS` | HTTP headers sent with every export request, as comma separated `key=value` pairs, e.g. `Authorization=Basic ...` for a receiver with basic auth | | ||
| | `certificate` | `OTEL_EXPORTER_OTLP_CERTIFICATE` | CA bundle for the server certificate | | ||
| | `timeout` | `OTEL_EXPORTER_OTLP_TIMEOUT`, 10 seconds | export timeout in seconds | | ||
| | `export_control_structures` | `false` | spans for `FOR`, `WHILE`, `IF`, `TRY` and `VAR` | | ||
| | `export_keyword_arguments` | `true` | `rf.keyword.args` on keyword spans | | ||
| | `max_log_length` | `500` | log bodies are cut at this length | | ||
| | `max_image_size` | 2 MiB | an embedded screenshot larger than this (base64 bytes) is dropped | | ||
| | `max_image_size_total` | 8 MiB | embedded screenshots beyond this total per run are dropped | | ||
|
|
||
| Robot Framework splits listener arguments at `:`, so use `;` as the separator | ||
| whenever a value contains a colon: | ||
|
|
||
| ```sh | ||
| robot --listener "RobotmkLibrary.instrumentation.listener.RobotmkListener;endpoint=https://site.example.com:4318;timeout=5" suite.robot | ||
| ``` | ||
|
|
||
| ### Development | ||
|
|
||
| ```sh | ||
| uv sync | ||
| uv run ruff format --diff | ||
| uv run ruff check | ||
| uv run mypy src tests | ||
| uv run pytest | ||
| ``` |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -6,31 +6,38 @@ readme = "README.md" | |
| authors = [ | ||
| { name = "Jörg Herbel", email = "joerg.herbel@checkmk.com" } | ||
| ] | ||
| requires-python = ">=3.8" | ||
| dependencies = [] | ||
| requires-python = ">=3.11" | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. see above |
||
| dependencies = [ | ||
| "opentelemetry-exporter-otlp-proto-http>=1.45.2", | ||
| "opentelemetry-sdk>=1.45.2", | ||
| "robotframework>=7.5.1", | ||
| ] | ||
|
|
||
| [build-system] | ||
| requires = ["hatchling"] | ||
| build-backend = "hatchling.build" | ||
|
|
||
| [dependency-groups] | ||
| dev = [ | ||
| "mypy>=1.14.1", | ||
| "ruff>=0.9.1", | ||
| "mypy>=2.4.1", | ||
| "pytest>=9.2", | ||
| "ruff>=0.17.3", | ||
| ] | ||
|
|
||
| [tool.ruff.lint] | ||
| select = ["ALL"] | ||
| ignore = [ | ||
| "COM812", | ||
| "D103", | ||
| "D104", | ||
| "D203", | ||
| "D213", | ||
| "CPY", | ||
| "D", | ||
| "E501", | ||
| "N999", | ||
| "TC", | ||
| ] | ||
|
|
||
| [tool.ruff.lint.per-file-ignores] | ||
| "tests/**" = ["PLR2004", "S101"] | ||
|
|
||
| [tool.mypy] | ||
| strict = true | ||
| enable_error_code = [ | ||
|
|
@@ -40,6 +47,21 @@ enable_error_code = [ | |
| ] | ||
| disallow_any_expr = true | ||
| disallow_any_explicit = true | ||
| # Calls into these packages are untyped at their end, not ours. | ||
| untyped_calls_exclude = ["robot"] | ||
|
|
||
| [[tool.mypy.overrides]] | ||
| module = "tests.*" | ||
| # Robot Framework and the SDK hand the tests Any-typed values (run results, span ids). | ||
| disallow_any_expr = false | ||
|
|
||
| [[tool.mypy.overrides]] | ||
| module = "robot.*" | ||
| ignore_missing_imports = true | ||
|
|
||
| [tool.pytest.ini_options] | ||
| testpaths = ["tests"] | ||
| filterwarnings = ["error::UserWarning"] | ||
|
|
||
| [tool.hatch.build.targets.wheel] | ||
| packages = ["src/RobotmkLibrary"] | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,14 +1,13 @@ | ||
| from __future__ import annotations | ||
| """Keyword library for Robot Framework suites run by Robotmk. | ||
|
|
||
| import logging | ||
| ``Library RobotmkLibrary`` imports the keywords, which live in | ||
| ``RobotmkLibrary.runtime``. The trace listener is | ||
| ``RobotmkLibrary.instrumentation.listener.RobotmkListener``. | ||
| """ | ||
|
|
||
| from importlib.metadata import version | ||
|
|
||
| def monitor_subsequent_keyword_runtime( | ||
| *, | ||
| discover_as: str | None = None, | ||
| ) -> None: | ||
| logging.info( | ||
| "The subsequent keyword will be discovered in Checkmk using its own name" | ||
| if discover_as is None | ||
| else f'The subsequent keyword will be discovered in Checkmk as: "{discover_as}"', | ||
| ) | ||
| from RobotmkLibrary.runtime import monitor_subsequent_keyword_runtime | ||
|
|
||
| __version__ = version("robotframework-robotmklibrary") | ||
| __all__ = ["monitor_subsequent_keyword_runtime"] |
Empty file.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,25 @@ | ||
| """Errors raised by the trace listener.""" | ||
|
|
||
|
|
||
| class GlobalTracerProviderAlreadySetError(RuntimeError): | ||
| """Another instrumentation registered the global tracer provider first.""" | ||
|
|
||
| def __init__(self, registered: object) -> None: | ||
| """Name the provider found in the global slot.""" | ||
| super().__init__( | ||
| "The global OpenTelemetry tracer provider is already " | ||
| f"{type(registered).__module__}.{type(registered).__name__}; " | ||
| "another instrumentation configured OpenTelemetry before the listener" | ||
| ) | ||
|
|
||
|
|
||
| class GlobalLoggerProviderAlreadySetError(RuntimeError): | ||
| """Another instrumentation registered the global logger provider first.""" | ||
|
|
||
| def __init__(self, registered: object) -> None: | ||
| """Name the provider found in the global slot.""" | ||
| super().__init__( | ||
| "The global OpenTelemetry logger provider is already " | ||
| f"{type(registered).__module__}.{type(registered).__name__}; " | ||
| "another instrumentation configured OpenTelemetry before the listener" | ||
| ) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,31 @@ | ||
| """The OpenTelemetry logs signal, gathered in one place because it is experimental. | ||
|
|
||
| OpenTelemetry Python logs are in an experimental state. The APIs within | ||
| ``opentelemetry._logs``, ``opentelemetry.sdk._logs`` and the OTLP log exporter | ||
| are subject to change in minor/patch releases and make no backward | ||
| compatibility guarantees at this time; the leading underscore marks that state, | ||
| there is no other import path. The listener imports the names from here, so | ||
| that a rename upstream is a change to this module only. | ||
| See: https://opentelemetry-python.readthedocs.io/en/latest/examples/logs/README.html | ||
| """ | ||
|
|
||
| from opentelemetry._logs import ( | ||
| Logger, | ||
| SeverityNumber, | ||
| get_logger_provider, | ||
| set_logger_provider, | ||
| ) | ||
| from opentelemetry.exporter.otlp.proto.http._log_exporter import OTLPLogExporter | ||
| from opentelemetry.sdk._logs import LoggerProvider | ||
| from opentelemetry.sdk._logs.export import BatchLogRecordProcessor, LogRecordExporter | ||
|
|
||
| __all__ = [ | ||
| "BatchLogRecordProcessor", | ||
| "LogRecordExporter", | ||
| "Logger", | ||
| "LoggerProvider", | ||
| "OTLPLogExporter", | ||
| "SeverityNumber", | ||
| "get_logger_provider", | ||
| "set_logger_provider", | ||
| ] |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,69 @@ | ||
| """Types for the objects Robot Framework passes to the listener hooks. | ||
|
|
||
| Robot Framework calls ``start_suite(data, result)``, ``end_keyword(data, result)``, | ||
| ``log_message(message)`` and so on with its own ``robot.result`` and | ||
| ``robot.model`` objects, but ships no type information for them. The protocols | ||
| below declare the attributes the listener reads from those objects, suite and | ||
| test results, keyword and body item results, log messages, so that the hooks | ||
| are type-checked without importing ``robot``. | ||
| """ | ||
|
|
||
| from collections.abc import Sequence | ||
| from datetime import datetime | ||
| from enum import StrEnum | ||
| from typing import Protocol | ||
|
|
||
|
|
||
| class Status(StrEnum): | ||
| """The result status of a Robot Framework suite, test, keyword or body item.""" | ||
|
|
||
| PASS = "PASS" # noqa: S105 Flagged by ruffs credentials heuristics | ||
| FAIL = "FAIL" | ||
| SKIP = "SKIP" | ||
| NOT_RUN = "NOT RUN" | ||
|
|
||
|
|
||
| class StatusResult(Protocol): | ||
| """The parts of a Robot Framework result object every span needs.""" | ||
|
|
||
| @property | ||
| def status(self) -> str: ... | ||
| @property | ||
| def message(self) -> str: ... | ||
| @property | ||
| def start_time(self) -> datetime | None: ... | ||
| @property | ||
| def end_time(self) -> datetime | None: ... | ||
|
|
||
|
|
||
| class Result(StatusResult, Protocol): | ||
| @property | ||
| def name(self) -> str: ... | ||
|
|
||
|
|
||
| class TestResult(Result, Protocol): | ||
| @property | ||
| def full_name(self) -> str: ... | ||
|
|
||
|
|
||
| class KeywordResult(TestResult, Protocol): | ||
| @property | ||
| def type(self) -> str: ... | ||
| @property | ||
| def args(self) -> Sequence[object]: ... | ||
|
|
||
|
|
||
| class BodyItemResult(StatusResult, Protocol): | ||
| @property | ||
| def type(self) -> str: ... | ||
|
|
||
|
|
||
| class Message(Protocol): | ||
| @property | ||
| def message(self) -> str: ... | ||
| @property | ||
| def level(self) -> str: ... | ||
| @property | ||
| def html(self) -> bool: ... | ||
| @property | ||
| def timestamp(self) -> datetime | None: ... |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should this be in here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You mean in this PR? I have a separate PR for only the python changes and the packaging, but I have never used stacked PRs and was not sure how to get this into review without including the full chain. See: #6
I considered looking into stacked PRs but I figured its small enough so we can do this in a single PR. If you prefer we can merge #6 before and then rebase this change before merging.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I tried my luck here: https://github.com/Checkmk/robotframework-robotmklibrary/stack/13