Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 5 additions & 4 deletions .github/workflows/tests.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -11,12 +11,11 @@ jobs:
strategy:
matrix:
python_version:
- "3.8"

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be in here?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You mean in this PR? I have a separate PR for only the python changes and the packaging, but I have never used stacked PRs and was not sure how to get this into review without including the full chain. See: #6

I considered looking into stacked PRs but I figured its small enough so we can do this in a single PR. If you prefer we can merge #6 before and then rebase this change before merging.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

- "3.9"
- "3.10"
- "3.11"
- "3.12"
- "3.13"
- "3.14"
- "3.15"

steps:
- uses: actions/checkout@v4
Expand All @@ -32,4 +31,6 @@ jobs:

- run: uv run ruff check

- run: uv run mypy src
- run: uv run mypy src tests

- run: uv run pytest
63 changes: 63 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -1 +1,64 @@
A keyword library for Robot Framework intended to be used in combination with Robotmk.

## Keywords

```robotframework
*** Settings ***
Library RobotmkLibrary
```

| Keyword | Effect |
| --- | --- |
| `Monitor Subsequent Keyword Runtime discover_as=` | logs the marker that makes Checkmk discover the runtime of the next keyword, under its own name or the given one |

## Trace listener

`RobotmkLibrary.instrumentation.listener.RobotmkListener` exports a robot run as one OpenTelemetry
trace: the root span is the top level suite, child suites, tests and keywords nest
below it, and every log message becomes a log record of the innermost open span.
The export goes over OTLP/HTTP to the collector of a Checkmk site, which stores the
data in its synthetic monitoring database.

### Usage

```sh
pip install robotframework-robotmklibrary
export OTEL_EXPORTER_OTLP_ENDPOINT=https://site.example.com:4318
export CHECKMK_RESOURCE_ATTRIBUTES=cmk.application=otel_demo,cmk.suite=otel_demo_http,cmk.variant=,cmk.machine_id=426f61d315ae408796589657a55d53a1,cmk.plan_run_id=run-1,cmk.attempt=1
robot --listener RobotmkLibrary.instrumentation.listener.RobotmkListener suite.robot
```

In an RCC environment, add the package to the `pip:` section of `conda.yaml`.
The Robotmk scheduler sets the environment variables and the listener option for
every attempt of a plan.

### Listener arguments

| Argument | Fallback | Meaning |
| --- | --- | --- |
| `endpoint` | `OTEL_EXPORTER_OTLP_ENDPOINT` | base URL of the collector; `/v1/traces` and `/v1/logs` are appended |
| `headers` | `OTEL_EXPORTER_OTLP_HEADERS` | HTTP headers sent with every export request, as comma separated `key=value` pairs, e.g. `Authorization=Basic ...` for a receiver with basic auth |
| `certificate` | `OTEL_EXPORTER_OTLP_CERTIFICATE` | CA bundle for the server certificate |
| `timeout` | `OTEL_EXPORTER_OTLP_TIMEOUT`, 10 seconds | export timeout in seconds |
| `export_control_structures` | `false` | spans for `FOR`, `WHILE`, `IF`, `TRY` and `VAR` |
| `export_keyword_arguments` | `true` | `rf.keyword.args` on keyword spans |
| `max_log_length` | `500` | log bodies are cut at this length |
| `max_image_size` | 2 MiB | an embedded screenshot larger than this (base64 bytes) is dropped |
| `max_image_size_total` | 8 MiB | embedded screenshots beyond this total per run are dropped |

Robot Framework splits listener arguments at `:`, so use `;` as the separator
whenever a value contains a colon:

```sh
robot --listener "RobotmkLibrary.instrumentation.listener.RobotmkListener;endpoint=https://site.example.com:4318;timeout=5" suite.robot
```

### Development

```sh
uv sync
uv run ruff format --diff
uv run ruff check
uv run mypy src tests
uv run pytest
```
38 changes: 30 additions & 8 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -6,31 +6,38 @@ readme = "README.md"
authors = [
{ name = "Jörg Herbel", email = "joerg.herbel@checkmk.com" }
]
requires-python = ">=3.8"
dependencies = []
requires-python = ">=3.11"

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

see above

dependencies = [
"opentelemetry-exporter-otlp-proto-http>=1.45.2",
"opentelemetry-sdk>=1.45.2",
"robotframework>=7.5.1",
]

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[dependency-groups]
dev = [
"mypy>=1.14.1",
"ruff>=0.9.1",
"mypy>=2.4.1",
"pytest>=9.2",
"ruff>=0.17.3",
]

[tool.ruff.lint]
select = ["ALL"]
ignore = [
"COM812",
"D103",
"D104",
"D203",
"D213",
"CPY",
"D",
"E501",
"N999",
"TC",
]

[tool.ruff.lint.per-file-ignores]
"tests/**" = ["PLR2004", "S101"]

[tool.mypy]
strict = true
enable_error_code = [
Expand All @@ -40,6 +47,21 @@ enable_error_code = [
]
disallow_any_expr = true
disallow_any_explicit = true
# Calls into these packages are untyped at their end, not ours.
untyped_calls_exclude = ["robot"]

[[tool.mypy.overrides]]
module = "tests.*"
# Robot Framework and the SDK hand the tests Any-typed values (run results, span ids).
disallow_any_expr = false

[[tool.mypy.overrides]]
module = "robot.*"
ignore_missing_imports = true

[tool.pytest.ini_options]
testpaths = ["tests"]
filterwarnings = ["error::UserWarning"]

[tool.hatch.build.targets.wheel]
packages = ["src/RobotmkLibrary"]
21 changes: 10 additions & 11 deletions src/RobotmkLibrary/__init__.py
Original file line number Diff line number Diff line change
@@ -1,14 +1,13 @@
from __future__ import annotations
"""Keyword library for Robot Framework suites run by Robotmk.

import logging
``Library RobotmkLibrary`` imports the keywords, which live in
``RobotmkLibrary.runtime``. The trace listener is
``RobotmkLibrary.instrumentation.listener.RobotmkListener``.
"""

from importlib.metadata import version

def monitor_subsequent_keyword_runtime(
*,
discover_as: str | None = None,
) -> None:
logging.info(
"The subsequent keyword will be discovered in Checkmk using its own name"
if discover_as is None
else f'The subsequent keyword will be discovered in Checkmk as: "{discover_as}"',
)
from RobotmkLibrary.runtime import monitor_subsequent_keyword_runtime

__version__ = version("robotframework-robotmklibrary")
__all__ = ["monitor_subsequent_keyword_runtime"]
Empty file.
25 changes: 25 additions & 0 deletions src/RobotmkLibrary/instrumentation/_exceptions.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
"""Errors raised by the trace listener."""


class GlobalTracerProviderAlreadySetError(RuntimeError):
"""Another instrumentation registered the global tracer provider first."""

def __init__(self, registered: object) -> None:
"""Name the provider found in the global slot."""
super().__init__(
"The global OpenTelemetry tracer provider is already "
f"{type(registered).__module__}.{type(registered).__name__}; "
"another instrumentation configured OpenTelemetry before the listener"
)


class GlobalLoggerProviderAlreadySetError(RuntimeError):
"""Another instrumentation registered the global logger provider first."""

def __init__(self, registered: object) -> None:
"""Name the provider found in the global slot."""
super().__init__(
"The global OpenTelemetry logger provider is already "
f"{type(registered).__module__}.{type(registered).__name__}; "
"another instrumentation configured OpenTelemetry before the listener"
)
31 changes: 31 additions & 0 deletions src/RobotmkLibrary/instrumentation/_logs.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
"""The OpenTelemetry logs signal, gathered in one place because it is experimental.

OpenTelemetry Python logs are in an experimental state. The APIs within
``opentelemetry._logs``, ``opentelemetry.sdk._logs`` and the OTLP log exporter
are subject to change in minor/patch releases and make no backward
compatibility guarantees at this time; the leading underscore marks that state,
there is no other import path. The listener imports the names from here, so
that a rename upstream is a change to this module only.
See: https://opentelemetry-python.readthedocs.io/en/latest/examples/logs/README.html
"""

from opentelemetry._logs import (
Logger,
SeverityNumber,
get_logger_provider,
set_logger_provider,
)
from opentelemetry.exporter.otlp.proto.http._log_exporter import OTLPLogExporter
from opentelemetry.sdk._logs import LoggerProvider
from opentelemetry.sdk._logs.export import BatchLogRecordProcessor, LogRecordExporter

__all__ = [
"BatchLogRecordProcessor",
"LogRecordExporter",
"Logger",
"LoggerProvider",
"OTLPLogExporter",
"SeverityNumber",
"get_logger_provider",
"set_logger_provider",
]
69 changes: 69 additions & 0 deletions src/RobotmkLibrary/instrumentation/_protocols.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
"""Types for the objects Robot Framework passes to the listener hooks.

Robot Framework calls ``start_suite(data, result)``, ``end_keyword(data, result)``,
``log_message(message)`` and so on with its own ``robot.result`` and
``robot.model`` objects, but ships no type information for them. The protocols
below declare the attributes the listener reads from those objects, suite and
test results, keyword and body item results, log messages, so that the hooks
are type-checked without importing ``robot``.
"""

from collections.abc import Sequence
from datetime import datetime
from enum import StrEnum
from typing import Protocol


class Status(StrEnum):
"""The result status of a Robot Framework suite, test, keyword or body item."""

PASS = "PASS" # noqa: S105 Flagged by ruffs credentials heuristics
FAIL = "FAIL"
SKIP = "SKIP"
NOT_RUN = "NOT RUN"


class StatusResult(Protocol):
"""The parts of a Robot Framework result object every span needs."""

@property
def status(self) -> str: ...
@property
def message(self) -> str: ...
@property
def start_time(self) -> datetime | None: ...
@property
def end_time(self) -> datetime | None: ...


class Result(StatusResult, Protocol):
@property
def name(self) -> str: ...


class TestResult(Result, Protocol):
@property
def full_name(self) -> str: ...


class KeywordResult(TestResult, Protocol):
@property
def type(self) -> str: ...
@property
def args(self) -> Sequence[object]: ...


class BodyItemResult(StatusResult, Protocol):
@property
def type(self) -> str: ...


class Message(Protocol):
@property
def message(self) -> str: ...
@property
def level(self) -> str: ...
@property
def html(self) -> bool: ...
@property
def timestamp(self) -> datetime | None: ...
Loading
Loading