Skip to content

docs: pipeline: inputs: opentelemetry: document v5.2 request limits #2831

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
eschabell merged 1 commit into from
Nov 9, 2026
Merged

docs: pipeline: inputs: opentelemetry: document v5.2 request limits #2831

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 16 additions & 4 deletions pipeline/inputs/opentelemetry.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,14 +26,15 @@ For a cross-plugin explanation of the shared listener settings, see
|-------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------|
| `alias` | Sets an alias for multiple instances of the same input plugin. If no alias is specified, a default name will be assigned using the plugin name followed by a dot and a sequence number. | _none_ |
| `buffer_chunk_size` | Size of each buffer chunk allocated for HTTP requests (advanced users only). | `512K` |
| `buffer_max_size` | Maximum size of the HTTP request buffer in `KB`, `MB`, or `GB`. | `4M` |
| `buffer_max_size` | Maximum size of an OTLP request payload, as received and after decompression. Compatibility alias for `http_server.buffer_max_size`. See [Request size limits](#request-size-limits). | 64 MiB |
| `encode_profiles_as_log` | Encode profiles received as text and ingest them in the logging pipeline. | `true` |
| `host` | The hostname. | `localhost` |
| `http2` | Enable HTTP/2 protocol support for the OpenTelemetry receiver. | `true` |
| `http_server.max_connections` | Maximum number of concurrent active HTTP connections. `0` means unlimited. | `0` |
| `http_server.workers` | Number of HTTP listener worker threads. | `1` |
| `http_server.ingress_queue_event_limit` | Maximum number of deferred ingress queue entries. Applies only when `http_server.workers` is greater than `1`. | `8192` |
| `http_server.ingress_queue_byte_limit` | Maximum size of the deferred ingress queue. Applies only when `http_server.workers` is greater than `1`. | `256M` |
| `http_server.max_decoded_body_size` | Maximum size of an `OTLP/HTTP` body after decompression. Defaults to `buffer_max_size`. Supported in v5.2 or later. See [Request size limits](#request-size-limits). | 64 MiB |
| `listen` | The network address to listen on. | `0.0.0.0` |
| `log_level` | Specifies the log level for this plugin. If not set here, the plugin uses the global log level specified in the `service` section of your configuration file. | `info` |
| `log_suppress_interval` | Suppresses log messages from this plugin that appear similar within a specified time interval. `0` no suppression. | `0` |
Expand All @@ -55,7 +56,8 @@ For a cross-plugin explanation of the shared listener settings, see
| `oauth2.validate` | Enable `OAuth 2.0` `JWT` validation for incoming requests. | `false` |
| `port` | The port for Fluent Bit to listen for incoming connections. | `4318` |
| `profiles_support` | This is an experimental feature, feel free to test it but don't enable this in production environments. | `false` |
| `raw_traces` | Forward traces without processing. When set to `false` (default), traces are processed using the unified JSON parser with strict validation. When set to `true`, trace data is forwarded as raw log messages without validation or processing. | `false` |
| `protobuf_decode_max_size` | Maximum estimated memory to decode the Protocol Buffers payloads in one request. Must be a positive size. Supported in v5.2 or later. See [Request size limits](#request-size-limits). | 128 MiB |
| `raw_traces` | Forward JSON traces without processing. When set to `false` (default), JSON traces are processed using the unified JSON parser with strict validation. When set to `true`, JSON trace data is forwarded as raw log messages without validation or processing. Protocol Buffers and gRPC traces are always decoded as traces. | `false` |
| `routable` | If set to `true`, the data generated by the plugin will be routable, meaning that it can be forwarded to other plugins or outputs. If set to `false`, the data will be discarded. | `true` |
| `storage.pause_on_chunks_overlimit` | Enable pausing on an input when they reach their chunks limit. | _none_ |
| `storage.type` | Sets the storage type for this input, one of: `filesystem`, `memory` or `memrb`. | `memory` |
Expand Down Expand Up @@ -85,7 +87,7 @@ The `http_server.ingress_queue_event_limit` and
`http_server.ingress_queue_byte_limit` settings matter only when
`http_server.workers` is greater than `1`.

When `raw_traces` is set to `false` (default), the traces endpoint (`/v1/traces`) processes incoming trace data using the unified JSON parser with strict validation. The endpoint accepts both `protobuf` and `JSON` encoded payloads. When `raw_traces` is set to `true`, any data forwarded to the traces endpoint will be packed and forwarded as a log message without processing, validation, or conversion to the Fluent Bit internal trace format.
When `raw_traces` is set to `false` (default), the traces endpoint (`/v1/traces`) processes incoming trace data using the unified JSON parser with strict validation. The endpoint accepts both `protobuf` and `JSON` encoded payloads. When `raw_traces` is set to `true`, JSON payloads sent to the traces endpoint are packed and forwarded as a log message without processing, validation, or conversion to the Fluent Bit internal trace format. `raw_traces` doesn't apply to Protocol Buffers or gRPC payloads: they still go through the checks in [Request size limits](#request-size-limits), including the decode budget, and are decoded as traces.

### OpenTelemetry transport protocol endpoints

Expand All @@ -112,11 +114,21 @@ For `OTLP/GRPC`:
- `/opentelemetry.proto.collector.trace.v1.TraceService/Export`
- `/opentelemetry.proto.collector.traces.v1.TracesService/Export`

### Request size limits

In v5.2 or later, the plugin applies the following limits to each request:

- `buffer_max_size` limits the request payload. It defaults to 64 MiB (67,108,864 bytes), the size recommended by the OTLP specification. Earlier versions used the shared `4M` default. Fluent Bit adds 64 KiB to the HTTP request buffer for headers and gRPC framing. An `OTLP/HTTP` request whose payload is larger than `buffer_max_size`, before or after decompression, is rejected with `413`. When an uncompressed payload reaches the plugin, the response includes the message `OTLP request size exceeds limit`. A request larger than the HTTP request buffer (`buffer_max_size` plus 64 KiB) is rejected by the listener before it reaches the plugin. On HTTP/1.1, the listener sends a `413` response with an empty body and closes the connection. On HTTP/2, the request fails without a `413` response. A compressed body that exceeds the limit during decompression gets a `413` response with an empty body.
- `http_server.max_decoded_body_size` limits an `OTLP/HTTP` body while it's decompressed. If it isn't set, it equals the `buffer_max_size` value. Setting it to `0` selects `100M`. Because the decompressed payload is also checked against `buffer_max_size`, a higher value doesn't allow larger payloads.
- `protobuf_decode_max_size` limits the estimated memory needed to decode the Protocol Buffers payload of a request, summed across all of its gRPC messages. The default is 128 MiB (134,217,728 bytes). Fluent Bit checks this budget before it ingests any data from the request. A request over the budget is rejected with `413` or gRPC status `8 (RESOURCE_EXHAUSTED)`, and a malformed payload with `400` or gRPC status `3 (INVALID_ARGUMENT)`. Both use the message `protobuf decode budget exceeded or invalid payload`. The same value also limits the memory used to convert each Protocol Buffers logs message of 8 KiB or more. A logs message that exceeds it during conversion fails with `400` or a gRPC error. JSON payloads aren't affected.
Comment thread
eschabell marked this conversation as resolved.
Resolved

Size values such as `64M` use decimal units. To set exactly 64 MiB, use `67108864`. If you raise `buffer_max_size`, you might also need to raise `protobuf_decode_max_size`, because larger payloads need more memory to decode.

### gRPC message size limits

In v5.1.4 or later, the plugin enforces the following limits on `OTLP/GRPC` requests:

- Each gRPC message can't exceed 16 MiB, both as received and after decompression.
- Each gRPC message can't exceed `buffer_max_size` in v5.2 or later, or 16 MiB in earlier versions, both as received and after decompression.
- The combined size of all messages in a request, after decompression, can't exceed `buffer_max_size`.

Messages compressed with `gzip`, `zstd`, or `snappy` (set by the `grpc-encoding` header) are decompressed in bounded steps, so a highly compressed payload is rejected as soon as its expanded size reaches the limit. Requests that exceed a limit are rejected with gRPC status `8 (RESOURCE_EXHAUSTED)` and the message `gRPC message size exceeds limit` or `expanded gRPC payload size exceeds limit`. Compressed messages that use any other `grpc-encoding` value are rejected with gRPC status `12 (UNIMPLEMENTED)`. Uncompressed messages aren't affected by the `grpc-encoding` value.
Expand Down
Loading