diff --git a/README.md b/README.md index 78c32e516..06be3cd54 100644 --- a/README.md +++ b/README.md @@ -2,10 +2,12 @@ description: High Performance Telemetry Agent for Logs, Metrics and Traces --- -# Fluent Bit documentation +# Fluent Bit 5.2 documentation
+This manual covers the upcoming Fluent Bit `5.2` release, developed on the `master` branch. For release highlights and migration guidance, see [What's new in Fluent Bit v5](installation/whats-new-in-fluent-bit-v5.md#fluent-bit-v52) and [Upgrade notes](installation/upgrade-notes.md#fluent-bit-v52). + [Fluent Bit](https://fluentbit.io) is a fast and lightweight telemetry agent for logs, metrics, and traces for Linux, macOS, Windows, and BSD family operating systems. Fluent Bit has been made with a strong focus on performance to allow the collection and processing of telemetry data from different sources without complexity.![](https://static.scarf.sh/a.png?x-pxid=71f0e011-761f-4c6f-9a89-38817887faae) ## Features @@ -37,7 +39,7 @@ description: High Performance Telemetry Agent for Logs, Metrics and Traces For more details about changes in each release, refer to the [official release notes](https://fluentbit.io/announcements/). -If you are upgrading from the Fluent Bit `4.2` series, start with [What's new in Fluent Bit v5](installation/whats-new-in-fluent-bit-v5.md) and [Upgrade notes](installation/upgrade-notes.md). +If you are upgrading from Fluent Bit `5.1` or an earlier series, review [What's new in Fluent Bit v5](installation/whats-new-in-fluent-bit-v5.md) and [Upgrade notes](installation/upgrade-notes.md). ## Fluent Bit, Fluentd, and CNCF diff --git a/SUMMARY.md b/SUMMARY.md index 55d79a4ad..d960952c1 100644 --- a/SUMMARY.md +++ b/SUMMARY.md @@ -1,6 +1,6 @@ # Table of contents -* [Fluent Bit documentation](README.md) +* [Fluent Bit 5.2 documentation](README.md) ## About diff --git a/installation/downloads.md b/installation/downloads.md index cb8a73d06..a6382acb1 100644 --- a/installation/downloads.md +++ b/installation/downloads.md @@ -5,6 +5,8 @@ Fluent Bit is compatible with most x86-based, x86_64-based, arm32v7-based, and arm64v8-based systems. +This manual covers the upcoming Fluent Bit `5.2` release. Until `5.2.0` is released, the package and container download pages list published releases from earlier series. To try `5.2` before its release, [build the development source code](downloads/source.md#development). + ## Build from source code You can [build and install Fluent Bit from its source code](../installation/downloads/source.md). There are also platform-specific guides for building Fluent Bit from source on [macOS](../installation/downloads/macos.md#compile-from-source) and [Windows](../installation/downloads/windows.md#compile-from-source). diff --git a/installation/downloads/source.md b/installation/downloads/source.md index d50a132cf..435850a01 100644 --- a/installation/downloads/source.md +++ b/installation/downloads/source.md @@ -21,7 +21,7 @@ If you want to contribute to Fluent Bit, you should use the most recent code. Yo git clone https://github.com/fluent/fluent-bit ``` -The `master` branch is where the development of Fluent Bit happens. Development version users should expect issues when compiling or at run time. +The `master` branch is where development of the upcoming Fluent Bit `5.2` release happens. This manual documents that release series. Development version users should expect issues when compiling or at run time. Fluent Bit users are encouraged to help test every development version to ensure a stable release. diff --git a/installation/upgrade-notes.md b/installation/upgrade-notes.md index 3d3aba9ac..64e236430 100644 --- a/installation/upgrade-notes.md +++ b/installation/upgrade-notes.md @@ -8,6 +8,35 @@ Release notes will be prepared in advance of a Git tag for a release. An officia The tag drives the binary release process. Release binaries (containers and packages) will appear after a tag and its associated release note. This lets users to expect the new release binary to appear and allow/deny/update it as appropriate in their infrastructure. +## Fluent Bit v5.2 + +Fluent Bit `v5.2` is the upcoming release developed on `master`. Review [What's new in Fluent Bit v5.2](whats-new-in-fluent-bit-v5.md#fluent-bit-v52) before testing an upgrade from `5.1` or an earlier series. + +### Native log processing + +Version `5.2` changes internal log decoding, processor execution, and chunk allocation. Before deploying an upgrade, test representative log payloads and processor chains with your buffering and backpressure settings. Check record contents, routing, and delivery under load. + +### Opt in to Fluent Typed Scripting + +The new `fts` processor is configured through the YAML `processors` section. Existing Lua filters continue to use the `lua` filter plugin. To adopt Fluent Typed Scripting (FTS), write policies against its typed API and configure an `fts` processor for each signal you want to process. See the [FTS processor guide](https://github.com/fluent/fluent-bit/blob/master/plugins/processor_fts/README.md) for its callback contract and resource limits. + +### Stricter validation and security fixes + +Version `5.2` tightens several checks that can change the behavior of existing configurations. Review the following changes before you upgrade: + +- `https` URLs with an explicit port, such as `https://example.com:8443`, now always use TLS. Earlier versions connected to some of these URLs without TLS, including the [Amazon S3](../pipeline/outputs/s3.md) `authorization_endpoint_url` and the [Azure Blob](../pipeline/outputs/azure_blob.md) `configuration_endpoint_url`. If you set the [Stackdriver](../pipeline/outputs/stackdriver.md) or [Google Cloud Storage](../pipeline/outputs/gcs.md) `metadata_server` to an `https` address, it now fails with `TLS context required for URL`. Use the default `http` address instead. +- The [NATS](../pipeline/outputs/nats.md) output rejects tags that aren't valid NATS subjects, such as tags that contain spaces, `*`, or `>`, and drops those chunks. Rename the tags, or set `invalid_subject_action` to `encode`, which publishes those tags, and tags that already start with `_flb_encoded.`, to `_flb_encoded.` subjects that subscribers and permissions must then cover. +- The [File](../pipeline/outputs/file.md) output escapes more characters in the `out_file`, CSV, and LTSV formats. Update any tools that parse these files. +- The [Forward](../pipeline/inputs/forward.md) and [Syslog](../pipeline/inputs/syslog.md) inputs refuse to start when the Unix socket path is a symbolic link, such as `/dev/log` on many systemd-based systems, or is too long. The Syslog input also fails to start when `buffer_chunk_size` or `buffer_max_size` is out of range. +- The [Rewrite tag](../pipeline/filters/rewrite-tag.md) filter can no longer be used as an output processor, and Fluent Bit fails to start with that configuration. +- HTTP-based inputs close the connection, without a response, for HTTP/1 requests whose request line, headers, and closing blank line exceed `32KiB`. If clients send larger headers, raise `http_server.http1_max_header_size`. See [Pipeline](../administration/configuring-fluent-bit/yaml/pipeline-section.md). +- Inputs that validate `OAuth 2.0` tokens accept `JWKS` responses up to `1M`, including headers, which raises the earlier limit of 4,096 bytes, and apply a new `30s` time limit to the `JWKS` response. Adjust `oauth2.jwks_buffer_max_size` or `oauth2.jwks_response_timeout` if needed. +- The [OpenTelemetry](../pipeline/outputs/opentelemetry.md) output, when it uses HTTP/2 or gRPC, drops chunks whose responses exceed the new `http_client.*` size and header limits. Responses that exceed `http_client.response_timeout` are retried. See [Service](../administration/configuring-fluent-bit/yaml/service-section.md). +- `AWS_CONTAINER_CREDENTIALS_FULL_URI` now requires `http` hosts to be IP addresses in the allowed loopback and link-local ranges, or the exact ECS and EKS Pod Identity addresses, or the `Greengrass` endpoint. Host names that only started with or contained an allowed address, such as `127.example.com`, are rejected. If it's rejected, Fluent Bit can fall back to the EC2 instance role. See [AWS credentials](../administration/aws-credentials.md). +- The [Amazon S3](../pipeline/outputs/s3.md) authorization endpoint and the [Azure Blob](../pipeline/outputs/azure_blob.md) configuration endpoint now verify TLS certificates using the output's `tls.verify` and certificate authority settings. Earlier versions didn't verify them. A configuration endpoint that fails verification stops Fluent Bit from starting. +- Regular expression searches that exceed a fixed work limit fail. Filters, routing, and multiline parsing treat a failed search as no match, while some include and exclude lists in metrics inputs treat it as a match. See [Regular expression format](../pipeline/parsers/regular-expression.md). +- The [Prometheus text file](../pipeline/inputs/prometheus-textfile.md) input sends one merged payload for each scrape and adds a `fluentbit_textfile_scrape_error` metric. + ## Fluent Bit v5.1 ### Debian and Ubuntu package upgrades restart the service diff --git a/installation/whats-new-in-fluent-bit-v5.md b/installation/whats-new-in-fluent-bit-v5.md index 98a796bd9..e628ede44 100644 --- a/installation/whats-new-in-fluent-bit-v5.md +++ b/installation/whats-new-in-fluent-bit-v5.md @@ -4,6 +4,32 @@ This page gives a quick user-focused overview of the main changes in the Fluent For migration-impacting changes, see [Upgrade notes](upgrade-notes.md). +## Fluent Bit v5.2 + +Fluent Bit `v5.2` is the upcoming release developed on `master`. It adds Fluent Typed Scripting (FTS) and improves log ingestion, processor execution, and buffering. The following highlights describe the development code ahead of the `5.2.0` release. + +### Fluent Typed Scripting + +The new `fts` processor lets you write Lua policies for logs, metrics, and traces. Typed access preserves integer widths, binary values, and nanosecond timestamps. Policies can also access OpenTelemetry resources, scopes, and attributes. FTS provides instruction, memory, and output limits, with configurable handling of policy errors. See the [FTS processor guide](https://github.com/fluent/fluent-bit/blob/master/plugins/processor_fts/README.md) for configuration and examples. + +### Performance and scalability + +#### OpenTelemetry log ingestion and processing + +The [OpenTelemetry input](../pipeline/inputs/opentelemetry.md) adds direct internal log decoding for JSON and Protobuf payloads. Compatible native log processor chains can run in input workers, reducing intermediate encoding and decoding. These improvements apply automatically to eligible pipelines. + +#### Processor chains + +Compatible native log processors can share processing work across a chain. Large log buffers are processed in windows to reduce temporary allocation pressure. Existing processor configurations continue to select the processors and their order. + +#### HTTP worker connection balancing + +The shared HTTP server balances accepted connections across workers. HTTP-based inputs configured with multiple workers can distribute connection handling more evenly. + +#### Buffer allocation + +Input chunks use adaptive size projections and append-sized storage hints to reduce allocation overhead as data arrives. See [Buffering](../pipeline/buffering.md) for storage configuration. + ## Fluent Bit v5.1 Fluent Bit `v5.1` focuses on network ingestion throughput, adds several new inputs and outputs, and hardens TLS and FIPS compliance options. diff --git a/vale-styles/FluentBit/Headings.yml b/vale-styles/FluentBit/Headings.yml index 445b1c9fa..a38a68816 100644 --- a/vale-styles/FluentBit/Headings.yml +++ b/vale-styles/FluentBit/Headings.yml @@ -67,8 +67,10 @@ exceptions: - FAQ - FlowCounter - Fluent Bit + - Fluent Typed Scripting - Fluentd - Forward + - FTS - GCP - GitLab - GitHub