Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,12 @@
description: High Performance Telemetry Agent for Logs, Metrics and Traces
---

# Fluent Bit documentation
# Fluent Bit 5.2 documentation

<figure><img src=".gitbook/assets/fluent_bit_logo.png" alt=""><figcaption></figcaption></figure>

This manual covers the upcoming Fluent Bit `5.2` release, developed on the `master` branch. For release highlights and migration guidance, see [What's new in Fluent Bit v5](installation/whats-new-in-fluent-bit-v5.md#fluent-bit-v52) and [Upgrade notes](installation/upgrade-notes.md#fluent-bit-v52).

[Fluent Bit](https://fluentbit.io) is a fast and lightweight telemetry agent for logs, metrics, and traces for Linux, macOS, Windows, and BSD family operating systems. Fluent Bit has been made with a strong focus on performance to allow the collection and processing of telemetry data from different sources without complexity.![](https://static.scarf.sh/a.png?x-pxid=71f0e011-761f-4c6f-9a89-38817887faae)

## Features
Expand Down Expand Up @@ -37,7 +39,7 @@ description: High Performance Telemetry Agent for Logs, Metrics and Traces

For more details about changes in each release, refer to the [official release notes](https://fluentbit.io/announcements/).

If you are upgrading from the Fluent Bit `4.2` series, start with [What's new in Fluent Bit v5](installation/whats-new-in-fluent-bit-v5.md) and [Upgrade notes](installation/upgrade-notes.md).
If you are upgrading from Fluent Bit `5.1` or an earlier series, review [What's new in Fluent Bit v5](installation/whats-new-in-fluent-bit-v5.md) and [Upgrade notes](installation/upgrade-notes.md).

## Fluent Bit, Fluentd, and CNCF

Expand Down
2 changes: 1 addition & 1 deletion SUMMARY.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Table of contents

* [Fluent Bit documentation](README.md)
* [Fluent Bit 5.2 documentation](README.md)

## About

Expand Down
2 changes: 2 additions & 0 deletions installation/downloads.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@

Fluent Bit is compatible with most x86-based, x86_64-based, arm32v7-based, and arm64v8-based systems.

This manual covers the upcoming Fluent Bit `5.2` release. Until `5.2.0` is released, the package and container download pages list published releases from earlier series. To try `5.2` before its release, [build the development source code](downloads/source.md#development).

## Build from source code

You can [build and install Fluent Bit from its source code](../installation/downloads/source.md). There are also platform-specific guides for building Fluent Bit from source on [macOS](../installation/downloads/macos.md#compile-from-source) and [Windows](../installation/downloads/windows.md#compile-from-source).
Expand Down
2 changes: 1 addition & 1 deletion installation/downloads/source.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ If you want to contribute to Fluent Bit, you should use the most recent code. Yo
git clone https://github.com/fluent/fluent-bit
```

The `master` branch is where the development of Fluent Bit happens. Development version users should expect issues when compiling or at run time.
The `master` branch is where development of the upcoming Fluent Bit `5.2` release happens. This manual documents that release series. Development version users should expect issues when compiling or at run time.

Fluent Bit users are encouraged to help test every development version to ensure a stable release.

Expand Down
29 changes: 29 additions & 0 deletions installation/upgrade-notes.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,35 @@ Release notes will be prepared in advance of a Git tag for a release. An officia

The tag drives the binary release process. Release binaries (containers and packages) will appear after a tag and its associated release note. This lets users to expect the new release binary to appear and allow/deny/update it as appropriate in their infrastructure.

## Fluent Bit v5.2

Fluent Bit `v5.2` is the upcoming release developed on `master`. Review [What's new in Fluent Bit v5.2](whats-new-in-fluent-bit-v5.md#fluent-bit-v52) before testing an upgrade from `5.1` or an earlier series.

### Native log processing

Version `5.2` changes internal log decoding, processor execution, and chunk allocation. Before deploying an upgrade, test representative log payloads and processor chains with your buffering and backpressure settings. Check record contents, routing, and delivery under load.

### Opt in to Fluent Typed Scripting
Comment thread
eschabell marked this conversation as resolved.
Resolved

The new `fts` processor is configured through the YAML `processors` section. Existing Lua filters continue to use the `lua` filter plugin. To adopt Fluent Typed Scripting (FTS), write policies against its typed API and configure an `fts` processor for each signal you want to process. See the [FTS processor guide](https://github.com/fluent/fluent-bit/blob/master/plugins/processor_fts/README.md) for its callback contract and resource limits.

### Stricter validation and security fixes

Version `5.2` tightens several checks that can change the behavior of existing configurations. Review the following changes before you upgrade:

- `https` URLs with an explicit port, such as `https://example.com:8443`, now always use TLS. Earlier versions connected to some of these URLs without TLS, including the [Amazon S3](../pipeline/outputs/s3.md) `authorization_endpoint_url` and the [Azure Blob](../pipeline/outputs/azure_blob.md) `configuration_endpoint_url`. If you set the [Stackdriver](../pipeline/outputs/stackdriver.md) or [Google Cloud Storage](../pipeline/outputs/gcs.md) `metadata_server` to an `https` address, it now fails with `TLS context required for URL`. Use the default `http` address instead.
- The [NATS](../pipeline/outputs/nats.md) output rejects tags that aren't valid NATS subjects, such as tags that contain spaces, `*`, or `>`, and drops those chunks. Rename the tags, or set `invalid_subject_action` to `encode`, which publishes those tags, and tags that already start with `_flb_encoded.`, to `_flb_encoded.` subjects that subscribers and permissions must then cover.
- The [File](../pipeline/outputs/file.md) output escapes more characters in the `out_file`, CSV, and LTSV formats. Update any tools that parse these files.
- The [Forward](../pipeline/inputs/forward.md) and [Syslog](../pipeline/inputs/syslog.md) inputs refuse to start when the Unix socket path is a symbolic link, such as `/dev/log` on many systemd-based systems, or is too long. The Syslog input also fails to start when `buffer_chunk_size` or `buffer_max_size` is out of range.
- The [Rewrite tag](../pipeline/filters/rewrite-tag.md) filter can no longer be used as an output processor, and Fluent Bit fails to start with that configuration.
- HTTP-based inputs close the connection, without a response, for HTTP/1 requests whose request line, headers, and closing blank line exceed `32KiB`. If clients send larger headers, raise `http_server.http1_max_header_size`. See [Pipeline](../administration/configuring-fluent-bit/yaml/pipeline-section.md).
- Inputs that validate `OAuth 2.0` tokens accept `JWKS` responses up to `1M`, including headers, which raises the earlier limit of 4,096 bytes, and apply a new `30s` time limit to the `JWKS` response. Adjust `oauth2.jwks_buffer_max_size` or `oauth2.jwks_response_timeout` if needed.
- The [OpenTelemetry](../pipeline/outputs/opentelemetry.md) output, when it uses HTTP/2 or gRPC, drops chunks whose responses exceed the new `http_client.*` size and header limits. Responses that exceed `http_client.response_timeout` are retried. See [Service](../administration/configuring-fluent-bit/yaml/service-section.md).
- `AWS_CONTAINER_CREDENTIALS_FULL_URI` now requires `http` hosts to be IP addresses in the allowed loopback and link-local ranges, or the exact ECS and EKS Pod Identity addresses, or the `Greengrass` endpoint. Host names that only started with or contained an allowed address, such as `127.example.com`, are rejected. If it's rejected, Fluent Bit can fall back to the EC2 instance role. See [AWS credentials](../administration/aws-credentials.md).
- The [Amazon S3](../pipeline/outputs/s3.md) authorization endpoint and the [Azure Blob](../pipeline/outputs/azure_blob.md) configuration endpoint now verify TLS certificates using the output's `tls.verify` and certificate authority settings. Earlier versions didn't verify them. A configuration endpoint that fails verification stops Fluent Bit from starting.
- Regular expression searches that exceed a fixed work limit fail. Filters, routing, and multiline parsing treat a failed search as no match, while some include and exclude lists in metrics inputs treat it as a match. See [Regular expression format](../pipeline/parsers/regular-expression.md).
- The [Prometheus text file](../pipeline/inputs/prometheus-textfile.md) input sends one merged payload for each scrape and adds a `fluentbit_textfile_scrape_error` metric.

## Fluent Bit v5.1

### Debian and Ubuntu package upgrades restart the service
Expand Down
26 changes: 26 additions & 0 deletions installation/whats-new-in-fluent-bit-v5.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,32 @@ This page gives a quick user-focused overview of the main changes in the Fluent

For migration-impacting changes, see [Upgrade notes](upgrade-notes.md).

## Fluent Bit v5.2

Fluent Bit `v5.2` is the upcoming release developed on `master`. It adds Fluent Typed Scripting (FTS) and improves log ingestion, processor execution, and buffering. The following highlights describe the development code ahead of the `5.2.0` release.

### Fluent Typed Scripting

The new `fts` processor lets you write Lua policies for logs, metrics, and traces. Typed access preserves integer widths, binary values, and nanosecond timestamps. Policies can also access OpenTelemetry resources, scopes, and attributes. FTS provides instruction, memory, and output limits, with configurable handling of policy errors. See the [FTS processor guide](https://github.com/fluent/fluent-bit/blob/master/plugins/processor_fts/README.md) for configuration and examples.

### Performance and scalability

#### OpenTelemetry log ingestion and processing

The [OpenTelemetry input](../pipeline/inputs/opentelemetry.md) adds direct internal log decoding for JSON and Protobuf payloads. Compatible native log processor chains can run in input workers, reducing intermediate encoding and decoding. These improvements apply automatically to eligible pipelines.

#### Processor chains

Compatible native log processors can share processing work across a chain. Large log buffers are processed in windows to reduce temporary allocation pressure. Existing processor configurations continue to select the processors and their order.

#### HTTP worker connection balancing

The shared HTTP server balances accepted connections across workers. HTTP-based inputs configured with multiple workers can distribute connection handling more evenly.

#### Buffer allocation

Input chunks use adaptive size projections and append-sized storage hints to reduce allocation overhead as data arrives. See [Buffering](../pipeline/buffering.md) for storage configuration.

## Fluent Bit v5.1

Fluent Bit `v5.1` focuses on network ingestion throughput, adds several new inputs and outputs, and hardens TLS and FIPS compliance options.
Expand Down
2 changes: 2 additions & 0 deletions vale-styles/FluentBit/Headings.yml
Original file line number Diff line number Diff line change
Expand Up @@ -67,8 +67,10 @@ exceptions:
- FAQ
- FlowCounter
- Fluent Bit
- Fluent Typed Scripting
- Fluentd
- Forward
- FTS
- GCP
- GitLab
- GitHub
Expand Down
Loading