Skip to content

docs: administration: document processor hold metrics and reload validation #2815

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
eschabell merged 2 commits into from
Nov 9, 2026
Merged

docs: administration: document processor hold metrics and reload validation #2815

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
f4aa5f1
docs: administration: document processor hold metrics and reload vali…
eschabell Nov 6, 2026
dedc3a7
docs: administration: address review feedback on processor validation…
eschabell Nov 7, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions administration/hot-reload.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,19 @@ If Fluent Bit can't preserve the directory while reloading, it stops the reload,
[reload] copying configuration path failed. Reloading is halted
```

## Processor validation

Fluent Bit version 5.2 and greater validates the processors attached to every input and output before it replaces the running pipeline. If this validation fails, Fluent Bit stops the reload, keeps the previous configuration running, and logs:

```text
[reload] processor configuration preflight failed
[reload] reloaded config is invalid. Reloading is halted
```

Validation checks the property names and values of every processor, including [filters used as processors](../pipeline/processors/filters.md). Deeper checks run only for processors that support them: the `fts` processor, and the Lua and Wasm filters used as processors, unless the filter sets `active: false`.

Validation doesn't catch every error. Errors that only appear when a processor starts, such as an invalid regular expression in a `content_modifier` processor, are found after Fluent Bit has stopped the previous pipeline. In that case, the reload is aborted and Fluent Bit exits instead of keeping the previous configuration. Test configuration changes in a test environment before you reload a production instance. The `--dry-run` option runs the same validation, so it doesn't catch these errors either.

## How to reload

After updating the configuration, use one of the following methods to perform a hot reload:
Expand Down
3 changes: 3 additions & 0 deletions administration/monitoring.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -236,10 +236,13 @@
| `fluentbit_output_retried_records_total` | name: the name or alias for the output instance | The number of log records that experienced a retry. This metric is calculated at the chunk level, the count increased when an entire chunk is marked for retry. An output plugin might perform multiple actions that generate many error messages when uploading a single chunk. | counter | records |
| `fluentbit_output_retries_failed_total` | name: the name or alias for the output instance | The number of times that retries expired for a chunk. Each plugin configures a `Retry_Limit`, which applies to chunks. When the `Retry_Limit` is exceeded, the chunk is discarded and this metric is incremented. | counter | chunks |
| `fluentbit_output_retries_total` | name: the name or alias for the output instance | The number of times this output instance requested a retry for a chunk. | counter | chunks |
| `fluentbit_processor_accounting_errors_total` | scope: the plugin type that owns the processor (`input` or `output`), owner: the name or alias of the input or output instance that owns the processor chain, processor: the name of the processor unit, stage: the zero-based position of this processor in the chain, signal: the data type being processed (`logs`, `metrics`, `traces`, or `profiles`) | The total number of invalid record retention accounting operations by a processor unit that holds log records. Fluent Bit also logs `invalid retention accounting operation` the first time this happens for each processor unit. Only reported for log processors attached to inputs. | counter | errors |

Check warning on line 239 in administration/monitoring.md

View workflow job for this annotation

GitHub Actions / runner / vale 

[vale] reported by reviewdog 🐶
[FluentBit.SentenceLengthLong] Improve readability by using fewer than 35 words in this sentence.

Raw Output:
{"message":"[FluentBit.SentenceLengthLong] Improve readability by using fewer than 35 words in this sentence.","location":{"path":"administration/monitoring.md","range":{"start":{"line":239,"column":51},"end":{"line":239,"column":56}}},"severity":"INFO","code":{"value":"FluentBit.SentenceLengthLong"}}
| `fluentbit_processor_errors_total` | scope: the plugin type that owns the processor (`input` or `output`), owner: the name or alias of the input or output instance that owns the processor chain, processor: the name of the processor unit, stage: the zero-based position of this processor in the chain, signal: the data type being processed (`logs`, `metrics`, `traces`, or `profiles`) | The total number of errors that a processor unit encountered. | counter | errors |
| `fluentbit_processor_invocations_total` | scope: the plugin type that owns the processor (`input` or `output`), owner: the name or alias of the input or output instance that owns the processor chain, processor: the name of the processor unit, stage: the zero-based position of this processor in the chain, signal: the data type being processed (`logs`, `metrics`, `traces`, or `profiles`) | The total number of processor unit invocations. | counter | invocations |
| `fluentbit_processor_items_add_total` | scope: the plugin type that owns the processor (`input` or `output`), owner: the name or alias of the input or output instance that owns the processor chain, processor: the name of the processor unit, stage: the zero-based position of this processor in the chain, signal: the data type being processed (`logs`, `metrics`, `traces`, or `profiles`) | The total number of items that a processor unit added. Incremented when `items_out_total` exceeds `items_in_total`. | counter | records |
| `fluentbit_processor_items_drop_total` | scope: the plugin type that owns the processor (`input` or `output`), owner: the name or alias of the input or output instance that owns the processor chain, processor: the name of the processor unit, stage: the zero-based position of this processor in the chain, signal: the data type being processed (`logs`, `metrics`, `traces`, or `profiles`) | The total number of items that a processor unit dropped. Incremented when `items_in_total` exceeds `items_out_total`. | counter | records |
| `fluentbit_processor_items_held_current` | scope: the plugin type that owns the processor (`input` or `output`), owner: the name or alias of the input or output instance that owns the processor chain, processor: the name of the processor unit, stage: the zero-based position of this processor in the chain, signal: the data type being processed (`logs`, `metrics`, `traces`, or `profiles`) | The number of log records currently held by a processor unit, such as records retained by the `fts` processor. Only reported for log processors attached to inputs. | gauge | records |

Check warning on line 244 in administration/monitoring.md

View workflow job for this annotation

GitHub Actions / runner / vale 

[vale] reported by reviewdog 🐶
[FluentBit.SentenceLengthLong] Improve readability by using fewer than 35 words in this sentence.

Raw Output:
{"message":"[FluentBit.SentenceLengthLong] Improve readability by using fewer than 35 words in this sentence.","location":{"path":"administration/monitoring.md","range":{"start":{"line":244,"column":46},"end":{"line":244,"column":51}}},"severity":"INFO","code":{"value":"FluentBit.SentenceLengthLong"}}
| `fluentbit_processor_items_held_total` | scope: the plugin type that owns the processor (`input` or `output`), owner: the name or alias of the input or output instance that owns the processor chain, processor: the name of the processor unit, stage: the zero-based position of this processor in the chain, signal: the data type being processed (`logs`, `metrics`, `traces`, or `profiles`) | The total number of log records that a processor unit admitted for retention. Only reported for log processors attached to inputs. | counter | records |

Check warning on line 245 in administration/monitoring.md

View workflow job for this annotation

GitHub Actions / runner / vale 

[vale] reported by reviewdog 🐶
[FluentBit.SentenceLengthLong] Improve readability by using fewer than 35 words in this sentence.

Raw Output:
{"message":"[FluentBit.SentenceLengthLong] Improve readability by using fewer than 35 words in this sentence.","location":{"path":"administration/monitoring.md","range":{"start":{"line":245,"column":44},"end":{"line":245,"column":49}}},"severity":"INFO","code":{"value":"FluentBit.SentenceLengthLong"}}
| `fluentbit_processor_items_in_total` | scope: the plugin type that owns the processor (`input` or `output`), owner: the name or alias of the input or output instance that owns the processor chain, processor: the name of the processor unit, stage: the zero-based position of this processor in the chain, signal: the data type being processed (`logs`, `metrics`, `traces`, or `profiles`) | The total number of items that a processor unit received. | counter | records |
| `fluentbit_processor_items_out_total` | scope: the plugin type that owns the processor (`input` or `output`), owner: the name or alias of the input or output instance that owns the processor chain, processor: the name of the processor unit, stage: the zero-based position of this processor in the chain, signal: the data type being processed (`logs`, `metrics`, `traces`, or `profiles`) | The total number of items that a processor unit produced. | counter | records |
| `fluentbit_uptime` | hostname: the hostname on running Fluent Bit | The number of seconds that Fluent Bit has been running. | counter | seconds |
Expand Down
Loading