From 9e6e33013ca50e6dbe4da99da85c32b65ac578e3 Mon Sep 17 00:00:00 2001 From: "Eric D. Schabell" Date: Mon, 18 May 2026 20:27:27 +0200 Subject: [PATCH 1/2] docs: storage: dead-letter-queue: document new config parameter - added storage.rejected.limit and "Limit DLQ size" section with YAML example Signed-off-by: Eric D. Schabell --- administration/dead-letter-queue.md | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/administration/dead-letter-queue.md b/administration/dead-letter-queue.md index 3fcad6960..51fe9fb72 100644 --- a/administration/dead-letter-queue.md +++ b/administration/dead-letter-queue.md @@ -22,6 +22,18 @@ service: storage.rejected.path: rejected ``` +## Limit DLQ size + +To prevent the dead letter queue from growing unbounded, set `storage.rejected.limit` in the `service` section. When the total size of rejected chunks reaches this limit, Fluent Bit skips copying additional chunks and logs a warning. Accepts size values such as `100M` or `1G`. If not set, the DLQ size is unlimited. + +```yaml +service: + storage.path: /var/log/flb-storage/ + storage.keep.rejected: on + storage.rejected.path: rejected + storage.rejected.limit: 500M +``` + ## Format Each dead letter queue file is named using this format: From 9b4490359afe8f9631247a65911d1a7dcab0d1b3 Mon Sep 17 00:00:00 2001 From: "Eric D. Schabell" Date: Mon, 18 May 2026 20:29:42 +0200 Subject: [PATCH 2/2] docs: storage: config: yaml: service-section: added new config parameter - add storage.rejected.limit row to storage parameters table Signed-off-by: Eric D. Schabell --- administration/configuring-fluent-bit/yaml/service-section.md | 1 + 1 file changed, 1 insertion(+) diff --git a/administration/configuring-fluent-bit/yaml/service-section.md b/administration/configuring-fluent-bit/yaml/service-section.md index 672609956..0a4d2b0df 100644 --- a/administration/configuring-fluent-bit/yaml/service-section.md +++ b/administration/configuring-fluent-bit/yaml/service-section.md @@ -53,6 +53,7 @@ The following storage-related keys can be set as children to the `storage` key: | `storage.max_chunks_up` | Sets the number of chunks that can be `up` in memory for input plugins that use filesystem storage. | `128` | | `storage.metrics` | If `http_server` option is enabled in the main `service` section, this option registers a new endpoint where internal metrics of the storage layer can be consumed. For more details, see [Monitoring](../../monitoring.md). Possible values: `off` or `on`. | `off` | | `storage.path` | Sets a location to store streams and chunks of data. If this parameter isn't set, input plugins can't use filesystem buffering. | _none_ | +| `storage.rejected.limit` | Sets the maximum total size of the dead letter queue. Accepts size values such as `100M` or `1G`. When the limit is reached, new rejected chunks are skipped and a warning is logged. If not set, the DLQ size is unlimited. | _none_ | | `storage.rejected.path` | Sets the subdirectory name under `storage.path` for storing rejected chunks in the dead letter queue. | `rejected` | | `storage.sync` | Configures the synchronization mode used to store data in the file system. Using `full` increases the reliability of the filesystem buffer and ensures that data is guaranteed to be synced to the filesystem even if Fluent Bit crashes. On Linux, `full` corresponds with the `MAP_SYNC` option for [memory mapped files](https://man7.org/linux/man-pages/man2/mmap.2.html). Possible values: `normal`, `full`. | `normal` | | `storage.trim_files` | If enabled, Fluent Bit trims chunk files in the filesystem to reclaim disk space after data is flushed. Possible values: `off` or `on`. | `off` |