diff --git a/administration/configuring-fluent-bit/yaml/service-section.md b/administration/configuring-fluent-bit/yaml/service-section.md index 672609956..0a4d2b0df 100644 --- a/administration/configuring-fluent-bit/yaml/service-section.md +++ b/administration/configuring-fluent-bit/yaml/service-section.md @@ -53,6 +53,7 @@ The following storage-related keys can be set as children to the `storage` key: | `storage.max_chunks_up` | Sets the number of chunks that can be `up` in memory for input plugins that use filesystem storage. | `128` | | `storage.metrics` | If `http_server` option is enabled in the main `service` section, this option registers a new endpoint where internal metrics of the storage layer can be consumed. For more details, see [Monitoring](../../monitoring.md). Possible values: `off` or `on`. | `off` | | `storage.path` | Sets a location to store streams and chunks of data. If this parameter isn't set, input plugins can't use filesystem buffering. | _none_ | +| `storage.rejected.limit` | Sets the maximum total size of the dead letter queue. Accepts size values such as `100M` or `1G`. When the limit is reached, new rejected chunks are skipped and a warning is logged. If not set, the DLQ size is unlimited. | _none_ | | `storage.rejected.path` | Sets the subdirectory name under `storage.path` for storing rejected chunks in the dead letter queue. | `rejected` | | `storage.sync` | Configures the synchronization mode used to store data in the file system. Using `full` increases the reliability of the filesystem buffer and ensures that data is guaranteed to be synced to the filesystem even if Fluent Bit crashes. On Linux, `full` corresponds with the `MAP_SYNC` option for [memory mapped files](https://man7.org/linux/man-pages/man2/mmap.2.html). Possible values: `normal`, `full`. | `normal` | | `storage.trim_files` | If enabled, Fluent Bit trims chunk files in the filesystem to reclaim disk space after data is flushed. Possible values: `off` or `on`. | `off` | diff --git a/administration/dead-letter-queue.md b/administration/dead-letter-queue.md index 3fcad6960..51fe9fb72 100644 --- a/administration/dead-letter-queue.md +++ b/administration/dead-letter-queue.md @@ -22,6 +22,18 @@ service: storage.rejected.path: rejected ``` +## Limit DLQ size + +To prevent the dead letter queue from growing unbounded, set `storage.rejected.limit` in the `service` section. When the total size of rejected chunks reaches this limit, Fluent Bit skips copying additional chunks and logs a warning. Accepts size values such as `100M` or `1G`. If not set, the DLQ size is unlimited. + +```yaml +service: + storage.path: /var/log/flb-storage/ + storage.keep.rejected: on + storage.rejected.path: rejected + storage.rejected.limit: 500M +``` + ## Format Each dead letter queue file is named using this format: