diff --git a/docs/admin-manual/system-tables/information_schema/file_cache_info.md b/docs/admin-manual/system-tables/information_schema/file_cache_info.md new file mode 100644 index 0000000000000..ef7e82176c23f --- /dev/null +++ b/docs/admin-manual/system-tables/information_schema/file_cache_info.md @@ -0,0 +1,82 @@ +--- +{ + "title": "file_cache_info", + "language": "en", + "description": "View block-level File Cache entries on BE nodes. This system table is supported in Doris 4.1 and later." +} +--- + +## Overview + +The `file_cache_info` system table exposes block-level File Cache entries on all alive BE nodes. Use it to analyze cache space by tablet, BE, cache path, or cache type, and to investigate cache skew or unexpected cache growth. + +:::tip Version + +This system table is supported in Doris 4.1 and later. It is not available in Doris 4.0.x. + +::: + +Each row represents one cached block. The result reflects the current cache state and can change while queries populate or evict cache entries. + +:::caution + +Querying this table scans persisted File Cache metadata on the selected BE nodes and can generate additional I/O when the cache contains many blocks. Avoid frequent unfiltered queries. Filter by `BE_ID` when you only need data from specific BE nodes. + +::: + +## Database + +`information_schema` + +## Table Information + +| Column Name | Type | Description | +|---|---|---| +| `HASH` | STRING | Hash of the remote file to which the cached block belongs. | +| `OFFSET` | BIGINT | Starting offset of the cached block in the remote file, in bytes. | +| `TABLET_ID` | BIGINT | ID of the tablet associated with the cached block. The value is `0` when the block is not associated with a tablet. | +| `SIZE` | BIGINT | Size of the cached block, in bytes. | +| `TYPE` | STRING | Cache type. Possible values are `normal`, `index`, `ttl`, and `disposable`. | +| `REMOTE_PATH` | STRING | Reserved for the remote file path. Currently returns an empty string. | +| `CACHE_PATH` | STRING | Local File Cache root path on the BE node. | +| `BE_ID` | BIGINT | ID of the BE node that stores the cached block. | + +## Examples + +### Query the cache entries of a tablet + +```sql +SELECT * +FROM information_schema.file_cache_info +WHERE TABLET_ID = 1761571031445; +``` + +```text ++----------------------------------+--------+---------------+-------+-------+-------------+------------------------------+---------------+ +| HASH | OFFSET | TABLET_ID | SIZE | TYPE | REMOTE_PATH | CACHE_PATH | BE_ID | ++----------------------------------+--------+---------------+-------+-------+-------------+------------------------------+---------------+ +| 468448215c52334ae5bee147259b1027 | 0 | 1761571031445 | 15120 | index | | /mnt/disk1/project/filecache | 1761571031251 | +| 71bb73d34cd8ffe280b16dd329df5ba1 | 15120 | 1761571031445 | 13117 | index | | /mnt/disk1/project/filecache | 1761571031251 | +| 77c6b69d1a7c4fe740a11bab5c1bbaa3 | 28237 | 1761571031445 | 12249 | index | | /mnt/disk1/project/filecache | 1761571031251 | ++----------------------------------+--------+---------------+-------+-------+-------------+------------------------------+---------------+ +``` + +### Summarize cache usage + +The following query summarizes the cache space occupied by a tablet on each BE and for each cache type: + +```sql +SELECT BE_ID, TABLET_ID, TYPE, SUM(SIZE) AS CACHE_BYTES +FROM information_schema.file_cache_info +WHERE TABLET_ID = 1761571031445 +GROUP BY BE_ID, TABLET_ID, TYPE +ORDER BY CACHE_BYTES DESC; +``` + +```text ++---------------+---------------+-------+-------------+ +| BE_ID | TABLET_ID | TYPE | CACHE_BYTES | ++---------------+---------------+-------+-------------+ +| 1761571031251 | 1761571031445 | index | 40486 | ++---------------+---------------+-------+-------------+ +``` diff --git a/docs/compute-storage-decoupled/file-cache/file-cache.md b/docs/compute-storage-decoupled/file-cache/file-cache.md index 817371586b8a7..a5104d3fdd41c 100644 --- a/docs/compute-storage-decoupled/file-cache/file-cache.md +++ b/docs/compute-storage-decoupled/file-cache/file-cache.md @@ -180,6 +180,10 @@ Doris provides both synchronous and asynchronous cache clearing methods: +### Cache Block Details + +Starting from Doris 4.1, you can query [`information_schema.file_cache_info`](../../admin-manual/system-tables/information_schema/file_cache_info.md) to inspect block-level cache entries and summarize cache space by tablet, BE, cache path, or cache type. Doris 4.0.x does not support this system table. + ### Hotspot Information Doris collects cache hotspot information for each compute group every 10 minutes and writes it to the internal system table `__internal_schema.cloud_cache_hotspot`. You can analyze hot data with the following queries to guide cache planning. diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/current/admin-manual/system-tables/information_schema/file_cache_info.md b/i18n/zh-CN/docusaurus-plugin-content-docs/current/admin-manual/system-tables/information_schema/file_cache_info.md new file mode 100644 index 0000000000000..ff1dc81452fee --- /dev/null +++ b/i18n/zh-CN/docusaurus-plugin-content-docs/current/admin-manual/system-tables/information_schema/file_cache_info.md @@ -0,0 +1,82 @@ +--- +{ + "title": "file_cache_info", + "language": "zh-CN", + "description": "查看各个 BE 节点上的 File Cache 缓存块明细。该系统表从 Doris 4.1 开始支持。" +} +--- + +## 概述 + +`file_cache_info` 系统表展示所有存活 BE 节点上的 File Cache 缓存块明细。可以使用该表按照 Tablet、BE、缓存路径或缓存类型分析空间占用，排查缓存分布不均或缓存空间异常增长等问题。 + +:::tip 版本说明 + +该系统表从 Doris 4.1 开始支持，Doris 4.0.x 不支持该系统表。 + +::: + +每一行表示一个缓存块。查询结果反映当前缓存状态；查询填充缓存或缓存淘汰发生时，结果可能随之变化。 + +:::caution 注意 + +查询该表会扫描所选 BE 节点上持久化的 File Cache 元数据。当缓存块数量较多时，查询会产生额外的 I/O。请避免频繁执行无过滤条件的查询；如果只需查看特定 BE 节点，请使用 `BE_ID` 过滤。 + +::: + +## 所属数据库 + +`information_schema` + +## 表信息 + +| 列名 | 类型 | 描述 | +|---|---|---| +| `HASH` | STRING | 缓存块所属远程文件的哈希值。 | +| `OFFSET` | BIGINT | 缓存块在远程文件中的起始偏移量，单位为字节。 | +| `TABLET_ID` | BIGINT | 缓存块关联的 Tablet ID。缓存块未关联 Tablet 时，该值为 `0`。 | +| `SIZE` | BIGINT | 缓存块大小，单位为字节。 | +| `TYPE` | STRING | 缓存类型。可能的值为 `normal`、`index`、`ttl` 和 `disposable`。 | +| `REMOTE_PATH` | STRING | 远程文件路径的预留字段，当前返回空字符串。 | +| `CACHE_PATH` | STRING | 缓存块所在 BE 节点的本地 File Cache 根目录。 | +| `BE_ID` | BIGINT | 保存该缓存块的 BE 节点 ID。 | + +## 示例 + +### 查询指定 Tablet 的缓存明细 + +```sql +SELECT * +FROM information_schema.file_cache_info +WHERE TABLET_ID = 1761571031445; +``` + +```text ++----------------------------------+--------+---------------+-------+-------+-------------+------------------------------+---------------+ +| HASH | OFFSET | TABLET_ID | SIZE | TYPE | REMOTE_PATH | CACHE_PATH | BE_ID | ++----------------------------------+--------+---------------+-------+-------+-------------+------------------------------+---------------+ +| 468448215c52334ae5bee147259b1027 | 0 | 1761571031445 | 15120 | index | | /mnt/disk1/project/filecache | 1761571031251 | +| 71bb73d34cd8ffe280b16dd329df5ba1 | 15120 | 1761571031445 | 13117 | index | | /mnt/disk1/project/filecache | 1761571031251 | +| 77c6b69d1a7c4fe740a11bab5c1bbaa3 | 28237 | 1761571031445 | 12249 | index | | /mnt/disk1/project/filecache | 1761571031251 | ++----------------------------------+--------+---------------+-------+-------+-------------+------------------------------+---------------+ +``` + +### 汇总缓存空间占用 + +以下查询按照 BE 和缓存类型汇总指定 Tablet 占用的缓存空间： + +```sql +SELECT BE_ID, TABLET_ID, TYPE, SUM(SIZE) AS CACHE_BYTES +FROM information_schema.file_cache_info +WHERE TABLET_ID = 1761571031445 +GROUP BY BE_ID, TABLET_ID, TYPE +ORDER BY CACHE_BYTES DESC; +``` + +```text ++---------------+---------------+-------+-------------+ +| BE_ID | TABLET_ID | TYPE | CACHE_BYTES | ++---------------+---------------+-------+-------------+ +| 1761571031251 | 1761571031445 | index | 40486 | ++---------------+---------------+-------+-------------+ +``` diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/current/compute-storage-decoupled/file-cache/file-cache.md b/i18n/zh-CN/docusaurus-plugin-content-docs/current/compute-storage-decoupled/file-cache/file-cache.md index 9294ba956db3b..2fb320da231f3 100644 --- a/i18n/zh-CN/docusaurus-plugin-content-docs/current/compute-storage-decoupled/file-cache/file-cache.md +++ b/i18n/zh-CN/docusaurus-plugin-content-docs/current/compute-storage-decoupled/file-cache/file-cache.md @@ -180,6 +180,10 @@ Doris 提供同步与异步两种缓存清理方式： +### 缓存块明细 + +从 Doris 4.1 开始，可以查询 [`information_schema.file_cache_info`](../../admin-manual/system-tables/information_schema/file_cache_info.md)，查看缓存块明细，并按照 Tablet、BE、缓存路径或缓存类型汇总缓存空间。Doris 4.0.x 不支持该系统表。 + ### 热点信息 Doris 每 10 分钟收集各计算组的缓存热点信息，并写入内部系统表 `__internal_schema.cloud_cache_hotspot`。可通过以下查询语句分析热点数据，指导缓存规划。 diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/version-4.x/admin-manual/system-tables/information_schema/file_cache_info.md b/i18n/zh-CN/docusaurus-plugin-content-docs/version-4.x/admin-manual/system-tables/information_schema/file_cache_info.md new file mode 100644 index 0000000000000..ff1dc81452fee --- /dev/null +++ b/i18n/zh-CN/docusaurus-plugin-content-docs/version-4.x/admin-manual/system-tables/information_schema/file_cache_info.md @@ -0,0 +1,82 @@ +--- +{ + "title": "file_cache_info", + "language": "zh-CN", + "description": "查看各个 BE 节点上的 File Cache 缓存块明细。该系统表从 Doris 4.1 开始支持。" +} +--- + +## 概述 + +`file_cache_info` 系统表展示所有存活 BE 节点上的 File Cache 缓存块明细。可以使用该表按照 Tablet、BE、缓存路径或缓存类型分析空间占用，排查缓存分布不均或缓存空间异常增长等问题。 + +:::tip 版本说明 + +该系统表从 Doris 4.1 开始支持，Doris 4.0.x 不支持该系统表。 + +::: + +每一行表示一个缓存块。查询结果反映当前缓存状态；查询填充缓存或缓存淘汰发生时，结果可能随之变化。 + +:::caution 注意 + +查询该表会扫描所选 BE 节点上持久化的 File Cache 元数据。当缓存块数量较多时，查询会产生额外的 I/O。请避免频繁执行无过滤条件的查询；如果只需查看特定 BE 节点，请使用 `BE_ID` 过滤。 + +::: + +## 所属数据库 + +`information_schema` + +## 表信息 + +| 列名 | 类型 | 描述 | +|---|---|---| +| `HASH` | STRING | 缓存块所属远程文件的哈希值。 | +| `OFFSET` | BIGINT | 缓存块在远程文件中的起始偏移量，单位为字节。 | +| `TABLET_ID` | BIGINT | 缓存块关联的 Tablet ID。缓存块未关联 Tablet 时，该值为 `0`。 | +| `SIZE` | BIGINT | 缓存块大小，单位为字节。 | +| `TYPE` | STRING | 缓存类型。可能的值为 `normal`、`index`、`ttl` 和 `disposable`。 | +| `REMOTE_PATH` | STRING | 远程文件路径的预留字段，当前返回空字符串。 | +| `CACHE_PATH` | STRING | 缓存块所在 BE 节点的本地 File Cache 根目录。 | +| `BE_ID` | BIGINT | 保存该缓存块的 BE 节点 ID。 | + +## 示例 + +### 查询指定 Tablet 的缓存明细 + +```sql +SELECT * +FROM information_schema.file_cache_info +WHERE TABLET_ID = 1761571031445; +``` + +```text ++----------------------------------+--------+---------------+-------+-------+-------------+------------------------------+---------------+ +| HASH | OFFSET | TABLET_ID | SIZE | TYPE | REMOTE_PATH | CACHE_PATH | BE_ID | ++----------------------------------+--------+---------------+-------+-------+-------------+------------------------------+---------------+ +| 468448215c52334ae5bee147259b1027 | 0 | 1761571031445 | 15120 | index | | /mnt/disk1/project/filecache | 1761571031251 | +| 71bb73d34cd8ffe280b16dd329df5ba1 | 15120 | 1761571031445 | 13117 | index | | /mnt/disk1/project/filecache | 1761571031251 | +| 77c6b69d1a7c4fe740a11bab5c1bbaa3 | 28237 | 1761571031445 | 12249 | index | | /mnt/disk1/project/filecache | 1761571031251 | ++----------------------------------+--------+---------------+-------+-------+-------------+------------------------------+---------------+ +``` + +### 汇总缓存空间占用 + +以下查询按照 BE 和缓存类型汇总指定 Tablet 占用的缓存空间： + +```sql +SELECT BE_ID, TABLET_ID, TYPE, SUM(SIZE) AS CACHE_BYTES +FROM information_schema.file_cache_info +WHERE TABLET_ID = 1761571031445 +GROUP BY BE_ID, TABLET_ID, TYPE +ORDER BY CACHE_BYTES DESC; +``` + +```text ++---------------+---------------+-------+-------------+ +| BE_ID | TABLET_ID | TYPE | CACHE_BYTES | ++---------------+---------------+-------+-------------+ +| 1761571031251 | 1761571031445 | index | 40486 | ++---------------+---------------+-------+-------------+ +``` diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/version-4.x/compute-storage-decoupled/file-cache/file-cache.md b/i18n/zh-CN/docusaurus-plugin-content-docs/version-4.x/compute-storage-decoupled/file-cache/file-cache.md index 9294ba956db3b..2fb320da231f3 100644 --- a/i18n/zh-CN/docusaurus-plugin-content-docs/version-4.x/compute-storage-decoupled/file-cache/file-cache.md +++ b/i18n/zh-CN/docusaurus-plugin-content-docs/version-4.x/compute-storage-decoupled/file-cache/file-cache.md @@ -180,6 +180,10 @@ Doris 提供同步与异步两种缓存清理方式： +### 缓存块明细 + +从 Doris 4.1 开始，可以查询 [`information_schema.file_cache_info`](../../admin-manual/system-tables/information_schema/file_cache_info.md)，查看缓存块明细，并按照 Tablet、BE、缓存路径或缓存类型汇总缓存空间。Doris 4.0.x 不支持该系统表。 + ### 热点信息 Doris 每 10 分钟收集各计算组的缓存热点信息，并写入内部系统表 `__internal_schema.cloud_cache_hotspot`。可通过以下查询语句分析热点数据，指导缓存规划。 diff --git a/sidebars.ts b/sidebars.ts index 9d4b1f83dd1ca..e8c3b23b22d5f 100644 --- a/sidebars.ts +++ b/sidebars.ts @@ -978,6 +978,7 @@ const sidebars: SidebarsConfig = { 'admin-manual/system-tables/information_schema/columns', 'admin-manual/system-tables/information_schema/engines', 'admin-manual/system-tables/information_schema/events', + 'admin-manual/system-tables/information_schema/file_cache_info', 'admin-manual/system-tables/information_schema/file_cache_statistics', 'admin-manual/system-tables/information_schema/files', 'admin-manual/system-tables/information_schema/global_variables', diff --git a/versioned_docs/version-4.x/admin-manual/system-tables/information_schema/file_cache_info.md b/versioned_docs/version-4.x/admin-manual/system-tables/information_schema/file_cache_info.md new file mode 100644 index 0000000000000..ef7e82176c23f --- /dev/null +++ b/versioned_docs/version-4.x/admin-manual/system-tables/information_schema/file_cache_info.md @@ -0,0 +1,82 @@ +--- +{ + "title": "file_cache_info", + "language": "en", + "description": "View block-level File Cache entries on BE nodes. This system table is supported in Doris 4.1 and later." +} +--- + +## Overview + +The `file_cache_info` system table exposes block-level File Cache entries on all alive BE nodes. Use it to analyze cache space by tablet, BE, cache path, or cache type, and to investigate cache skew or unexpected cache growth. + +:::tip Version + +This system table is supported in Doris 4.1 and later. It is not available in Doris 4.0.x. + +::: + +Each row represents one cached block. The result reflects the current cache state and can change while queries populate or evict cache entries. + +:::caution + +Querying this table scans persisted File Cache metadata on the selected BE nodes and can generate additional I/O when the cache contains many blocks. Avoid frequent unfiltered queries. Filter by `BE_ID` when you only need data from specific BE nodes. + +::: + +## Database + +`information_schema` + +## Table Information + +| Column Name | Type | Description | +|---|---|---| +| `HASH` | STRING | Hash of the remote file to which the cached block belongs. | +| `OFFSET` | BIGINT | Starting offset of the cached block in the remote file, in bytes. | +| `TABLET_ID` | BIGINT | ID of the tablet associated with the cached block. The value is `0` when the block is not associated with a tablet. | +| `SIZE` | BIGINT | Size of the cached block, in bytes. | +| `TYPE` | STRING | Cache type. Possible values are `normal`, `index`, `ttl`, and `disposable`. | +| `REMOTE_PATH` | STRING | Reserved for the remote file path. Currently returns an empty string. | +| `CACHE_PATH` | STRING | Local File Cache root path on the BE node. | +| `BE_ID` | BIGINT | ID of the BE node that stores the cached block. | + +## Examples + +### Query the cache entries of a tablet + +```sql +SELECT * +FROM information_schema.file_cache_info +WHERE TABLET_ID = 1761571031445; +``` + +```text ++----------------------------------+--------+---------------+-------+-------+-------------+------------------------------+---------------+ +| HASH | OFFSET | TABLET_ID | SIZE | TYPE | REMOTE_PATH | CACHE_PATH | BE_ID | ++----------------------------------+--------+---------------+-------+-------+-------------+------------------------------+---------------+ +| 468448215c52334ae5bee147259b1027 | 0 | 1761571031445 | 15120 | index | | /mnt/disk1/project/filecache | 1761571031251 | +| 71bb73d34cd8ffe280b16dd329df5ba1 | 15120 | 1761571031445 | 13117 | index | | /mnt/disk1/project/filecache | 1761571031251 | +| 77c6b69d1a7c4fe740a11bab5c1bbaa3 | 28237 | 1761571031445 | 12249 | index | | /mnt/disk1/project/filecache | 1761571031251 | ++----------------------------------+--------+---------------+-------+-------+-------------+------------------------------+---------------+ +``` + +### Summarize cache usage + +The following query summarizes the cache space occupied by a tablet on each BE and for each cache type: + +```sql +SELECT BE_ID, TABLET_ID, TYPE, SUM(SIZE) AS CACHE_BYTES +FROM information_schema.file_cache_info +WHERE TABLET_ID = 1761571031445 +GROUP BY BE_ID, TABLET_ID, TYPE +ORDER BY CACHE_BYTES DESC; +``` + +```text ++---------------+---------------+-------+-------------+ +| BE_ID | TABLET_ID | TYPE | CACHE_BYTES | ++---------------+---------------+-------+-------------+ +| 1761571031251 | 1761571031445 | index | 40486 | ++---------------+---------------+-------+-------------+ +``` diff --git a/versioned_docs/version-4.x/compute-storage-decoupled/file-cache/file-cache.md b/versioned_docs/version-4.x/compute-storage-decoupled/file-cache/file-cache.md index 817371586b8a7..a5104d3fdd41c 100644 --- a/versioned_docs/version-4.x/compute-storage-decoupled/file-cache/file-cache.md +++ b/versioned_docs/version-4.x/compute-storage-decoupled/file-cache/file-cache.md @@ -180,6 +180,10 @@ Doris provides both synchronous and asynchronous cache clearing methods: +### Cache Block Details + +Starting from Doris 4.1, you can query [`information_schema.file_cache_info`](../../admin-manual/system-tables/information_schema/file_cache_info.md) to inspect block-level cache entries and summarize cache space by tablet, BE, cache path, or cache type. Doris 4.0.x does not support this system table. + ### Hotspot Information Doris collects cache hotspot information for each compute group every 10 minutes and writes it to the internal system table `__internal_schema.cloud_cache_hotspot`. You can analyze hot data with the following queries to guide cache planning. diff --git a/versioned_sidebars/version-4.x-sidebars.json b/versioned_sidebars/version-4.x-sidebars.json index 95f8ddedf7006..adf3a957336d3 100644 --- a/versioned_sidebars/version-4.x-sidebars.json +++ b/versioned_sidebars/version-4.x-sidebars.json @@ -1153,6 +1153,7 @@ "admin-manual/system-tables/information_schema/columns", "admin-manual/system-tables/information_schema/engines", "admin-manual/system-tables/information_schema/events", + "admin-manual/system-tables/information_schema/file_cache_info", "admin-manual/system-tables/information_schema/file_cache_statistics", "admin-manual/system-tables/information_schema/files", "admin-manual/system-tables/information_schema/global_variables",