volcengine · yaozheng-fang · Jul 19, 2026 · Jul 19, 2026 · Jul 19, 2026
diff --git a/docs/content/docs/framework/frontend.en.mdx b/docs/content/docs/framework/frontend.en.mdx
@@ -15,7 +15,12 @@ VeADK ships a React frontend that renders [A2UI](/en/docs/framework/a2ui) (agent
 ### What it can do
 
 - **Chat**: multi-turn conversations that render streamed A2UI cards, plus thinking, tool calls, token usage and timing.
-- **Agent picker**: switch agents from the top-left; hover an agent to see its model and mounted tools.
+- **Multimodal messages**: upload images, TXT/Markdown, PDF, and video; preview
+  and replay both user and model media from history. Chat images use compact
+  thumbnails and open in a zoomable full-screen viewer.
+- **Skill and sub-agent invocation**: type `/` to select a mounted skill or `@` to route the turn to an eligible sub-agent.
+- **Agent picker**: switch agents from the top-left; long lists scroll within
+  the viewport, and hovering an agent shows its model and mounted tools.
 - **Session history**: auto-saved, time-sorted, reopen or delete.
 - **Smart search**: the *Session* source full-text-searches the current agent's history; the *Web* source calls the agent's mounted web-search tool live (using credentials from the server's environment variables).
 - **Add an AgentKit agent**: paste a URL + API key to connect a remote agent over the ADK protocol; it then appears in the picker.
@@ -64,6 +69,62 @@ cd frontend && npm run dev                    # http://localhost:5173 (proxies A
   Outside dev mode, if the built frontend directory is not found, the command fails and asks you to run `npm run build` first (or use `--dev` with the Vite dev server).
 </Callout>
 
+### Multimodal attachments and storage
+
+The composer accepts PNG, JPEG, WebP, GIF, TXT, Markdown, PDF, MP4, WebM, and QuickTime files. The default per-file limit is 20 MB. The browser uploads binary form data and never writes attachment base64 into the session.
+
+Media bytes are stored separately from the ADK Session:
+
+- Local mode writes to `/tmp/veadk-media/apps/.../sessions/.../media/<media-id>/` by default. `content` contains the bytes and `metadata.json` contains the name, MIME type, size, and hash.
+- TOS mode writes the same two objects below `veadk-media/users/<encoded-username>/apps/<app>/sessions/<session>/media/<media-id>/` by default. The username is the outer tenant prefix, isolating each user's objects; username, app, and session segments are URL-encoded.
+- Session Events contain only a stable Google GenAI `FileData` reference such as `veadk-media://apps/.../media/<media-id>`; reopening history fetches the original object through the media API.
+
+Before a model call, TXT/Markdown is decoded into `Part.text`; images and video
+are read from the configured backend and converted to `Part.inline_data`, while
+PDFs are automatically rendered to page PNGs. The PDF rendering runtime is
+included in VeADK's default dependencies, with no extra installation or Agent
+callback required. Model-returned `inline_data` is persisted first, then
+replaced with a stable reference before the Event is saved or emitted over SSE.
+A 15-minute TOS signed URL is used only for browser download and preview, never
+as the model's `FileData` URI.
+
+When a cloud Agent Runtime is selected, upload, preview, and deletion still use
+Studio's own `/web/media` API instead of being forwarded to the Runtime. Studio
+resolves stable references into model-ready Parts only while proxying
+`/run_sse`. It preserves the original `veadkMedia` metadata so history reloads
+the original file type and name. This works with both `/tmp` and TOS and does
+not require the remote Runtime to mount media HTTP routes.
+
+| Environment variable | Default | Description |
+| :-- | :-- | :-- |
+| `VEADK_MEDIA_STORAGE` | `local` | Select `local` or `tos`. |
+| `VEADK_MEDIA_LOCAL_DIR` | `/tmp/veadk-media` | Local media root. |
+| `VEADK_MEDIA_MAX_FILE_BYTES` | `20971520` | Byte limit for one upload or model output. |
+| `VEADK_MEDIA_TOS_PREFIX` | `veadk-media` | TOS object-key prefix. |
+| `DATABASE_TOS_BUCKET` | — | TOS bucket. |
+| `DATABASE_TOS_REGION` / `DATABASE_TOS_ENDPOINT` | cloud-aware defaults | TOS region and endpoint. |
+| `VOLCENGINE_ACCESS_KEY` / `VOLCENGINE_SECRET_KEY` | — | TOS credentials. |
+| `VOLCENGINE_SESSION_TOKEN` | — | Optional temporary credential token. |
+
+Removing an unsent attachment deletes its object; deleting a Session cleans up all media scoped to that Session. Local mode needs no additional configuration, but `/tmp` may be cleared when the process or host is replaced. Use TOS when attachments must remain durable. Minimal TOS configuration:
+
+```bash
+export VEADK_MEDIA_STORAGE=tos
+export DATABASE_TOS_BUCKET=<bucket>
+export DATABASE_TOS_REGION=cn-beijing
+export VOLCENGINE_ACCESS_KEY=<access-key>
+export VOLCENGINE_SECRET_KEY=<secret-key>
+veadk frontend --agents-dir examples
+```
+
+### Using skills and sub-agents
+
+Type `/` in the composer to search skills mounted on the current agent. Type `@` to search eligible descendants in its agent tree. Use the arrow keys to move, Enter or Tab to select, and Escape to close the menu. A selection becomes a removable chip instead of remaining ordinary message text.
+
+After selecting a sub-agent, the `/` menu shows skills mounted on that target. Changing or removing the target clears its selected skills, preventing a skill from being sent to an agent that does not own it. `task` and `single_turn` workflow nodes remain visible in the topology but cannot be selected with `@`.
+
+The frontend sends structured `veadkInvocation` metadata instead of inferring invocation intent from the message string. The backend plugin directs ADK to call the skill tool or invoke `transfer_to_agent` one tree edge at a time until it reaches the target. The same metadata is written to the first Google GenAI `Part`, so reopening session history restores the `/skill` and `@agent` chips.
+
 ### IAM permissions for `veadk studio deploy`
 
 When `--iam-role` is omitted, the deploy command creates or reuses `VeADKFrontendServiceRole` and ensures that the role has these Volcengine system policies:
@@ -148,9 +209,9 @@ username (letters + digits, ≤16), stored locally and used as the `user_id`.
 
 <Callout type="info">
   Login state is cached: SSO via the `veadk_session` cookie, local mode via
-  `localStorage`. The session itself is created lazily on the first message (not
-  on page load). Logout is a local logout (clears the session and returns to the
-  login page).
+  `localStorage`. The session itself is created lazily on the first message or
+  attachment upload (not on page load). Logout is a local logout (clears the
+  session and returns to the login page).
 </Callout>
 
 ### How it works
@@ -161,6 +222,14 @@ username (letters + digits, ≤16), stored locally and used as the `user_id`.
 `adk/client.ts` calls `/list-apps`, creates a session, and streams `/run_sse`.
 </Step>
 
+<Step>
+`veadk.multimodal` validates uploads, manages local/TOS storage, resolves media references before model calls, and persists model-returned media before history is saved.
+</Step>
+
+<Step>
+`veadk.cli.frontend_invocation` exposes mounted skills and translates structured `/skill` and `@agent` selections into ADK skill-tool and agent-transfer directives.
+</Step>
+
 <Step>
 `a2ui/extract.ts` pulls A2UI messages out of the `send_a2ui_json_to_client` tool response (`validated_a2ui_json`).
 </Step>

diff --git a/docs/content/docs/framework/frontend.mdx b/docs/content/docs/framework/frontend.mdx
@@ -15,7 +15,10 @@ VeADK 自带一个 React 前端，用于渲染智能体经由 Google ADK API Ser
 ### 功能一览
 
 - **对话**：与智能体多轮对话，渲染流式返回的 A2UI 卡片，并展示思考过程、工具调用、Token 与用时。
-- **Agent 选择器**：左上角切换 agent；悬停可查看该 agent 的模型与挂载的工具。
+- **多模态消息**：上传图片、TXT/Markdown、PDF 与视频；用户附件和模型返回的媒体均支持预览与历史回放。
+  对话中的图片默认以紧凑尺寸展示，点击后可全屏缩放预览。
+- **技能与子 Agent 调用**：在输入框输入 `/` 选择挂载技能，输入 `@` 将本轮交给可选的子 Agent。
+- **Agent 选择器**：左上角切换 agent；长列表在视口内独立滚动，悬停可查看该 agent 的模型与挂载的工具。
 - **历史会话**：自动保存、按时间排序，可重新打开或删除。
 - **智能搜索**：「会话」源在当前 agent 的历史消息中做全文检索；「网页」源调用该 agent 挂载的联网搜索工具实时检索（使用服务端环境变量里的凭据）。
 - **添加 AgentKit 智能体**：填入访问地址 + API Key，按 ADK 协议接入远程 agent，接入后出现在选择器中。
@@ -64,6 +67,57 @@ cd frontend && npm run dev                    # http://localhost:5173（代理 A
   非开发模式下，若未找到已构建的前端目录，命令会报错提示先执行 `npm run build`（或改用 `--dev` 配合 Vite 开发服务器）。
 </Callout>
 
+### 多模态附件与存储
+
+输入框支持 PNG、JPEG、WebP、GIF、TXT、Markdown、PDF、MP4、WebM 和 QuickTime。默认单文件上限为 20 MB。浏览器使用二进制表单上传，不会把附件转成 base64 写入会话。
+
+媒体本体与 ADK Session 分开保存：
+
+- 本地模式默认写入 `/tmp/veadk-media/apps/.../sessions/.../media/<media-id>/`，其中 `content` 是文件本体，`metadata.json` 是文件名、MIME、大小和哈希等元数据。
+- TOS 模式默认写入 `veadk-media/users/<encoded-username>/apps/<app>/sessions/<session>/media/<media-id>/` 下的同名两个对象。用户名位于最外层租户 prefix，不同用户的数据相互隔离；username、app 和 session 路径段都会进行 URL 编码。
+- Session Event 只保存 Google GenAI `FileData` 稳定引用，例如 `veadk-media://apps/.../media/<media-id>`；重新打开历史会话时再通过媒体 API 读取本体。
+
+调用模型前，TXT/Markdown 会解码为 `Part.text`，图片和视频会从当前存储读取并转换为
+`Part.inline_data`，PDF 则自动渲染为逐页 PNG 图片。PDF 渲染运行时包含在 VeADK
+默认依赖中，不需要额外安装或配置 Agent callback。模型返回的 `inline_data` 会先落到
+媒体存储，再在 Event 持久化和 SSE 输出前改写为稳定引用。TOS 的 15 分钟签名 URL
+只用于浏览器下载/预览，不作为模型的 `FileData` URI。
+
+选择云端 Agent Runtime 时，上传、预览和删除仍由 Studio 自身的 `/web/media`
+接口处理，不会转发到 Runtime。Studio 仅在代理 `/run_sse` 时读取稳定引用，将内容转换为
+远端模型可直接消费的 Part；原始 `veadkMedia` 元数据会保留，因此历史消息仍按原文件类型
+和名称加载。这个流程同时支持默认 `/tmp` 与 TOS，无需远端 Runtime 挂载媒体 HTTP 路由。
+
+| 环境变量 | 默认值 | 说明 |
+| :-- | :-- | :-- |
+| `VEADK_MEDIA_STORAGE` | `local` | 选择 `local` 或 `tos`。 |
+| `VEADK_MEDIA_LOCAL_DIR` | `/tmp/veadk-media` | 本地媒体根目录。 |
+| `VEADK_MEDIA_MAX_FILE_BYTES` | `20971520` | 单个上传文件或模型输出的字节上限。 |
+| `VEADK_MEDIA_TOS_PREFIX` | `veadk-media` | TOS 对象 Key 前缀。 |
+| `DATABASE_TOS_BUCKET` | — | TOS Bucket。 |
+| `DATABASE_TOS_REGION` / `DATABASE_TOS_ENDPOINT` | 按云环境推导 | TOS Region 与 Endpoint。 |
+| `VOLCENGINE_ACCESS_KEY` / `VOLCENGINE_SECRET_KEY` | — | TOS 访问凭据。 |
+| `VOLCENGINE_SESSION_TOKEN` | — | 可选的临时凭据 Token。 |
+
+移除尚未发送的附件会立即删除对应对象；删除 Session 会清理该 Session 下的全部媒体。本地模式无需额外配置，但 `/tmp` 可能随进程或宿主机回收而被清理；需要持久保存附件时应使用 TOS。启用 TOS 的最小配置如下：
+
+```bash
+export VEADK_MEDIA_STORAGE=tos
+export DATABASE_TOS_BUCKET=<bucket>
+export DATABASE_TOS_REGION=cn-beijing
+export VOLCENGINE_ACCESS_KEY=<access-key>
+export VOLCENGINE_SECRET_KEY=<secret-key>
+veadk frontend --agents-dir examples
+```
+
+### 使用技能与子 Agent
+
+在输入框中输入 `/` 可搜索当前 Agent 挂载的技能，输入 `@` 可搜索其 Agent 树中允许转移的后代节点。使用上下方向键移动，按 Enter 或 Tab 选择，按 Escape 关闭菜单。选中项会变成可移除的 chip，不会作为普通文本留在消息中。
+
+选中子 Agent 后，`/` 菜单会改为展示目标 Agent 自己挂载的技能。切换或移除目标时会清空原有技能，避免把技能发送给没有挂载它的 Agent。`task` 和 `single_turn` 工作流节点仍会显示在拓扑中，但不能通过 `@` 选择。
+
+前端不会从消息字符串中猜测调用意图，而是发送结构化的 `veadkInvocation` metadata。后端插件据此要求 ADK 调用技能工具，或沿 Agent 树逐级调用 `transfer_to_agent`，直到到达目标节点。同一份 metadata 也会写入首个 Google GenAI `Part`，因此重新加载历史会话后仍能恢复 `/skill` 与 `@agent` chip。
+
 ### `veadk studio deploy` 的 IAM 权限
 
 未传入 `--iam-role` 时，部署命令会创建或复用 `VeADKFrontendServiceRole`，并确保该 Role 绑定以下火山引擎系统策略：
@@ -128,7 +182,7 @@ Google 同理，把 `OAUTH2_PROVIDER` 换成 `google`；Keycloak / Auth0 / Okta
 **无 SSO（本地用户名）** —— 不传上述参数时，登录页会让用户输入一个用户名（字母 + 数字，≤16 位），保存在本地并作为 `user_id`。
 
 <Callout type="info">
-  登录态会被缓存：SSO 走 `veadk_session` Cookie，本地模式走 `localStorage`。会话本身**在发送第一条消息时才创建**（而非打开页面时）。退出登录为本地登出（清除会话并回到登录页）。
+  登录态会被缓存：SSO 走 `veadk_session` Cookie，本地模式走 `localStorage`。会话本身**在发送第一条消息或上传第一个附件时才创建**（而非打开页面时）。退出登录为本地登出（清除会话并回到登录页）。
 </Callout>
 
 ### 工作原理
@@ -139,6 +193,14 @@ Google 同理，把 `OAUTH2_PROVIDER` 换成 `google`；Keycloak / Auth0 / Okta
 `adk/client.ts` 调用 `/list-apps`，创建会话，并通过 `/run_sse` 进行流式接收。
 </Step>
 
+<Step>
+`veadk.multimodal` 校验上传，统一管理本地/TOS 存储，在模型调用前解析媒体引用，并在历史持久化前保存模型返回的媒体。
+</Step>
+
+<Step>
+`veadk.cli.frontend_invocation` 暴露 Agent 挂载的技能，并把结构化的 `/skill`、`@agent` 选择转换为 ADK 技能工具与 Agent 转移指令。
+</Step>
+
 <Step>
 `a2ui/extract.ts` 从 `send_a2ui_json_to_client` 工具的响应（`validated_a2ui_json`）中提取 A2UI 消息。
 </Step>

diff --git a/examples/README.md b/examples/README.md
@@ -22,8 +22,12 @@ and a bilingual README (English + 中文).
 | 10 | [Agent routing](./10_agent_routing/) | Complex | A coordinator that delegates to specialists dynamically |
 | 11 | [Tracing](./11_tracing/) | Complex | Observe LLM/tool calls; dump or export spans |
 
-There is also [`a2ui_agent/`](./a2ui_agent/) — a demo of agent-driven UI, run with
-`veadk frontend --agents-dir examples`.
+There are also frontend-focused demos that run with
+`veadk frontend --agents-dir examples`:
+
+- [`a2ui_agent/`](./a2ui_agent/) demonstrates agent-driven UI.
+- [`multimodal_agent/`](./multimodal_agent/) analyzes images, TXT/Markdown,
+  PDFs, and videos uploaded from the chat composer.
 
 For a deployable **full app** (web UI + agent API in one container, shipped to
 Volcengine AgentKit via `veadk agentkit`), see [`basic-app/`](./basic-app/).

diff --git a/examples/README.zh.md b/examples/README.zh.md
@@ -21,8 +21,11 @@
 | 10 | [智能体路由](./10_agent_routing/) | 复杂 | 协调者动态委派给专家智能体 |
 | 11 | [链路追踪](./11_tracing/) | 复杂 | 观测大模型/工具调用；导出 span |
 
-另外还有 [`a2ui_agent/`](./a2ui_agent/) —— 一个由智能体驱动 UI 的示例，
-可通过 `veadk frontend --agents-dir examples` 运行。
+另外还有可通过 `veadk frontend --agents-dir examples` 运行的 frontend 示例：
+
+- [`a2ui_agent/`](./a2ui_agent/) 展示由智能体驱动的 UI。
+- [`multimodal_agent/`](./multimodal_agent/) 分析从聊天输入框上传的图片、
+  TXT/Markdown、PDF 和视频。
 
 如需一个可部署的**完整应用**（Web 前端 + Agent API 同处一个容器，通过
 `veadk agentkit` 部署到火山引擎 AgentKit），参见 [`basic-app/`](./basic-app/)。

diff --git a/examples/multimodal_agent/.env.example b/examples/multimodal_agent/.env.example
@@ -0,0 +1,13 @@
+# Copy this file to the repository-root `.env` before starting the frontend.
+
+MODEL_AGENT_PROVIDER=openai
+MODEL_AGENT_NAME=doubao-seed-2-1-pro-260628
+MODEL_AGENT_API_BASE=https://ark.cn-beijing.volces.com/api/v3/
+MODEL_AGENT_API_KEY=your-ark-api-key-here
+
+# Optional generation-tool overrides. The API keys fall back to
+# MODEL_AGENT_API_KEY when omitted.
+MODEL_IMAGE_NAME=doubao-seedream-5-0-260128
+MODEL_IMAGE_API_BASE=https://ark.cn-beijing.volces.com/api/v3/
+MODEL_VIDEO_NAME=doubao-seedance-2-0-260128
+MODEL_VIDEO_API_BASE=https://ark.cn-beijing.volces.com/api/v3/
diff --git a/examples/multimodal_agent/README.md b/examples/multimodal_agent/README.md
@@ -0,0 +1,36 @@
+# Multimodal Agent
+
+This demo uses the VeADK frontend's multimodal pipeline to analyze images,
+TXT/Markdown documents, PDFs, and videos, and mounts the `image_generate` and
+`video_generate` tools. The frontend uploads files, `MultimodalMediaPlugin`
+resolves them into Google GenAI Parts, and `doubao-seed-2-1-pro-260628` analyzes
+them. Image or video creation requests call the corresponding tool.
+
+## Run
+
+From the repository root:
+
+```bash
+cp examples/multimodal_agent/.env.example .env
+uv run veadk frontend --agents-dir examples --dev
+```
+
+Open <http://127.0.0.1:8000>, select `multimodal_agent`, then use the `+` button
+in the composer to upload one or more files.
+
+Example prompts:
+
+- Image: `Describe the scene and extract all visible text.`
+- TXT/Markdown: `Summarize this document into five actionable points.`
+- PDF: `Explain the main argument and list the supporting evidence.`
+- Video: `Give me a timeline of the important events in this video.`
+- Mixed files: `Compare these files and identify any contradictions.`
+- Image generation: `Create a cinematic image of Shanghai on a rainy night.`
+- Video generation: `Create a time-lapse video of sunrise over the ocean.`
+
+The Agent explicitly uses VeADK's default 2.1 Pro model. The generation tools
+fall back to `MODEL_AGENT_API_KEY`; set `MODEL_IMAGE_API_KEY` and
+`MODEL_VIDEO_API_KEY` to override them independently. Local uploads are stored
+under `/tmp/veadk-media` by default; configure TOS for durable storage. PDFs
+are automatically rendered to page images before the model call, with no extra
+dependency or Agent callback required.
diff --git a/examples/multimodal_agent/README.zh.md b/examples/multimodal_agent/README.zh.md
@@ -0,0 +1,34 @@
+# 多模态 Agent
+
+这个 Demo 使用 VeADK frontend 的多模态链路分析图片、TXT/Markdown、PDF
+和视频，并挂载 `image_generate` 与 `video_generate` 工具。frontend 负责上传
+文件，`MultimodalMediaPlugin` 将文件还原为 Google GenAI Parts，再交给
+`doubao-seed-2-1-pro-260628` 分析；生成图片或视频时则调用对应工具。
+
+## 运行
+
+在仓库根目录执行：
+
+```bash
+cp examples/multimodal_agent/.env.example .env
+uv run veadk frontend --agents-dir examples --dev
+```
+
+打开 <http://127.0.0.1:8000>，选择 `multimodal_agent`，然后通过输入框中的
+`+` 按钮上传一个或多个文件。
+
+示例问题：
+
+- 图片：`描述画面，并提取所有可见文字。`
+- TXT/Markdown：`把这份文档总结成五条可执行建议。`
+- PDF：`解释核心观点，并列出支撑证据。`
+- 视频：`按时间线总结视频中的重要事件。`
+- 混合文件：`比较这些文件，并找出相互矛盾的地方。`
+- 图片生成：`生成一张雨夜上海街头的电影感图片。`
+- 视频生成：`生成一段海边日出的延时摄影视频。`
+
+Agent 明确使用 VeADK 默认的 2.1 Pro 模型。图片与视频生成工具默认复用
+`MODEL_AGENT_API_KEY`；也可以分别设置 `MODEL_IMAGE_API_KEY` 和
+`MODEL_VIDEO_API_KEY`。本地上传默认保存在 `/tmp/veadk-media`；需要持久化时
+请配置 TOS。PDF 会在调用模型前自动渲染为页面图片，无需安装额外依赖或配置
+Agent callback。