Skip to content

Phase 2 blocked: live inline [image-N] on clipboard paste needs pi API support #1

Description

@rz1989s

Context

Phase 1.5 (vision-delegate) ships working image understanding via a scoped vision sub-agent — primary model never changes, [image-N] markers generated for the sub-agent, @<image-path> works interactively, multi-image mapping verified.

Phase 2 goal: when the user pastes an image (Ctrl+V), show a live inline [image-N] token in the editor at the cursor (visual feedback + positional anchor), instead of today's behavior where a pasted image shows nothing inline (the editor is text-only; images are app-layer attachments).

Why it's blocked on pi API support

Investigated the editor extension API:

  • EditorComponent is text-onlygetText/setText/insertTextAtCursor/getExpandedText, no image/attachment methods. Images are attached at the app layer, not in editor text.
  • CustomEditor.onPasteImage is set by the app — interactive-mode assigns it to handleClipboardImagePaste. A custom editor can't cleanly intercept it (constructor assignment gets overwritten). Registering onAction("app.clipboard.pasteImage", …) would take over paste-image and break the app's image attachment (the editor has no way to attach images itself).
  • readClipboardImage() is not a public export (dist/utils/clipboard-image.ts exists but isn't in index.d.ts) — deep import is fragile across pi updates.
  • No hook to "insert a marker when an image is pasted while the app still attaches it."

Pi API requests (upstream — earendil-works/pi-mono)

  1. Public export readClipboardImage (and the ClipboardImage type) from @earendil-works/pi-coding-agent.
  2. An editor hook that fires when an image is pasted, before/alongside the app's attachment, letting an extension insert a text marker (e.g. [image-N]) at the cursor without disabling the app's image attachment. Candidate shapes:
    • onPasteImage?(image: ClipboardImage): string | void — return a marker string to insert at the cursor; app still attaches the image.
    • or a paste_image extension event with the image + cursor position.
  3. (Nice-to-have) a way for a custom editor to render attached images inline as labeled tokens.

What works today (Phase 1.5)

  • @<image-path> in the editor → [image-N] marker generated for the sub-agent + image attached (parsed in before_agent_start). User sees the @path text (not [image-N]), but the mapping is exact.
  • Clipboard paste (Ctrl+V) → image attaches (event.images) → delegation works; just no inline [image-N] visual.

Acceptance criteria (Phase 2, unblocked)

  • Pi exports readClipboardImage publicly
  • Pi provides a paste-image hook that allows marker insertion without breaking attachment
  • armory custom editor shows [image-N] inline on Ctrl+V, colored, with the image still attached and delegated

Escalation

File upstream issues on earendil-works/pi-mono for items 1 & 2. Until then, Phase 2 is deferred; Phase 1.5 is the shipped milestone.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions