Skip to content

fix: route X11 keyboard input through xdotool XTEST (#58)#59

Merged
avifenesh merged 1 commit into
mainfrom
fix/x11-xtest-keyboard
Jul 25, 2026
Merged

fix: route X11 keyboard input through xdotool XTEST (#58)#59
avifenesh merged 1 commit into
mainfrom
fix/x11-xtest-keyboard

Conversation

@avifenesh

Copy link
Copy Markdown
Collaborator

Summary

Fixes #58. Thanks @martin152 — the raw-ydotool key vs tool comparison in the report isolated this precisely.

Root cause. press_key and type_text both went through ydotool on X11. ydotool writes raw evdev scancodes to a virtual uinput device; X11 then re-interprets those scancodes through the active XKB layout. So:

  • press_key "Return" / ctrl+a / ctrl+s → stray glyphs, no real key event
  • type_text → symbols and digits mangled (_%, 1+), letters survive

This is why raw ydotool key 28:1 28:0 worked in the reporter's test but the tool did not: the numeric-keycode path is fine at the uinput layer, but the scancode → layout translation is what breaks under X11.

Fix. On non-Wayland sessions with DISPLAY set and xdotool present, both tools use xdotool (XTEST), which resolves keysyms against the live layout — what X11 clients actually expect.

  • press_keyxdotool key --clearmodifiers <spec> (e.g. ctrl+a, Return, super+Return)
  • type_textxdotool type --clearmodifiers -- <text>
  • The xdotool spec is validated through the existing key_chord parser, so both backends accept exactly the same key surface — no silent widening/narrowing, and the "never silently dropped" error is preserved.
  • ydotool remains the fallback when xdotool is missing or the call fails.
  • Wayland is untouched (portal / KDE clipboard / ydotool ordering unchanged).
  • doctor now reports an xdotool input backend on X11.

Escape hatches:

COMPUTER_USE_LINUX_FORCE_YDOTOOL_KEYBOARD=1    # opt out, old behaviour
COMPUTER_USE_LINUX_FORCE_XDOTOOL_KEYBOARD=1    # force XTEST

Test plan

  • Unit tests: named-key → keysym mapping, chord modifier prefixes, bare modifier, and a grammar-parity test asserting xdotool_key_spec accepts exactly what key_chord accepts
  • cargo fmt / cargo clippy -D warnings / full cargo test green
  • Verified end-to-end on a real X server (Xvfb :97 + xterm, driven through the MCP server over stdio):
    • type_text "RAW_LINE1 _ 1 __ 11" → landed byte-exact, no %/+ mangling
    • press_key "Return" → produced a real newline (confirmed by reading what the terminal received, not by AT-SPI)
    • both responses reported Action sent through xdotool (X11 XTEST).

Note for @martin152: the xdotool package is now what makes X11 keyboard input correct — it's a soft dependency (ydotool still used as fallback), and doctor will list it once installed.

Closes #58

@gemini-code-assist

Copy link
Copy Markdown

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

press_key and type_text used ydotool on X11, which injects raw evdev
scancodes into a virtual uinput device. X11 then re-interprets those
scancodes through the active XKB layout, so named keys and chords
(Return, ctrl+a, ctrl+s) arrived as stray characters and literal text
mangled symbols and digits (_ -> %, 1 -> +) even on a plain US layout.

Prefer xdotool (XTEST) on non-Wayland sessions with DISPLAY set and the
binary present; XTEST resolves keysyms against the live layout. The
xdotool key grammar is validated through the existing key_chord parser
so both backends accept exactly the same keys. ydotool remains the
fallback when xdotool is absent or fails, and Wayland is untouched.

doctor reports an xdotool input backend on X11.
COMPUTER_USE_LINUX_FORCE_YDOTOOL_KEYBOARD=1 opts out;
COMPUTER_USE_LINUX_FORCE_XDOTOOL_KEYBOARD=1 forces it on.

Verified on a real Xvfb display driving an xterm through the MCP server:
type_text "RAW_LINE1 _ 1 __ 11" landed byte-exact and press_key Return
produced a real newline.
@avifenesh
avifenesh force-pushed the fix/x11-xtest-keyboard branch from a518ebf to 4329489 Compare July 25, 2026 01:09
@avifenesh
avifenesh merged commit 0d55e94 into main Jul 25, 2026
20 checks passed
@avifenesh
avifenesh deleted the fix/x11-xtest-keyboard branch July 25, 2026 01:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

press_key emits stray characters instead of key events on X11 (named keys and modifier chords broken)

1 participant