You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
desktop::uia live test rejects visually-hidden 1x1 elements far off-screen (fails locally on a web page) #52
desktop::uia::tests::live::the_focused_window_can_be_walked fails on a Windows desktop whose focused window is a web page with a screen-reader-only element:
panicked at src/desktop/uia.rs:1178:17
element Element { id: 153, depth: 12, role: Text,
name: "To view keyboard shortcuts, press question mark View keyboard shortcuts",
rect: Rect { x: 0, y: -16079, width: 1, height: 1 }, .. }
is nowhere near the desktop Rect { x: 0, y: 0, width: 1920, height: 1080 }
It fails 3 times out of 3 with that window focused. CI passes, because its runners have no such window.
Why
Pages hide text from sight but keep it for screen readers by drawing it 1×1 far off-screen (the usual "visually hidden" CSS).
UI Automation reports those rectangles faithfully.
The assertion at uia.rs:1178-1184 expects every element within one desktop's size of the desktop (meant to catch garbage coordinates, uia.rs:1174-1177). A real, valid element breaks that.
Two questions to decide
The test. Should the bounds check skip 1×1 (or zero-area-ish) elements, or only apply to elements large enough to be clicked? The intent, catching garbage pixels, holds either way.
The walk. Should the walk itself drop elements that are entirely off every screen? Today it keeps them (controls past the edge of a small screen are kept deliberately). A 1×1 element 16 000 px away cannot be clicked, but its text may still be useful context for the model.
The test half is fixed in #55 (merged). The live walk test now checks that coordinates are screen pixels at all (within 2^24 px), not how close they are to the screen, so a visually-hidden 1×1 element or a long page's below-the-fold content no longer fails it. A unit test pins the element from this issue.
Still open here: question 2. Should the walk itself drop elements that are entirely off every screen? Their text may still help the model, but they can't be clicked.
Solved in #55. The live test now accepts real off-screen elements (like this page's hidden 1×1 text) and still rejects corrupt coordinates. A unit test covers this exact element; CI is green on all three OSes. The remaining design question moved to #56.
Activity
Merge pull request #55 from incordai/fix/uia-live-offscreen
blockmandev commentedon Oct 22, 2026
The test half is fixed in #55 (merged). The live walk test now checks that coordinates are screen pixels at all (within 2^24 px), not how close they are to the screen, so a visually-hidden 1×1 element or a long page's below-the-fold content no longer fails it. A unit test pins the element from this issue.
Still open here: question 2. Should the walk itself drop elements that are entirely off every screen? Their text may still help the model, but they can't be clicked.
blockmandev commentedon Oct 23, 2026
Solved in #55. The live test now accepts real off-screen elements (like this page's hidden 1×1 text) and still rejects corrupt coordinates. A unit test covers this exact element; CI is green on all three OSes. The remaining design question moved to #56.