Skip to content

feat(a11y): Implement AccessKit accessibility support for all UI components#2407

Open
huacnlee wants to merge 2 commits into
mainfrom
accessbility
Open

feat(a11y): Implement AccessKit accessibility support for all UI components#2407
huacnlee wants to merge 2 commits into
mainfrom
accessbility

Conversation

@huacnlee

Copy link
Copy Markdown
Member

Summary

  • Adds Role, aria_* attributes, and on_a11y_action handlers to all major UI components using the AccessKit integration in the latest GPUI
  • Covers interactive controls, navigation, overlays, data display, containers, and form components (~25 component files)
  • Adds Root component Role::Application landmark so the accessibility tree has a proper application root below Window
  • Adds script/run-story-macos helper to build and launch Story gallery as a signed macOS .app bundle for VoiceOver testing

Component role mapping

Component Role Extra attributes
Button Button / Link aria_label, aria_selected
Toggle / ToggleGroup Button + aria_toggled, Toolbar
Checkbox CheckBox aria_toggled, aria_label
Radio / RadioGroup RadioButton, RadioGroup aria_selected, position_in_set, size_of_set
Input TextInput / MultilineTextInput
NumberInput SpinButton aria_numeric_value
Slider Slider aria_numeric_value, min/max, aria_orientation, Increment/Decrement a11y actions
Tab / TabBar Tab + aria_selected, TabList
PopupMenu / AppMenuBar Menu, MenuBar
MenuItem MenuItem aria_selected
Dialog / AlertDialog Dialog (configurable), AlertDialog
Alert Alert
Progress ProgressIndicator aria_numeric_value, min/max
List / ListItem List, ListItem aria_position_in_set, aria_size_of_set, aria_selected
Table / Header / Body / Row / Cell Table, RowGroup, Row, ColumnHeader, Cell aria_row_index, aria_column_index
Accordion / item header Group, Button + aria_expanded
GroupBox Group
Combobox ComboBox aria_expanded
DropdownButton Group
Stepper / StepperItem Group, ListItem aria_position_in_set
BreadcrumbItem Link / ListItem
FormField Group
Root Application

Verification guide

Prerequisites

AccessKit only builds the full accessibility tree when an assistive technology is active. Accessibility Inspector alone is not sufficient — you must enable VoiceOver first.

macOS — VoiceOver + Accessibility Inspector

# 1. Build and launch Story gallery as a signed .app bundle
./script/run-story-macos
# 2. Enable VoiceOver  (this activates AccessKit)
Cmd + F5

# 3. Open Accessibility Inspector
#    Xcode → Xcode menu → Open Developer Tool → Accessibility Inspector
#    (or: open -a "Accessibility Inspector")

# 4. In Accessibility Inspector, click the target crosshair icon,
#    then hover over or click elements in the Story window.
#    You should see the full hierarchy:
#    Application > Window > Application > Button / CheckBox / Tab / ...

# 5. Tab through interactive elements — VoiceOver announces each one.

# 6. Test Slider keyboard control:
#    Focus the Slider, then press Up/Down arrows (Increment/Decrement a11y actions).

Key things to verify

  • Button: role AXButton, label matches button text
  • Checkbox: role AXCheckBox, value changes on click
  • Toggle: role AXButton, AXValue reflects checked state
  • Radio group: each item reports AXPositionInSet / AXSizeOfSet
  • Tab bar: AXTabGroup containing AXTab items, selected tab has AXValue = 1
  • Dialog: role AXDialog when open, focus trapped inside
  • Slider: AXSlider with AXMinValue / AXMaxValue / AXValue, responds to VoiceOver Increment/Decrement

Windows — NVDA

cargo run
# Enable NVDA, Tab through the window.
# NVDA should announce component type and label on focus.

Linux — AT-SPI (Orca)

cargo run
# Enable Orca, Tab through the window.

Note: active_flag starts false and becomes true only on first AT query.
If Inspector shows only Window with no children, ensure VoiceOver is on,
then wait ~1 second and click Refresh in Accessibility Inspector.

🤖 Generated with Claude Code

huacnlee and others added 2 commits May 28, 2026 19:08
…onents

Adds ARIA roles and accessibility attributes to all major components,
leveraging the AccessKit integration shipped in the latest GPUI update.
Each component now exposes semantic role, state, and label information
to assistive technologies (VoiceOver, NVDA, AT-SPI).

Component coverage:
- Button: Role::Button / Role::Link, aria_label, aria_selected
- Toggle / ToggleGroup: Role::Button + aria_toggled, Role::Toolbar
- Checkbox: Role::CheckBox + aria_toggled + aria_label
- Radio / RadioGroup: Role::RadioButton + aria_selected + position/size_of_set, Role::RadioGroup
- Input: Role::TextInput / Role::MultilineTextInput
- NumberInput: Role::SpinButton + aria_numeric_value
- Slider: Role::Slider + aria_numeric_value + min/max + Increment/Decrement actions
- Tab / TabBar: Role::Tab + aria_selected, Role::TabList
- PopupMenu / AppMenuBar: Role::Menu, Role::MenuBar
- MenuItem: Role::MenuItem + aria_selected
- Dialog: Role::Dialog (configurable); AlertDialog: Role::AlertDialog
- Alert: Role::Alert
- Progress: Role::ProgressIndicator + aria_numeric_value + min/max
- List / ListItem: Role::List + Role::ListItem + position/size_of_set + aria_selected
- Table / TableHeader / TableBody / TableRow / TableHead / TableCell:
  Role::Table, Role::RowGroup, Role::Row + aria_row_index,
  Role::ColumnHeader / Role::Cell + aria_column_index
- Accordion: Role::Group; AccordionItem header: Role::Button + aria_expanded
- GroupBox: Role::Group
- Combobox: Role::ComboBox + aria_expanded
- DropdownButton: Role::Group
- Stepper / StepperItem: Role::Group, Role::ListItem + aria_position_in_set
- BreadcrumbItem: Role::Link / Role::ListItem
- FormField: Role::Group
- Root: Role::Application (window-level application landmark)

Also adds:
- SliderState::min_value() / max_value() / step_value() accessors
- Dialog::a11y_role field for role override
- Radio::position_in_set / size_of_set fields populated by RadioGroup
- script/run-story-macos: launch Story as a signed macOS .app bundle for VoiceOver testing

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@grishy

grishy commented Jun 1, 2026

Copy link
Copy Markdown

Great thx for your work!
One thing I think before merging: Button better to expose an explicit aria_label(...) override, especially for icon-only buttons.

like:

Button::new("close")
    .icon(IconName::X)
    .aria_label("Close")

@huacnlee

huacnlee commented Jun 2, 2026

Copy link
Copy Markdown
Member Author

Still not working, I don't know the reason. macOS Accessibility inspector can't select any elements.

@grishy

grishy commented Jun 2, 2026

Copy link
Copy Markdown

I think this is because it is not perfect in GPUI itself. I wrote a small Swift wrapper for tests myself.

It needs two actions:

  1. First ask AX on the app. It will switch GPUI into Access ON mode, return only the root object, and call update on the whole tree.
  2. Second ask AX on the app again. Now we have the proper AX tree of the app.

So, on the first try, it only activates the AX tree (GPUI here is lazy), but it returns the wrong tree initially. It should wait until all components are updated and only after that return to AX's caller.

If it will work fine for me, I will create a PR here 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants