IDLE: add an AST browser

[serhiy-storchaka]

Feature or enhancement

As a companion to the proposed token browser (#152941), visualizing the abstract syntax tree helps in learning how Python parses code and in understanding the structure that the compiler works from.

I propose an AST Browser added as an extension on the Tools menu (Shell and editor). It shows ast.parse of the current window as a hierarchical, fully expanded ttk.Treeview, each node labeled with its type and inline simple fields.

Interaction:

• Selecting a node highlights the corresponding source in the editor.
• The browser follows the editor: the editor selection (or cursor) selects the single smallest enclosing node.
• Scope is the editor selection if it parses, else the Shell's current input, else the whole window (a context-dependent fragment that does not parse reports the syntax error).

It shares the window skeleton and editor-sync mechanism with the token browser, so the two are best reviewed and landed together.

A PR will follow.

Linked PRs

• gh-152942: Add an AST browser to IDLE #152947

[iritkatriel]

We've been playing around with an educational tool showing the compilation pipeline in https://github.com/iritkatriel/codoscope

I'd be interested in your thoughts how this fits in. Maybe we could share some code, at the very least.

[iritkatriel]

See https://iritkatriel.github.io/codoscope/

[serhiy-storchaka]

Thanks @iritkatriel -- codoscope looks great, and covering the whole pipeline down to bytecode is exactly what I don't attempt here.

Mine are deliberately the small end: essentially a GUI over the tokenize and ast modules, integrated with the editor and synced both ways (select a token/node -> source highlights, and vice versa). Staying on public stdlib is what lets them ship in idlelib, but it's also what limits code sharing -- no internal compiler APIs, Tkinter not Textual/web. So the real overlap is the idea, mainly the source-range <-> token/node mapping. Happy to compare notes there, and to cross-link the two: IDLE as the in-editor stdlib view, codoscope as the full-pipeline one.

A symtable browser would be an easy next one. A bytecode/dis view is bigger -- to be useful it should tie into IDLE's debugger, so that's a separate, more complex effort.

[StanFromIreland]

I'm worried that this, #152941, and #152966 (and also applies to #152940) are a little beyond IDLE's scope, and should be left to a different tool, like Codoscope linked above. No other IDE provides such tools, they aren't generally used during development, even less so by beginners. IDLE already has a long TO-DO list, I don't think we should be expanding the maintenance burden there. @terryjreedy what do you think about these proposals?