gh-152942: Add an AST browser to IDLE#152947
Conversation
Documentation build overview
|
terryjreedy
left a comment
terryjreedy
left a comment
There was a problem hiding this comment.
Using the extension mechanism is handy for developing mulltiple features whose patches would confIict, but I really do not want to merge this way. I added comments about integrating features into normal code. (Not sure if complete.)
| # A browser for the Python tokens of the editor, opened from the Tools menu. | ||
| [TokenBrowser] | ||
| enable= 1 | ||
| [TokenBrowser_cfgBindings] |
There was a problem hiding this comment.
These extension def entries do nothing.
|
|
||
| extfiles = { # Map built-in config-extension section names to file names. | ||
| 'TokenBrowser': 'tokenbrowser', | ||
| 'ASTBrowser': 'astbrowser', |
There was a problem hiding this comment.
Please no. Just import files in code in the appropriate place. (See next comment.)
| ('tools', [ | ||
| ]), |
There was a problem hiding this comment.
Menu entries defining tool menu entries and virtual events should be added here. Each browser patch will have have to be finalized by basing it on the code as modified by the previous browser patch. Currently, corresponding objects are imported in editor.Editor and bound to events in init. (Yes, refactoring is needed.)
| ========================= | ||
|
|
||
|
|
||
| gh-152942: Add an AST Browser to IDLE, opened from the Tools menu. It |
There was a problem hiding this comment.
News3 items have to be added just before merging. I believe that this will be a merge conflict as soon as something else is added.
|
When you're done making the requested changes, leave the comment: And if you don't make the requested changes, you will be put in the comfy chair! |
Add a Token Browser command to a new Browse menu (Shell and editor). It opens a window listing the Python tokens of the editor content, or of the selection if there is one, with the token type names colored as by "python -m tokenize". There is one browser per editor; invoking the command again refreshes it and selects the token at the cursor. Selecting rows highlights the matching regions in the editor and, while the browser has focus, moves the editor cursor there; selecting text or moving the cursor in the editor selects the matching rows. Double-clicking a row (or pressing Escape) hides the browser, revealing the editor at the token. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add an AST Browser command to the Browse menu (Shell and editor). It opens a window showing the abstract syntax tree of the editor content (or, in the Shell, the current input), or of the selection if there is one. Selecting a node highlights the matching region in the editor and, while the browser has focus, moves the editor cursor there; selecting text or moving the cursor in the editor selects the innermost enclosing node. Double-clicking a node (or pressing Escape) hides the browser, revealing the editor at the node. It shares the window skeleton and editor-sync mechanism with the token browser. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
serhiy-storchaka
force-pushed
the
idle-astbrowser
branch
from
d842aca to
eacd68e
Compare
|
Done — reworked all four from extensions to integrated features, like the Module and Path browsers. The three code browsers go on a new Browse menu (your name), stacked token → AST → disassembly. The Character Browser stays on Edit for now so its separate PR doesn't conflict; it can move later. I kept the News3 entries — resolving the conflict or adding them just before merge is the same edit either way. Folding the existing Path/Module browsers into Browse is a good follow-up. |
| first, scope = "1.0", "text" | ||
| self.base = tuple(int(i) for i in first.split(".")) | ||
| source = text.get(first, last) | ||
| self.source_lines = source.splitlines() |
There was a problem hiding this comment.
str.splitlines() breaks on several characters that the parser and the Text widget do not treat as line boundaries: form feed \f, vertical tab \v, \x1c-\x1e, \x85, and \u2028/\u2029. When the source contains one of them, source_lines gains an extra entry, and editor_index (which indexes source_lines[lineno - 1] to turn a byte column into a character column) reads the wrong line. Form feed is the realistic trigger, since it is a legal page separator in Python source.
>>> src = "x = 1\f\ny = 22\n" # a form feed ends the first line
>>> src.splitlines() # three entries for two real lines
['x = 1', '', 'y = 22']
>>> ast.parse(src).body[1].lineno # the y = 22 statement
2y = 22 is line 2 to the parser, but source_lines[1] is now '', so editor_index(2, 4) and editor_index(2, 6) both collapse to column 0 and node_range returns a zero-width ('2.0', '2.0'). show_highlight skips zero-width ranges (rng[0] != rng[1]), so that node highlights nothing; and because the extra source_lines entry shifts every line after the form feed, later nodes have their byte columns converted against the wrong line and highlight or navigate to the wrong place. Restricting the split to the parser's line endings keeps source_lines aligned.
There was a problem hiding this comment.
Good catch, thank you. Fixed by splitting on "\n" only, in both the AST and disassembly browsers, each with a form-feed regression test. (A lone \r is a separate parser-vs-Text case I'll handle later.)
str.splitlines() also breaks at form feed and similar characters that the parser and the Text widget keep within the line, which misaligned the column mapping for the following lines. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
3 participants
Add an AST Browser extension on the Tools menu (Shell and editor), a companion to the token browser (#152941). It shows
ast.parseof the current window as a hierarchical, fully expandedttk.Treeview, each node labeled with its type and inline simple fields.Selecting a node highlights the corresponding source in the editor; the editor selection (or cursor) selects the single smallest enclosing node. Scope is the editor selection if it parses, else the Shell's current input, else the whole window.
This PR is stacked on #152941 — it shares the window skeleton and editor-sync mechanism with the token browser, so it includes that commit and should be reviewed and merged after it.
🤖 Generated with Claude Code