cmd2 4.0 discussion

[tleonhardt]

Now that cmd2 3.0.0 is out, it's time to start discussing what we would like to see in the 4.0.0 release for cmd2.

Features @kmvanbrunt and I have discussed for 4.x include:

• Remove explicit dependency on Python's built-in cmd module, but retain backwards compatibility (PR Remove dependency on Python's built-in cmd module #1539)
• Replace dependency on twine for publishing PyPI package with one on uv publish (Use uv for publishing to PyPI instead of twine #1548)
• Replace use of invoke with use of make, primarily used for cleanup at this point (Add clean target to Makefile that runs what is in invoke clean #1549)
• Migrate documentation to Zensical which is essentially MkDocs 2.0, but implemented in Rust and much faster (PR Migrate documentation to use Zensical instead of MkDocs #1557)
• Remove dependency on Python's built-in readline module and replace it with a dependency on prompt-toolkit which is a pure Python replacement for GNU Readline (PR Migrate from using readline to prompt-toolkit #1553)
  • Support adding a bottom toolbar to display real-time status via prompt-toolkit (PR Migrate from using readline to prompt-toolkit #1553)
  • Add prompt-toolkit syntax highlighting of text entered by users, at least for command names (PR Migrate from using readline to prompt-toolkit #1553)
  • Add support for prompt-toolkit Input Validation - I looked into this and honestly if we add support for more advanced input validation, I would prefer to add a dependency on pydantic and go that route. But that is more than I want to bite off for the 4.0 release of cmd2.
  • Add auto-suggestions using prompt-toolkit similar to fish-shell where it will show a recommended completion grayed-out per user request in Add auto suggestion feature #1545 (PR Migrate from using readline to prompt-toolkit #1553)
• Handle prompt-toolkit completion edge cases related to adding quotes (PR Refactor completion API #1575)
• Consider implementing syntax that is essentially compatible with Typer for defining command arguments based on type hints. A user documented how to do this in Discussion A plugin for Typer command define within cmd2 #1530.
  • For this I envision something like a @cmd2.typer decorator that will auto-generate an argparse.ArgumentParser based on argument annotations
  • We won't be tackling this in the 4.0.0 release unless someone else wants to do the work
• Remove the transcript testing feature set (PR Remove the Transcript Testing feature #1583)
• Investigate migrating type-checking from mypy to ty now that ty is in beta
  • Will revisit this in the future after releasing 4.0.0. This isn't a backwards incompatible change for anyone and I'd like to wait for this library to mature a little more before migrating to it
• Replace restrictive type hints (e.g. list) with broader ones like Sequence or Iterable (PR Use more generic types where possible for input parameters #1584)
• Ensure examples all work
• Change async alert methods to rely on a queue (PR Refactored async alerts. #1586)
• Remove the default_to_shell flag since it can result in unintended execution which can be dangerous (PR Removed default_to_shell. #1590)
• Restore concatenating all lines of a multi-line command into one for the up-arrow history (PR Switch to using prompt-toolkit's multiline filtering. #1589)
• Fix Cmd2Lexer coloring of command names used on continued lines in multiline commands (PR Fixed multiline command highlighting in Cmd2Lexer #1591)
• Ensure documentation and CHANGELOG are fully up to date

Moving to prompt-toolkit and no longer depending on cmd will be a large paradigm shift for cmd2. But this will open doors for easier and more consistent cross-platform support as well as being able to provide some more advanced features that we cannot now.

[tleonhardt]

PR #1539 handles removing the explicit dependency on Python's built-in cmd module

[tleonhardt]

I plan to educate myself on Zensical over the holidays.

I think @kmvanbrunt has already started looking into prompt-toolkit.

@theagilehacker @joqual If you would like to help with anything here, we'd love some more developers to get involved. Also if you have ideas for what you would like to see in cmd2, please suggest them. Most of the planned work requires replacing readline with prompt-toolkit first. But the Typer stuff is fully independent of that. I suspect Kevin wouldn't mind help with prompt-tookit work, so familiarizing yourself with that also would be helpful.

[tleonhardt]

I attempted to port the documentation build to use Zensical which is effectively the next generation of MkDocs. To do this I needed to eliminate use of the markdown-include plugin and replace it with usage of the pymdownx.snippets extension. That is all working fine.

Building with Zensical almost worked. The one one limitation I ran into is that currently the mkdocstrings extension for Zensical does not support cross-references yet:

The mkdocstrings integration is preliminary, which means some features are not yet supported, specifically cross-references and backlinks. We're working on bringing these features into Zensical.

mkdocstrings support was added to Zensical first on 2025-12-03, so this is under very rapid development and I expect within a month or so support for mkdocstrings cross-references will be available. Once that is there, everything should "just work". The documentation builds fast and looks even better than it did with MkDocs and the Material Theme.

[tleonhardt]

I created an issue in the Zensical project about support for mkdocstrings cross-reference support: zensical/zensical#237

Once that is available, I will re-visit porting the documentation to Zensical.

[tleonhardt]

I've started experimenting with migrating to prompt-toolkit from readline on the prompt-toolkit branch.

Things are getting close to working in a very basic rudimentary state. rl_utils.py has been deleted and all use of readline has been replaced with use of prompt-toollkit. All tests have been updated and are passing. A new file pt_utils.py was added as well as a new unit test file with tests to completely cover the code in the new file.

There are inevitably other things that need to be addressed related to asyncio functionality, signal handlers, and blocking popen calls. We need to do a lot of manual testing to make sure everything appears to work as intended

[tleonhardt]

prompt-toolkit support is working and a PR is up to review for it: #1553

The documentation has been migrated to zensical in a separate PR which is also ready: #1557

[tleonhardt]

Documentation has been ported to Zensical and is looking fine on ReadTheDocs (RTD) as well as locally. Differential build is much faster as is the live preview reloading in the browser.

[tleonhardt]

Removed the Transcript Testing feature set in PR #1583