Move away from ruamel.yaml and pyyaml somehow

[virtuald]

ruamel.yaml is used by semiwrap update-yaml command specifically because it can read a YAML file and write it while retaining comments. Unfortunately, it's not pure python, and the creator decided to no longer publish wheels for the C portion starting with release 0.19. This is annoying for a bunch of reasons, so I'd like to move away from it.

Possibilities:

• Switch to some other configuration language
  • I tried TOML and hated it
• Find a pure python YAML library that can round-trip comments
  • strictyaml claims to do this, but it has its own weird quirks. Would need to look harder at it sometime
• Alternatively, maybe we could just vibe-code a pure python implementation of the C library using the existing test suite as a starting point? (prior art: https://simonwillison.net/2025/Dec/15/porting-justhtml/)
  • ruamel.yaml has a pure python implementation, forking that so that it's used by default might be a way to go?

I've pinned to 0.18 for now, but once Python 3.15 comes out then we'll need to have figured something out.

[AvdN]

ruamel.yaml can be used in pure python mode. Specifically the functionality retaining comment is not using ruamel.yaml.clibz (or has been using ruamel.yaml.clib) at the moment.

Installing with ruamel.yaml with --no-deps should work for you, this works for all YAML(typ=X) instantiations ( the default "rt", "safe", etc.).

Although I intend to add speed ups to the default, similar that are provided for the safe loader with the C extensions, I do not have plans to drop the pure Python implementation of that.

[flying-sheep]

@AvdN --no-deps is a non-starter for almost every use case. People specify dependencies in an array and let software manage them, they don't install them by hand.

Instead, you could make the speedup package an extra / optional dependency, then people could specify ruamel-yaml[clibz] or so if they want it. It could still be the default and just raise a custom ImportError instructing people how to specify the extra or switch the mode if the package is missing.

That would also break the CI of all default mode users, but would give users choice on how to fix it: safe mode, clib, or clibz.

And in the future, once the default extras PEP is accepted, you can make it the default. Then people can still opt out but get the speedup by default.

[virtuald]

@AvdN thanks for the comment, I hadn't looked carefully at its internals (yet) so I hadn't realized there was a pure python option. As @flying-sheep points out, --no-deps is absolutely not doable for anyone except a primary user of ruamel.yaml.

Consider: if I ask a user to do pip install semiwrap, how do they specify --no-deps for only ruamel.yaml. They can't. If they want to opt out, they have to figure out what all of semiwrap's dependencies are and install all of those manually, in addition to ruamel.yaml. That's just awful for everyone. I'd rather just use a different library.

My preference would be what @flying-sheep said, if you're not going to distribute wheels for the optimized version (which... I get it, making 50+ wheels sucks but that's how python packaging works) that the optimized version is 'opt-in' instead by installing ruamel-yaml[clibz] or whatever. I (and I suspect most users of YAML libraries) don't care about performance, the files are small. I just want it to work.

PS: Thanks for ruamel.yaml. The round-trip support is fantastic and has saved me a ton of time for a variety of projects.

[AvdN]

I have uploaded ruamel.yaml==0.19.1 and it no longer has default requirements (neither the old ruamel.yaml.clib nor the new ruamel.yaml.clibz). There two additional optional dependencies: ruamel.yaml[oldlibyaml] getting the old lib and ruamel.yaml[libyaml] getting you the new one (i.e. installing ruamel.yaml.clibz, which now also forces the install of the latest setuptools, which caused problems for many).

If you use ruamel.yaml in default (roundtrip) mode, which I guess from your original post you do (because you preserve comments), only install ruamel.yaml and you will be pure-python. Roundtrip mode does not use the C extension at the moment.

If you load files with YAML documents using YAML(typ='safe') there will be a performance hit, but I would only worry about that with for files with at least a few hundred lines. If you need the performance right now, install using ruamel.yaml[oldlibyaml]

I was not aware of the difficulty some might have to insert --nodeps in their setup/environment. I long hold back from removing the requirement (and of course for the longest time ruamel.yaml versions came bundled with the binaries, requiriing hours long builds of wheels, even if just some Python code or the README changed), as I was sure someone was going to complain about performance loss (not having read the README). This even though the roundtrip mode doesn't use the C based tokenizer (as it drop the comments). Oh well it has been a day.

I hope the above solves this issue for you. On the plus side you will not be installing a binary library that you were not using anymore.

[virtuald]

Thanks! I'll take a look this weekend.

[flying-sheep]

Amazing, thank you!

which... I get it, making 50+ wheels sucks but that's how python packaging works

That's actually not true! With ABI3 wheels, the wheels are not bound to one specific python version. This means that you need one per CPU architecture, and that wheel will work for all python versions at or above the minimum one (current and future)