AI tooling in littlefs?

[geky]

@Ryan-CW-Code raised an interesting question in the v3 thread (#1114 (comment)):

Have you considered leveraging AI coding tools to accelerate the development progress of V3? For instance, I've been using codex in my projects to assist with generating unit tests and fuzzing test cases. Among the many AI coding tools I've tried, codex has proven to be the most reliable.

However, I apologize that due to my current schedule constraints, I'm unable to submit PR contributions to the littlefs project at this time.

I'll copy my main thoughts and additional comments below to give this its own space for discussion. Feel free to weigh in with thoughts.

[geky]

@geky #1114 (comment)

Have you considered leveraging AI coding tools to accelerate the development progress of V3?

Not really, to be honest. I think littlefs in particular is a poor fit for AI generated code, as we have a relatively small amount of code where correctness is important, and bugs can be extremely subtle.

This goes into the tests too. Writing tests requires quite a bit of thought to create scenarios where things are likely to break, without taking forever to run. There's also a large body of work around fuzz testing which deserves exploration (#1030).

But one area where AI could be useful is all the scripts around littlefs. That's a lot of low-risk work.

But I also haven't explored AI tools enough to have a strong opinion: (1) I'm personally a bit subscription-averse, (2) I can't shake the paranoia they're going to introduce difficult-to-reason-about bugs, (3) Has the whole copyright thing been settled? Maybe this shows my ignorance/lack of paying attention, but I'd really like to not accidentally poison a codebase if they regurgitate someone else's code...

[geky]

@Ryan-CW-Code #1114 (comment)

littlefs is indeed not well-suited for direct AI code generation. However, I believe that with comprehensive documentation—covering littlefs's design philosophy, internal implementation mechanisms, external Flash constraints, and other core concepts—AI can excel at code review tasks.

My typical workflow is: first, have the AI review the code based on the documentation and propose improvement suggestions; then, after developer review and approval, decide whether to generate code based on actual needs, rather than letting AI write core logic directly. This "human-first" approach leverages AI's efficiency advantages while ensuring code quality and system security.

Taking my ongoing project [RyanJson](https://github.com/Ryan-CW-Code/RyanJson/tree/next) as an example, which employs compact bit-field operations (though the README is slightly outdated), Codex (with xhight thinking mode enabled) has impressed me compared to Gemini and Claude. Especially when handling low-level embedded code, Codex demonstrates better understanding of memory layout and bit manipulation logic.

I believe AI is documentation-driven. By having AI continuously summarize experiences and iterate optimizations during the review process, we can gradually accumulate high-quality technical documentation suitable for AI consumption. This documentation not only serves the current project but also helps other developers in the community—and even various AI models on the market—quickly understand littlefs's architecture and design philosophy, including usage patterns, common pitfalls, and important considerations, thereby creating a positive feedback loop.

I believe littlefs fully meets the application criteria for the Codex Open Source Program and recommend applying: https://openai.com/form/codex-for-oss/

[geky]

@Ryan-CW-Code #1114 (comment)

littlefs has quite a few configuration options, and ensuring that each one is thoroughly tested is an extremely challenging task. Therefore, having AI write scripts to achieve full configuration coverage testing is particularly necessary.
Additionally, I personally believe that as an infrastructure component for embedded systems, littlefs should ideally strive for fuzzing branch coverage approaching 100%, which is significant for improving the filesystem's reliability and security.

[geky]

I believe littlefs fully meets the application criteria for the Codex Open Source Program and recommend applying: https://openai.com/form/codex-for-oss/

I appreciate the idea, but (1) it looks limited (6 months, credits) in a way that may be meant to drive subscriptions, and (2) it risks turning littlefs into an advertising vehicle for OpenAI, which I'm not very comfortable with.

But this gets into where I'm pretty subscription adverse. I've not bothered to experiment with AWS for similar reasons, probably to my own disadvantage.

Local LLMs are more interesting to me for this reason, but my understanding is they're not as capable.

---

The documentation-first perspective is interesting. Documentation is the main hurdle for v3 right now, and is something I tend to procrastinate on.

It's interesting to think about. Some of the reason I prefer to just code is it's the "solution", while documentation is the "path to the solution". littlefs has involved a lot of exploratory work, where we try a solution -> find fundamental problems -> backtrack and try a different solution. If that involved sufficient documentation for someone else to arrive at a solution at each step, it sounds kind of exhausting.

But maybe that's just because I'm comfortable with the current state of things.

---

littlefs has quite a few configuration options, and ensuring that each one is thoroughly tested is an extremely challenging task. Therefore, having AI write scripts to achieve full configuration coverage testing is particularly necessary.
Additionally, I personally believe that as an infrastructure component for embedded systems, littlefs should ideally strive for fuzzing branch coverage approaching 100%, which is significant for improving the filesystem's reliability and security.

This is an interesting one. Right now, 100% line/branch coverage is explicitly a non-goal.

The reasoning is more-or-less that 100% line/branch coverage leads to a lot of tests, but not necessarily a lot of useful tests. And while things are still changing, we don't need a lot of tests, we need a focused set of useful tests to guarantee the properties we care about.

In the past couple of days, 7536/9303 LoC (81%) changes were test changes. And that's without line/branch coverage.

Eventually though, I agree, 100% line/branch would be nice to have, but it's not really the main goal.

Or for a different thought experiment, consider a perfect AI. We could ask it to write 100% line/branch coverage, and given enough time I'm sure it would end up with a perfect exhaustive set of tests that reaches the 100% number. But this would include tests that happily verify buggy functions reproduce said bugs, and we wouldn't be able to change anything because it'd break all the tests.