Pre-existing datasets in workspace are deleted or moved during node execution

[floo-dck]

Short Summary

When running TreeSearch, static assets like datasets (e.g., MovieLens) placed inside the workspace/ directory are permanently deleted or moved after the first node execution.

The Problem

The current logic assumes all files found in the workspace after a run were newly generated by the LLM code:

1. It collects everything (including pre-existing data folders).
2. If keep_only_relevant_files is active, it wipes out everything that isn't an image (.png, .jpg).
3. This deletes the dataset and crashes all subsequent experiment iterations with a FileNotFoundError.

[eisenbahnhero]

Exactly, AutoRecLab archives or deletes all files from the current workspace once a node is complete. This has been implemented deliberately.

Currently, there are no plans for subsequent nodes to reuse files already generated by the LLM. In my view, this would also be difficult to implement, as the next node could, in theory, be fundamentally different from the first node. In any case, in the current scenario, the loading of datasets and the training of models is automatically handled by omnirec.