feat: add scaffolding rollout workflow#1064
Open
WeiHaocheng wants to merge 20 commits intoinclusionAI:mainfrom
Open
feat: add scaffolding rollout workflow#1064WeiHaocheng wants to merge 20 commits intoinclusionAI:mainfrom
WeiHaocheng wants to merge 20 commits intoinclusionAI:mainfrom
Conversation
Contributor
|
Warning Gemini encountered an error creating the summary. You can try again by commenting |
…fixes - Fix data race in ScaffoldingLlm by cloning controller synchronously before async handoff - Fix sampling_params propagation in GSM8KScaffoldingWorkflow (delegate to parent build_scaffolding_llm) - Simplify controllers.py by removing unused code paths - Add chat_scaffolding example with YAML config - Add 2-node GSM8K RLVR scaffolding config - Simplify MathVerifyWorker (remove signal-based timeout) - Increase Ray scheduler startup_timeout to 600s for large clusters Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Revert controllers.py to scaffolding_pr state (no changes needed) - Revert workflow.py arun_episode to original approach (set task_data on trajectory_maker before generate_async) - Keep synchronous clone in scaffolding_llm.py but remove unused **kwargs Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…rams Without this, NativeGenerationController() is created with empty sampling_params, so max_tokens/temperature/stop are never set on tasks. SGLang defaults to ~16 tokens, producing near-zero rewards. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Keep explicit controller/trajectory_maker construction but pass max_tokens and temperature from gconfig to NativeGenerationController. Without sampling_params, SGLang defaults to ~16 tokens output. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
feat: improve scaffolding workflow with multi-worker support and bug fixes
garrett4wade
reviewed
Apr 7, 2026
Collaborator
garrett4wade
left a comment
garrett4wade
left a comment
There was a problem hiding this comment.
LGTM in general. Since the scaffolding module is relatively mature and not necessarily related to AReaL's core modules, I recommend to move all the new files into the examples directory. It'll be a complete and standalone example.
Comment on lines
+177
to
+178
| if self.worker is None: | ||
| self._lazy_init_scaffolding(engine) |
Collaborator
There was a problem hiding this comment.
A new rollout workflow object is created for each trajectory. It will also create a ScaffoldingLlm object. Will this be very expensive?
Author
Move code to the example. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Related Issue
Fixes #(issue)
Type of Change
Checklist
pre-commit run --all-files)./docs/build_all.sh)main/review-prcommand/create-prBreaking Change Details (if applicable):
Additional Context
Need help? Check the Contributing Guide or ask in
GitHub Discussions!