TCS: Two-Stage Reinforcement Learning for Sound and Adversarial Test Generation in Code LLMs

Official repository for the Findings of EMNLP 2026 paper "Two-Stage Reinforcement Learning for Sound and Adversarial Test Generation in Code LLMs".

Jiacheng Xu, Wentao Zhang, Zhiyi Lyu, Fuxiang Zhang, Chaojie Wang, Yang Liu, Bo An

Released artifacts

Artifact	Link
TCS-7B	XiaoBanni/TCS-7B
TCS-1.5B	XiaoBanni/TCS-1.5B
TACO training set	XiaoBanni/TACO-Train
LiveCodeBench evaluation set	XiaoBanni/LiveCodeBench-2408-2502

Installation

conda create -n tcs python=3.10
conda activate tcs
pip install -r requirements.txt
pip install -e .
pip install flash-attn --no-build-isolation
cp scripts/env.sh.template scripts/env.sh
# Edit scripts/env.sh to set DATA_ROOT and CKPT_ROOT for your machine.

Data

source scripts/env.sh
huggingface-cli download XiaoBanni/TACO-Train --repo-type dataset --include "*.pkl" --local-dir "$DATA_ROOT"
huggingface-cli download XiaoBanni/LiveCodeBench-2408-2502 --repo-type dataset --include "livecodebench_2408_2502*" --local-dir "$DATA_ROOT"

Training

source scripts/env.sh
# Stage 1
bash scripts/train/train.sh --train-type dynamic_test_case --model-path deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B

# Convert the Stage-1 checkpoint (step 200 for 1.5B, step 40 for 7B)
bash scripts/utils/convert_ckpt_to_hf.sh --base-path $CKPT_ROOT/DeepSeek-R1-Distill-Qwen-1.5B/<exp_name>/global_step_200

# Stage 2
bash scripts/train/train.sh --train-type adversarial_test_case --model-path $CKPT_ROOT/DeepSeek-R1-Distill-Qwen-1.5B/<exp_name>/global_step_200/huggingface

Evaluation on LiveCodeBench

bash scripts/eval/run_lcb.sh --code-model-path XiaoBanni/TCS-7B

Results are written to eval/lcb/<model>_<model>/code_output_unvalidated.txt.

Citation

@inproceedings{xu2026tcs,
  title     = {Two-Stage Reinforcement Learning for Sound and Adversarial Test Generation in Code {LLM}s},
  author    = {Jiacheng Xu and Wentao Zhang and Zhiyi Lyu and Fuxiang Zhang and Chaojie Wang and Yang Liu and Bo An},
  booktitle = {Findings of the Association for Computational Linguistics: {EMNLP} 2026},
  year      = {2026}
}

Acknowledgements

Our RL training is built on verl. We evaluate on TACO and LiveCodeBench.

License

Apache License 2.0. verl/utils/reward_score/livecodebench/lcb_runner is adapted from LiveCodeBench under the MIT License.

Contact

Questions and issues are welcome — open a GitHub issue or contact jiacheng005@e.ntu.edu.sg.