Adaptive Visual RL

Evaluation release for the CARLA 0.9.15 visual reinforcement-learning agent accompanying the paper:

Stable Visual Reinforcement Learning for Autonomous Driving With Backbone Adaptation and Guided Exploration

This repository releases the main method checkpoint, the evaluation code, the route files, and public demo GIFs. Training code, ablation scripts, and baseline implementations are not included in this evaluation package.

Highlights

• A compact visual RL agent for autonomous driving in CARLA.
• Driving-scene visual pretraining followed by DoRA-based backbone adaptation.
• Demonstration-guided exploration and channel-matched hybrid exploration noise studied in the accompanying paper.
• Dual visual inputs: semantic front view and BEV, both at 112x112 resolution.
• Leaderboard-style closed-loop evaluation with route completion, infraction penalty, and driving score metrics.

Contributions

• Backbone adaptation: the paper adapts a lightweight visual backbone to driving scenes and uses DoRA to update task-relevant visual layers efficiently.
• Guided exploration: demonstrations are used to stabilize early closed-loop exploration while the learned policy gradually takes over control.
• Hybrid action noise: exploration noise is matched to the action channels so longitudinal and lateral controls can be perturbed with different dynamics.
• Closed-loop evaluation: the released scripts evaluate the trained policy on CARLA Leaderboard-style route XMLs and report DS, RC, IP, and infractions.

Method Overview

Network Architecture

Semantic front view (112x112)
        |
        v
ConvNeXt-Tiny visual tower + DoRA adapters ----+
                                               |
BEV semantic view (112x112)                    |
        |                                      |
        v                                      |
ConvNeXt-Tiny visual tower + DoRA adapters ----+--> feature fusion --> Actor
                                               |                       |
Ego-state vector (14-D) -----------------------+                       v
                                                               [throttle/brake,
                                                                    steering]

The released checkpoint uses dual visual towers for the front semantic image and the BEV semantic image. The fused visual features are concatenated with the ego-state vector and passed to the policy head. The checkpoint also contains the critic networks required by the saved DDPG model, although public use is focused on inference and evaluation.

Visual Demos

Each GIF is a completed closed-loop route recorded with the released Ours checkpoint. The 2x2 panel shows third-person driving, semantic front-view with route overlay, BEV with route overlay, and the global route map.

Short routes:

Short 01	Short 02	Short 03	Short 04

Longest6 subset routes:

Long 01	Long 02	Long 03	Long 04

_{GIF playback is rendered at 2x normal CARLA simulation speed.}

Released Files

Adaptive_Visual_RL/
|-- assets/
|   `-- framework.png
|-- checkpoints/
|   `-- ours/
|       |-- config.json
|       `-- weights/
|-- evaluation/
|   |-- routes/
|   |-- results/
|   `-- scripts/
|-- main/
|   |-- env_carla/
|   |-- pretrained/
|   `-- modul.py
`-- results/
    `-- demo_routes/

Important files:

• Agent checkpoint: checkpoints/ours/weights/
• Checkpoint config: checkpoints/ours/config.json
• Driving-scene-pretrained backbone: main/pretrained/convnext_tiny_tavp.pth
• Route files: evaluation/routes/
• Evaluation scripts: evaluation/scripts/
• Reference per-route CSVs: evaluation/results/final/
• Public GIFs: results/demo_routes/

Requirements

• CARLA 0.9.15 server and matching PythonAPI egg.
• Python 3.7+.
• NVIDIA GPU with CUDA.
• Git LFS for downloading the released weights and large demo GIFs.

Install Python packages:

git clone https://github.com/xcc6219/Adaptive_Visual_RL.git
cd Adaptive_Visual_RL
git lfs install
git lfs pull
pip install -r requirements.txt

Install the CARLA 0.9.15 PythonAPI separately and make sure the matching carla-0.9.15-py3.7-*.egg is on PYTHONPATH. See the CARLA 0.9.15 quickstart: https://carla.readthedocs.io/en/0.9.15/start_quickstart/

Run Evaluation

Start CARLA 0.9.15 first. Example:

./CarlaUE4.sh -quality-level=Low -RenderOffScreen -carla-rpc-port=2000

Then run the released Ours checkpoint. The scripts change the working directory to main/ internally, so keep the ../ prefixes below when launching from the repository root.

python evaluation/scripts/eval_episode.py \
    --algo ddpg \
    --checkpoint ../checkpoints/ours/weights \
    --routes ../evaluation/routes/longest6/longest6_noT6.xml \
    --out ../results/ours_longest6.csv \
    --carla_host 127.0.0.1 \
    --carla_port 2000 \
    --max_time_episode 20000 \
    --stall_frames_eval 500 \
    --number_of_vehicles 100

Aggregate per-route CSVs into leaderboard-style metrics:

python evaluation/scripts/compute_leaderboard.py \
    --results_root evaluation/results \
    --benchmark longest6

License

Code is released under the MIT License. The released weights are provided for research reproduction of the evaluation results.

Citation

@article{xu2026adaptivevisualrl,
  title={Stable Visual Reinforcement Learning for Autonomous Driving With Backbone Adaptation and Guided Exploration},
  author={Xu, Chengcheng and Zhao, Haiyan and Zhou, Fangzhe and Lu, Xinghao and Gao, Bingzhao and Chen, Hong},
  journal={Journal or Conference Paper},
  year={2026}
}