[Regresion]: multi-cards tuning+eval causes exceptions at eval phase

[wenhuach21]

Problem Description

~~bash
"/models/opt-125m" --output_dir "/data2/wenhuach/haha" --scheme W2A16G64 --tasks lambada_openai --device_map "auto" --iters 1
~~

Reproduction Steps

~

Environment Information

~

Error Logs

File "/home/wenhuach/miniforge3/envs/autoround/lib/python3.13/site-packages/torch/nn/functional.py", line 2567, in embedding
return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)
~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: Expected all tensors to be on the same device, but got index is on cuda:0, different from other tensors on cuda:1 (when checking argument in method wrapper_CUDA__index_select)

Additional Context

No response

[xin3he]

根因：parallelize=True 让 lm-eval 用 device_map="auto" 加载模型，accelerate 把 OPT 的 lm_head（与 embed_tokens 共享权重）放到 cuda:1，但输入 input_ids 在 cuda:0 → tied weight 跨卡报错。

[xin3he]

报错逻辑发生在lm_head。 解决方案：

# 改前
model_args += ",parallelize=True"
# 改后
model_args += ",parallelize=True,device_map=sequential"

[xin3he]

#2047