DPO algo implementation

[ZKadelzq]

Question

Hi teams. It seems AReaL doesn't support DPO algorithm right now, will there be any use cases about implementing DPO via AReaL? Also, the WeChat Group QR-code expired, can you please share a updated one?

Question Scope

• Others

[garrett4wade]

The WeChat QR code is alive again. Implementing a DPO trainer should be similar to how to implement the pairwise reward model trainer @areal/trainer/rw_trainer.py , and would like to adopt community's implementation.