BitsAndBytesConfig cannot resolve torch.bfloat16

[qiisziilbash]

from transformers import BitsAndBytesConfig
quantization_config = BitsAndBytesConfig(
            load_in_4bit=True,
            bnb_4bit_quant_type="nf4",
            bnb_4bit_compute_dtype="torch.bfloat16",
)

Errors out:

AttributeError: module 'torch' has no attribute 'torch.bfloat16'

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB