# Training Arguments to do pure bf16 training?

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftraining-arguments-to-do-pure-bf16-training%2F66483%3E%5C
**Category:** 🤗Transformers\
**Created:** [December 20, 2023, 3:42pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftraining-arguments-to-do-pure-bf16-training%2F66483 "2023-12-20T15:42:53Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![chiragjn](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/chiragjn/32/22144_2.png) [@chiragjn](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fchiragjn)%5C
**Post date:** [December 20, 2023, 3:42pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftraining-arguments-to-do-pure-bf16-training%2F66483%2F1 "2023-12-20T15:42:53Z")

</div>

I am trying to finetune some large language models and I was wondering if there is a configuration to do pure bf16 training i.e. parameters, optimizer, gradients all in bf16?

I already know about `--bf16 True` which uses torch AMP, but I don’t want to use mixed precision at all if possible.  
I know there are certain risks involved with stability but getting rid of mixed precision will help reduce memory footprint and everything in bf16 will keep things fast.

I am currently exploring how Lightning does this with their `bf16-true` mode to replicate it with trainer.