# Is this needed: bnb 4bit use double quant = True?
**URL:**
**Author:**  [@cryptojointer](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fcryptojointer)%5C
**Post date:** [August 13, 2023, 5:14pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fis-this-needed-bnb-4bit-use-double-quant-true%2F50616%2F1 "2023-08-13T17:14:40Z")
bnb\_config = BitsAndBytesConfig(
load\_in\_4bit=True,
bnb\_4bit\_use\_double\_quant=True,
bnb\_4bit\_quant\_type=“nf4”,
bnb\_4bit\_compute\_dtype=torch.float16,
)
In the code above, I’ve seen two tutorials about QLoRA. One, didn’t have the ‘bnb\_4bit\_use\_double\_quant=True’ line, and the second did.
For a bit more clarity one was using a falcon 7b model / automodelfor casualLM and the one that didnt include the line of code was using a flacon model again but a sharded fp16 version. If that makes a difference?
Thanks, not a big deal, just tryna learn about it better.
---
**Author:**  [@pogpog](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fpogpog)%5C
**Post date:** [March 5, 2024, 5:03pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fis-this-needed-bnb-4bit-use-double-quant-true%2F50616%2F2 "2024-03-05T17:03:05Z")
It further reduces the average memory footprint by quantizing the quantization constants.
See [Making LLMs even more accessible with bitsandbytes, 4-bit quantization and QLoRA](https://huggingface.co/blog/4bit-transformers-bitsandbytes)
“options include `bnb_4bit_use_double_quant` which uses a second quantization after the first one to save an additional 0.4 bits per parameter.”
“A rule of thumb is: use double quant if you have problems with memory, use NF4 for higher precision, and use a 16-bit dtype for faster finetuning.”
---
**Author:**  [@baiyutinghuggingface](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fbaiyutinghuggingface)%5C
**Post date:** [June 27, 2024, 3:22am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fis-this-needed-bnb-4bit-use-double-quant-true%2F50616%2F3 "2024-06-27T03:22:30Z")
So, what is the cost of using `bnb_4bit_use_double_quant`? Does enabling `bnb_4bit_use_double_quant` make nan loss easier during training?
---
**Author:**  [@kyars](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fkyars)%5C
**Post date:** [March 7, 2025, 4:03pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fis-this-needed-bnb-4bit-use-double-quant-true%2F50616%2F4 "2025-03-07T16:03:38Z")
This is very clutch thanks for the rule of thumb