# Do I need to dequantization before merging the qlora
**URL:**
**Author:**  [@woshizouguo](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fwoshizouguo)%5C
**Post date:** [October 4, 2024, 10:50pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdo-i-need-to-dequantization-before-merging-the-qlora%2F110175%2F1 "2024-10-04T22:50:04Z")
In this DPO trainer [link](https://huggingface.co/docs/trl/main/en/dpo_trainer#downsides-to-merging-qlora-before-dpo-approach-2)
It says
```auto
As suggested by [Benjamin Marie](https://medium.com/@bnjmn_marie/dont-merge-your-lora-adapter-into-a-4-bit-llm-65b6da287997), the best option for merging QLoRA adapters is to first dequantize the base model, then merge the adapter. Something similar to [this script](https://github.com/jondurbin/qlora/blob/main/qmerge.py).
```
do we still need to do this merge? I feel the peft already integrated this feature?
---
**Author:**  [@John6666](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FJohn6666)%5C
**Post date:** [October 4, 2024, 11:36pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdo-i-need-to-dequantization-before-merging-the-qlora%2F110175%2F2 "2024-10-04T23:36:41Z")
> I feel the peft already integrated this feature?
I use the latest version of PEFT on pip, not as new as the github version…
Maybe PEFT alone support this feature.
However, applying LoRA with quantization either doesn’t work, or seems to work, but causes errors during inference.
The error is different for diffusers as well as transformers, but almost the same error occurs.
I am not sure if this is due to insufficient support in the HF library, or bugs or compatibility issues with PEFT, torch, bitsandbytes, or other libraries, but I too recommend dequantizing and then merging under the current circumstances.
---
**Author:**  [@woshizouguo](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fwoshizouguo)%5C
**Post date:** [October 5, 2024, 12:16am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdo-i-need-to-dequantization-before-merging-the-qlora%2F110175%2F3 "2024-10-05T00:16:10Z")
After I finetune the model with lora, and save the model (peftmodel),
then I run:
```auto
base_model = trainer.model.model
base_model_dequantized = qmerge.dequantize_model_2(model=base_model, dtype=torch.bfloat16)
peft_model = PeftModel.from_pretrained(base_model_dequantized, adapter_path)
merged_model = peft_model.merge_and_unload()
merged_model.save_pretrained(output_path, safe_serialization=False)
```
it shows this error when do the merging:
```auto
[rank0]: File "/opt/miniconda/lib/python3.9/site-packages/peft/tuners/lora/bnb.py", line 344, in merge
[rank0]: output = dequantize_bnb_weight(weight, state=weight.quant_state)
[rank0]: AttributeError: 'Parameter' object has no attribute 'quant_state'
```
---
**Author:**  [@woshizouguo](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fwoshizouguo)%5C
**Post date:** [October 5, 2024, 12:24am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdo-i-need-to-dequantization-before-merging-the-qlora%2F110175%2F4 "2024-10-05T00:24:35Z")
I this this function in lora merge: [peft/src/peft/tuners/lora/bnb.py at a0788a3f92c8220f68d2185aeef0266d6b725bfe · huggingface/peft · GitHub](https://github.com/huggingface/peft/blob/a0788a3f92c8220f68d2185aeef0266d6b725bfe/src/peft/tuners/lora/bnb.py#L339)
---
**Author:**  [@John6666](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FJohn6666)%5C
**Post date:** [October 5, 2024, 12:30am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdo-i-need-to-dequantization-before-merging-the-qlora%2F110175%2F5 "2024-10-05T00:30:02Z")
There are several possible causes, but it seems that there are many bitsandbytes version problems. Not sure if the latest version would work…
```auto
pip install -U bitsandbytes
```
---
**Author:**  [@woshizouguo](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fwoshizouguo)%5C
**Post date:** [October 5, 2024, 1:43am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdo-i-need-to-dequantization-before-merging-the-qlora%2F110175%2F6 "2024-10-05T01:43:00Z")
I think the reason is that in the function
`base_model_dequantized = qmerge.dequantize_model_2(model=base_model, dtype=torch.bfloat16)` it is using this function at [link](https://github.com/jondurbin/qlora/blob/cc471bbc2c802dbac67ca32bfc54a2ab875543cf/qmerge.py#L24)
So it is dequantized, but when I call merge\_and\_unload later, it seems trying to do dequantization again from the link above.
This is from this function
---
**Author:**  [@John6666](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FJohn6666)%5C
**Post date:** [October 5, 2024, 3:57am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdo-i-need-to-dequantization-before-merging-the-qlora%2F110175%2F7 "2024-10-05T03:57:04Z")
How about this?
```py
base_model_dequantized = base_model.dequantize()
```
> **[bitsandbytes](https://huggingface.co/docs/transformers/main/quantization/bitsandbytes#dequantizing-bitsandbytes-models)**
>
> We’re on a journey to advance and democratize artificial intelligence through open source and open science.
---
**Author:**  [@woshizouguo](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fwoshizouguo)%5C
**Post date:** [October 6, 2024, 7:14am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdo-i-need-to-dequantization-before-merging-the-qlora%2F110175%2F8 "2024-10-06T07:14:50Z")
The reason is` base_model = trainer.model.model` will not get the base model, the base model is modified in place so it is a lora model now, so when I run
`base_model_dequantized = qmerge.dequantize_model_2(model=base_model, dtype=torch.bfloat16)` it is not using base model but lora model.
The base model is `trainer.model.unload()`
---
**Author:**  [@John6666](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FJohn6666)%5C
**Post date:** [October 6, 2024, 9:24am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdo-i-need-to-dequantization-before-merging-the-qlora%2F110175%2F9 "2024-10-06T09:24:13Z")
Could this be PEFT’s merge\_and\_unload() calling dequantize\_model()?
If so, then dequantize would be executed in the following code.
```py
base_model = trainer.model.model
#base_model_dequantized = qmerge.dequantize_model_2(model=base_model, dtype=torch.bfloat16)
peft_model = PeftModel.from_pretrained(base_model, adapter_path)
merged_model = peft_model.merge_and_unload()
merged_model.save_pretrained(output_path, safe_serialization=False)
```
---
**Author:**  [@woshizouguo](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fwoshizouguo)%5C
**Post date:** [October 8, 2024, 5:21pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdo-i-need-to-dequantization-before-merging-the-qlora%2F110175%2F10 "2024-10-08T17:21:51Z")
The reason is that the trainer.model is still peft model. We can only get base model using unload method.
---
**Author:**  [@John6666](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FJohn6666)%5C
**Post date:** [October 9, 2024, 4:27am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdo-i-need-to-dequantization-before-merging-the-qlora%2F110175%2F11 "2024-10-09T04:27:26Z")
Hmmm, if that is the case, I wonder what is going on.
I think it might work anyway if we can stop the dequantize from being called twice…
I think this is a bug…
I wonder if the github version of PEFT could fix it?