# Lora: missing adapter keys while loading the checkpoint
**URL:**
**Author:**  [@suruti94](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsuruti94)%5C
**Post date:** [December 24, 2024, 8:36pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Flora-missing-adapter-keys-while-loading-the-checkpoint%2F132920%2F1 "2024-12-24T20:36:26Z")
Hi,
When I load a checkpoint during inference using “PeftModel.from\_pretrained”, I see missing adapter keys which are weights from {qproj,k\_proj, v\_proj, o\_proj, gate\_proj, up\_proj,down\_proj} of lora\_A and lora\_B from all layers. This seems to affect the inference. Googling this does not give me a good resolution. Can someone point me in the right direction ?
Thanks
Mohan
---
**Author:**  [@suruti94](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsuruti94)%5C
**Post date:** [January 5, 2025, 10:20pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Flora-missing-adapter-keys-while-loading-the-checkpoint%2F132920%2F2 "2025-01-05T22:20:53Z")
From what I can tell, If i push the model to the repository and later do this:
from peft import AutoPeftModelForCausalLM
model = AutoPeftModelForCausalLM.from\_pretrained()
I don’t get any warnings. The adapter-config.json in the repository points to the base model from which it was trained on. But using the above code to load from the checkpoint itself did not work. I am not sure why.
Thanks
Mohan
---
**Author:**  [@Alanturner2](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FAlanturner2)%5C
**Post date:** [January 6, 2025, 7:23am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Flora-missing-adapter-keys-while-loading-the-checkpoint%2F132920%2F3 "2025-01-06T07:23:13Z")
Hi Mohan,
The issue you’re describing typically occurs when the LoRA adapter weights were not saved correctly during training or when there’s a mismatch between the base model and the adapter configuration.
Here are a few things you can check to troubleshoot this issue:
1. **Verify LoRA Weights During Training**
2. **Check Base Model Compatibility**
3. **Adapter Configuration**
Verify that the adapter configuration (e.g., `PeftConfig`) is correctly set up. The `PeftModel.from_pretrained()` function depends on this configuration to properly initialize and load the adapter weights. Check the adapter type, rank, and any additional parameters.
4. **Use `ignore_mismatched_keys=True`**
When loading the model, you can set the `ignore_mismatched_keys=True` flag to avoid stopping the process due to missing keys. However, this is more of a workaround than a fix. Use this cautiously, as it may lead to incomplete or incorrect inference results.
5. **Log the Missing Keys**
Use the `from_pretrained()` function with verbose logging to identify the specific missing keys. For example:
6. **Re-Export the LoRA Adapter**
If the problem persists, re-export the trained LoRA adapter weights. Ensure you save them using `save_pretrained()` on the `PeftModel` after training:
If none of these resolve the issue, you might need to share more details about the training setup, base model, and adapter configuration for a more specific diagnosis.
Let me know how it goes!
Best,
Alan