# Direct Load vs. Base Model + LoRA: How Should You Use It?
**URL:**
**Author:**  [@adobby](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fadobby)%5C
**Post date:** [March 12, 2025, 5:07am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdirect-load-vs-base-model-lora-how-should-you-use-it%2F145328%2F1 "2025-03-12T05:07:19Z")
I have trained an embedding model using LoRA and have some questions regarding how to load the trained LoRA adapter.
There are two different ways to load the LoRA adapter:
(1) Direct loading method
```python
direct_adapter = SentenceTransformer(lora_adapter_path)
```
(2) Adding to a base model
```python
model = SentenceTransformer(base_model)
model.load_adapter(lora_adapter_path)
```
I would like to know if there are any **performance differences** between these two methods and the reason for using one method over the other.
---
**Author:**  [@John6666](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FJohn6666)%5C
**Post date:** [March 12, 2025, 10:10am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdirect-load-vs-base-model-lora-how-should-you-use-it%2F145328%2F2 "2025-03-12T10:10:12Z")
I think the former may be slightly faster. If you apply the **merge\_and\_unload()** function mentioned in the following issue to the latter, it should become a merged state similar to the former.
Once merged, there are no particular drawbacks other than the fact that it cannot be separated.
>
>
> I am working on fine tuning a 7B model and due to the size, we trained it with l…ora- by following the guidance (https://sbert.net/examples/training/peft/README.html)
> \`\`\`python
> peft\_config = LoraConfig(
> task\_type=TaskType.FEATURE\_EXTRACTION,
> inference\_mode=False,
> r=8,
> lora\_alpha=32,
> lora\_dropout=0.1,
> )
>
> model.add\_adapter(peft\_config)
> \`\`\`
>
> Training works great and we are looking for some guidances to merge the lora layer with the base model and saved.
>
> What we have tried:
> 1. \`model.save\_pretrained("")\` =\> only save the lora layer
> 2. using \`peft\` library: this doesn't seem to work correctly, as the inference result is the same as the base model.
> \`\`\`
> model.save\_pretrained(tmp\_path)
> base\_model = SentenceTransformer(model\_name\_or\_path=model\_path)
> adapter\_model = PeftModel.from\_pretrained(base\_model, adapter\_tmp\_path)
> merged\_model = adapter\_model.merge\_and\_unload()
> merged\_model.config = transformers.AutoConfig.from\_pretrained(model\_path)
> merged\_model.save\_pretrained(path)
> \`\`\`
>
> We are reaching out for insights about how to merge the sentence transformer trained peft model with the base model. Thanks!