# Loading and using Autotrain model error

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Floading-and-using-autotrain-model-error%2F63777%3E%5C
**Category:** 🤗AutoTrain\
**Created:** [November 28, 2023, 1:56am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Floading-and-using-autotrain-model-error%2F63777 "2023-11-28T01:56:08Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![palsc](https://avatars.discourse-cdn.com/v4/letter/p/9de053/32.png) [@palsc](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fpalsc)%5C
**Post date:** [November 28, 2023, 1:56am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Floading-and-using-autotrain-model-error%2F63777%2F1 "2023-11-28T01:56:08Z")

</div>

I trained llama-7b-chat-hf using the following command as I saw in the video  
autotrain llm --train --project\_name peft-tune-llama-credit-card-fraud-v2 --model meta-llama/Llama-2-7b-chat-hf --data\_path . --use\_peft --use\_int4 --learning\_rate 2e-4 --train\_batch\_size 6 --num\_train\_epochs 10 --trainer sft

On the same Ubunutu EC2 instance that I used to train the model, I am trying to load the model. The model is not pushed to the hub. I am running into issues as follows:

import torch  
import transformers  
from peft import PeftModel  
from transformers import AutoModelForCausalLM, AutoTokenizer, LlamaTokenizer, StoppingCriteria, StoppingCriteriaList, TextIteratorStreamer

model=‘/home/ubuntu/LLM/meta-llama/peft-tune-llama-2-7b-chat-hf-credit-card-fraud-v2’  
model\_name = “meta-llama/Llama-2-7b-chat-hf”  
adapters\_name = {‘privacy/repo\_name’: ‘/home/ubuntu/LLM/meta-llama/peft-tune-llama-2-7b-chat-hf-credit-card-fraud-v2’}

print(f"Starting to load the model {model} into memory")

m = AutoModelForCausalLM.from\_pretrained(  
model\_name,  
load\_in\_4bit=True,  
torch\_dtype=torch.bfloat16,  
device\_map={“”: 0}  
)

m = PeftModel.from\_pretrained(m, model)  
m = m.merge\_and\_unload()  
tok = LlamaTokenizer.from\_pretrained(model)  
tok.bos\_token\_id = 1  
stop\_token\_ids = [0]

However, getting an exception as follows

Traceback (most recent call last):  
File “/home/ubuntu/LLM/meta-llama/model\_loader.py”, line 19, in  
m = PeftModel.from\_pretrained(m, model)  
File “/home/ubuntu/LLM/meta-llama/llama-train/lib/python3.10/site-packages/peft/peft\_model.py”, line 278, in from\_pretrained  
model.load\_adapter(model\_id, adapter\_name, is\_trainable=is\_trainable, \*\*kwargs)  
File “/home/ubuntu/LLM/meta-llama/llama-train/lib/python3.10/site-packages/peft/peft\_model.py”, line 554, in load\_adapter  
adapters\_weights = load\_peft\_weights(model\_id, device=torch\_device, \*\*hf\_hub\_download\_kwargs)  
File “/home/ubuntu/LLM/meta-llama/llama-train/lib/python3.10/site-packages/peft/utils/save\_and\_load.py”, line 171, in load\_peft\_weights  
has\_remote\_safetensors\_file = hub\_file\_exists(  
File “/home/ubuntu/LLM/meta-llama/llama-train/lib/python3.10/site-packages/peft/utils/hub\_utils.py”, line 24, in hub\_file\_exists  
url = hf\_hub\_url(repo\_id=repo\_id, filename=filename, repo\_type=repo\_type, revision=revision)  
File “/home/ubuntu/LLM/meta-llama/llama-train/lib/python3.10/site-packages/huggingface\_hub/utils/\_validators.py”, line 110, in \_inner\_fn  
validate\_repo\_id(arg\_value)  
File “/home/ubuntu/LLM/meta-llama/llama-train/lib/python3.10/site-packages/huggingface\_hub/utils/\_validators.py”, line 158, in validate\_repo\_id  
raise HFValidationError(

Question:  
How do I fix this error, load the model and use it for a query?  
IF this is all wrong, can you provide a sample code to load and use the local saved model, without going to hub?