# Loading a locally saved model is very slow

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Floading-a-locally-saved-model-is-very-slow%2F58025%3E%5C
**Category:** 🤗Transformers\
**Created:** [October 10, 2023, 11:37am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Floading-a-locally-saved-model-is-very-slow%2F58025 "2023-10-10T11:37:19Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![shubhampatidar](https://avatars.discourse-cdn.com/v4/letter/s/59ef9b/32.png) [@shubhampatidar](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fshubhampatidar)%5C
**Post date:** [October 10, 2023, 11:37am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Floading-a-locally-saved-model-is-very-slow%2F58025%2F1 "2023-10-10T11:37:19Z")

</div>

Hello all, I am loading a llama 2 model from HF hub on g5.48xlarge on sagemaker notebook instance using the below commands and it take less than 5 minutes to completes the loading

```auto
model_id = "meta-llama/Llama-2-7b-chat-hf"
tokenizer = AutoTokenizer.from_pretrained(model_id)
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained(model_id,load_in_8bit=True, torch_dtype=torch.float16)

```

Thereafter I save the model locally using the below code

```auto
save_directory = "models/Llama-2-7b-chat-hf"
model.save_pretrained(save_directory)
tokenizer.save_pretrained(save_directory)

```

then I try loading the locally saved model using the below code

```auto
model = AutoModelForCausalLM.from_pretrained(save_directory, device_map = 'auto')
tokenizer = AutoTokenizer.from_pretrained(save_directory)

```

but it took around 30 minutes to load the same model from local as compared to less than 5 minute from HF hub, what can be the reason behind it and how can I load the model faster.

Any answer would be appreciated, Thanks.

---

<div class="post-metadata">

**Author:** ![anon57993558](https://avatars.discourse-cdn.com/v4/letter/a/df705f/32.png) [@anon57993558](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fanon57993558)%5C
**Post date:** [July 10, 2024, 2:53pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Floading-a-locally-saved-model-is-very-slow%2F58025%2F2 "2024-07-10T14:53:25Z")

</div>

Hi there, did you ever find a solution for this? - having the same issues here. Have run this code:  
config = AutoConfig.from\_pretrained(storage\_model\_path)

tokenizer = AutoTokenizer.from\_pretrained(storage\_model\_path)

model = AutoModelForCausalLM.from\_pretrained(  
storage\_model\_path,  
config=config,  
torch\_dtype=torch.bfloat16,  
trust\_remote\_code=True,  
local\_files\_only=True,  
)

And it still takes 30 mins as opposed to 45 seconds when loading from the hub directly.

Env requirements:  
transformers==4.41.2  
torch==2.2.2  
requests==2.31.0  
accelerate==0.31.0

Using Databricks 14.3 ML cluster with cuda version 11.8 - not sure if its a read bits per second setting or on the transformers side?

Would appreciate if anyone has a fix for this?