# "Some weights were not used" message with AutoModel
**URL:**
**Author:**  [@btlgs2000](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fbtlgs2000)%5C
**Post date:** [February 28, 2023, 8:29pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fsome-weights-were-not-used-message-with-automodel%2F32745%2F1 "2023-02-28T20:29:25Z")
Hi, I’m a newbie trying to learn Hugginface transformers api.
The line
model = AutoModel.from\_pretrained(“bert-base-cased”)
give the following message:
> Some weights of the model checkpoint at bert-base-cased were not used when initializing BertModel: [‘cls.predictions.transform.dense.bias’, ‘cls.predictions.bias’, ‘cls.seq\_relationship.weight’, ‘cls.predictions.transform.LayerNorm.bias’, ‘cls.seq\_relationship.bias’, ‘cls.predictions.transform.LayerNorm.weight’, ‘cls.predictions.decoder.weight’, ‘cls.predictions.transform.dense.weight’]
I didn’t expect that. I know you can obtain such a message if you init a ModelForSomething using a checkpoint for a basic model, but this is not the case.
Any help?
Thanks
---
**Author:**  [@btlgs2000](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fbtlgs2000)%5C
**Post date:** [February 28, 2023, 9:47pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fsome-weights-were-not-used-message-with-automodel%2F32745%2F2 "2023-02-28T21:47:16Z")
Probably I understood the problem: in the checkpoint there are some weights related to training layers that are dropped and are not present in BertModel class. So it shouln’t be a problem
---
**Author:**  [@sgugger](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsgugger)%5C
**Post date:** [March 1, 2023, 5:10pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fsome-weights-were-not-used-message-with-automodel%2F32745%2F3 "2023-03-01T17:10:06Z")
Yup, exactly!
---
**Author:**  [@btlgs2000](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fbtlgs2000)%5C
**Post date:** [March 1, 2023, 7:37pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fsome-weights-were-not-used-message-with-automodel%2F32745%2F4 "2023-03-01T19:37:28Z")
Thank you for replying… When I read more carefully the warning I realized that there were not uninitialized weights in the model (it can happen if you use a base model checkpoint for initializing a ModelForSomething) but the opposite: there were unused checkpoint weights
---
**Author:**  [@yasser97](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fyasser97)%5C
**Post date:** [May 21, 2024, 9:24am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fsome-weights-were-not-used-message-with-automodel%2F32745%2F5 "2024-05-21T09:24:56Z")
I am having the same problem but with TinyLlama, I fine-tuned a model for a specific task (question answering) and when I am loading the fine-tuned model it shows this warning in addition to:
Some weights of LlamaForCausalLM were not initialized from the model checkpoint at and are newly initialized: [‘lm\_head.weight’, ‘model.embed\_tokens.weight’, ‘model.norm.weight’]
what could be the problem?
Notes:
- I am using multu-gpu training and here is how I save the model:
```python
save_dir = os.path.join(project_dir, new_model)
print(f"Saving model to {save_dir}...")
# save the tokenizer
tokenizer.save_pretrained(save_dir)
# save the model
unwrapped_model = accelerator.unwrap_model(model)
unwrapped_model.save_pretrained(save_dir, is_main_process=accelerator.is_main_process, save_function=accelerator.save)
```
- here how I load the model:
```auto
model = AutoModelForCausalLM.from_pretrained(model_path, local_files_only=True)
```