# Loading trained model with new vocab

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Floading-trained-model-with-new-vocab%2F15291%3E%5C
**Category:** Beginners\
**Created:** [March 3, 2022, 3:19am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Floading-trained-model-with-new-vocab%2F15291 "2022-03-03T03:19:30Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![newbie001](https://avatars.discourse-cdn.com/v4/letter/n/c77e96/32.png) [@newbie001](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fnewbie001)%5C
**Post date:** [March 3, 2022, 3:19am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Floading-trained-model-with-new-vocab%2F15291%2F1 "2022-03-03T03:19:30Z")

</div>

I’m working with DistilBERT model. I created a few new vocab tokens and trained the model.

```auto
        tokenizer.add_tokens(["[NEW_TOKEN]"], special_tokens=True)
        model.resize_token_embeddings(len(tokenizer))
        tokenizer.save_pretrained(args.tokenizer_dir)

```

Everything is working fine but I’m hitting some issues while loading the trained model from a check-point:

```auto
$ model = BERTMODEL.from_pretrained(checkpoint_path) 

RuntimeError: Error(s) in loading state_dict for BERTMODEL:
    size mismatch for vocab_projector.weight: copying a param with shape torch.Size([30522, 768]) from checkpoint, the shape in current model is torch.Size([30524, 768]).

```

Size mismatch 30522 vs 30524 because I added 2 new tokens in the vocab. I’m not sure how to pass the new vocab config when loading the model from a checkpoint:

`model = BERTMODEL.from_pretrained(checkpoint_path) # <--- ????`

Any hint regarding what is missing in my code?

---

<div class="post-metadata">

**Author:** ![anwarika](https://avatars.discourse-cdn.com/v4/letter/a/e0b2c6/32.png) [@anwarika](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fanwarika)%5C
**Post date:** [March 4, 2022, 4:48pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Floading-trained-model-with-new-vocab%2F15291%2F2 "2022-03-04T16:48:34Z")

</div>

I don’t know if this would help, but you could pass `ignore_mismatched_sizes=True` when loading model.

---

<div class="post-metadata">

**Author:** ![Stephencoder](https://avatars.discourse-cdn.com/v4/letter/s/5e9695/32.png) [@Stephencoder](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FStephencoder)%5C
**Post date:** [April 10, 2024, 12:36am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Floading-trained-model-with-new-vocab%2F15291%2F3 "2024-04-10T00:36:38Z")

</div>

> [@newbie001](#):
>
> working fine but I’m hitting some issues while loading the tr

I got the same issue as you, adding some new tokens to the tokenizer,  
with code:  
`model.resize_token_embeddings(len(tokenizer))`  
training and save the model is OK, but the problem happened when I try to load the model  
the raising error is:  
`size mismatch for text_model.model.embed_tokens.weight: copying a param with shape torch.Size([65048, 2048]) from checkpoint, the shape in current model is torch.Size([65037, 2048]).`

Anyone can help please?  
Thanks a lot in advance!!