# Customizing T5 tokenizer for finetuning

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcustomizing-t5-tokenizer-for-finetuning%2F16084%3E%5C
**Category:** 🤗Transformers\
**Created:** [March 25, 2022, 4:54pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcustomizing-t5-tokenizer-for-finetuning%2F16084 "2022-03-25T16:54:11Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![phade160](https://avatars.discourse-cdn.com/v4/letter/p/bb73d2/32.png) [@phade160](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fphade160)%5C
**Post date:** [March 25, 2022, 4:54pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcustomizing-t5-tokenizer-for-finetuning%2F16084%2F1 "2022-03-25T16:54:11Z")

</div>

Hi,  
I am finetuning a T5 model for QA on my dataset but the vocab is so different than the tokenizer’s, which results in an excessive length of token\_ids/tokens. can I train a new tokenizer from the existing one and use it for finetuning? if yes, any tips/resources to aid?  
Thanks

---

<div class="post-metadata">

**Author:** ![0bzrv3](https://avatars.discourse-cdn.com/v4/letter/0/87869e/32.png) [@0bzrv3](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2F0bzrv3)%5C
**Post date:** [May 2, 2024, 7:16pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcustomizing-t5-tokenizer-for-finetuning%2F16084%2F2 "2024-05-02T19:16:16Z")

</div>

what i did was make a set of words i want to be tokenized and used `tokenizer.add_tokens(new_tokens)`.  
remember to resize embedding weights in the model as well: `model.resize_token_embeddings(len(tokenizer))`