# Vocabulary Expansion + Finetuning

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fvocabulary-expansion-finetuning%2F69471%3E%5C
**Category:** 🤗Transformers\
**Created:** [January 17, 2024, 10:06am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fvocabulary-expansion-finetuning%2F69471 "2024-01-17T10:06:33Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Saugatkafley](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/saugatkafley/32/26738_2.png) [@Saugatkafley](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FSaugatkafley)%5C
**Post date:** [January 17, 2024, 10:06am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fvocabulary-expansion-finetuning%2F69471%2F1 "2024-01-17T10:06:33Z")

</div>

I trying to finetune a model after the vocabulary expansion. I am facing some problems when the vocab is resized

```python
tokenizer.add_tokens(list(new_tokens))
model.resize_token_embeddings(len(tokenizer),)

```

```python
output_dir = "opt-125-alpaca-ne-vocab-expansion"
model_save_path = output_dir + "/saved_model"

trainer = Trainer(
    model=model,
    train_dataset=mapped_dataset,
    args=TrainingArguments(
        per_device_train_batch_size=1,
        gradient_accumulation_steps=4,
        warmup_steps=100,
        save_strategy="steps",
        save_steps=500,
        # max_steps=10, # args.max_steps,
        num_train_epochs=1,
        learning_rate=1e-5,
        fp16=True,
        logging_steps=1,
        output_dir=output_dir,
        ),
    # data_collator=collator,
    
    data_collator=DataCollatorForLanguageModeling(tokenizer, mlm=False,),
)

```

---

<div class="post-metadata">

**Author:** ![mertbozkir](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/mertbozkir/32/36929_2.png) [@mertbozkir](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fmertbozkir)%5C
**Post date:** [August 14, 2025, 1:10pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fvocabulary-expansion-finetuning%2F69471%2F2 "2025-08-14T13:10:57Z")

</div>

Any update on this? Have you able to solve it? 🙂