# Error loading tokenizer from local checkpoint directory

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ferror-loading-tokenizer-from-local-checkpoint-directory%2F44428%3E%5C
**Category:** 🤗Tokenizers\
**Created:** [June 25, 2023, 12:43am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ferror-loading-tokenizer-from-local-checkpoint-directory%2F44428 "2023-06-25T00:43:47Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![dmatekenya](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/dmatekenya/32/16995_2.png) [@dmatekenya](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fdmatekenya)%5C
**Post date:** [June 25, 2023, 12:43am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ferror-loading-tokenizer-from-local-checkpoint-directory%2F44428%2F1 "2023-06-25T00:43:47Z")

</div>

I’m fine-tuning Whisper for a low-resource language (Chichewa) and following [this tutorial](https://huggingface.co/blog/fine-tune-whisper). One change I have made is to provide a local directory to save the model instead of pushing to Hub. When its time to use the fine-tuned model using the pipeline module, I’m getting this error:  
`Can't load tokenizer for '/content/drive/My Drive/Chichewa-ASR/models/whisper-small-chich/checkpoint-1000. If you were trying to load it from 'https://huggingface.co/models', make sure you don't have a local directory with the same name. Otherwise, make sure '/content/drive/My Drive/Chichewa-ASR/models/whisper-small-chich/checkpoint-1000' is the correct path to a directory containing all relevant files for a WhisperTokenizer tokenizer.`  
When I check the model repository used by the tutorial author, I see that it has several files I don’t have in my model checkpoint directory such as `tokenizer_config.json`. I’m just wondering how do I ensure these files are saved in the local model checkpoint directories?

---

<div class="post-metadata">

**Author:** ![www007](https://avatars.discourse-cdn.com/v4/letter/w/3da27b/32.png) [@www007](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fwww007)%5C
**Post date:** [November 1, 2023, 2:45am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ferror-loading-tokenizer-from-local-checkpoint-directory%2F44428%2F2 "2023-11-01T02:45:20Z")

</div>

I had the same problem, have you solved it?

---

<div class="post-metadata">

**Author:** ![Unspoiled-Egg](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/unspoiled-egg/32/25070_2.png) [@Unspoiled-Egg](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FUnspoiled-Egg)%5C
**Post date:** [April 4, 2024, 7:12am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ferror-loading-tokenizer-from-local-checkpoint-directory%2F44428%2F3 "2024-04-04T07:12:28Z")

</div>

@sanchit-gandhi can you help with this, I have similar problem

---

<div class="post-metadata">

**Author:** ![Mutturaj](https://avatars.discourse-cdn.com/v4/letter/m/4af34b/32.png) [@Mutturaj](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FMutturaj)%5C
**Post date:** [May 13, 2024, 1:47am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ferror-loading-tokenizer-from-local-checkpoint-directory%2F44428%2F4 "2024-05-13T01:47:49Z")

</div>

I had same problem,  
In tried to copy the tokenizer\_config.json file from whisper model output directory to checkpoint directory

Then i load the model from checkpoint directory

It worked form me