# Tokenizer is not being loaded on Huggingface Inference

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftokenizer-is-not-being-loaded-on-huggingface-inference%2F23505%3E%5C
**Category:** 🤗Tokenizers\
**Created:** [September 22, 2022, 7:30pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftokenizer-is-not-being-loaded-on-huggingface-inference%2F23505 "2022-09-22T19:30:57Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![ritwikm](https://avatars.discourse-cdn.com/v4/letter/r/3bc359/32.png) [@ritwikm](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fritwikm)%5C
**Post date:** [September 22, 2022, 7:30pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftokenizer-is-not-being-loaded-on-huggingface-inference%2F23505%2F1 "2022-09-22T19:30:57Z")

</div>

I was using [this](https://reyfarhan.com/posts/easy-gpt2-finetuning-huggingface/) to finetune a GPT medium.

```auto
tokenizer = GPT2Tokenizer.from_pretrained('gpt2', bos_token='<|startoftext|>', eos_token='<|endoftext|>', pad_token='<|pad|>') #gpt2-medium
configuration = GPT2Config.from_pretrained('gpt2', output_hidden_states=False)
model = GPT2LMHeadModel.from_pretrained("gpt2", config=configuration)
model.resize_token_embeddings(len(tokenizer))

# after finetuning

model.save_pretrained(output_dir)
tokenizer.save_pretrained(output_dir)

```

But when I upload the model and tokenizer on the model hub ([ritwikm/gandhi-gpt](https://huggingface.co/ritwikm/gandhi-gpt)), I see the following error

```auto
Can't load tokenizer using from_pretrained, please update its configuration: No such file or directory (os error 2)

```

On my local machine, I am loading the same tokenizer and model using the following lines:

```auto
model = model.from_pretrained(output_dir).to(device)
tokenizer = tokenizer.from_pretrained(output_dir)

```

And it works fine. It just fails on the inference.

I have tried many solutions like using only those tokenizer files which are available in the official repo (`gpt2`).

I saw the post by bala1802, even that didn’t help.

What can be done?