# Cannot load tokenizer for llama2

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcannot-load-tokenizer-for-llama2%2F55046%3E%5C
**Category:** 🤗Tokenizers\
**Created:** [September 14, 2023, 5:12pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcannot-load-tokenizer-for-llama2%2F55046 "2023-09-14T17:12:26Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![saireddy](https://avatars.discourse-cdn.com/v4/letter/s/c57346/32.png) [@saireddy](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsaireddy)%5C
**Post date:** [September 14, 2023, 5:12pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcannot-load-tokenizer-for-llama2%2F55046%2F1 "2023-09-14T17:12:26Z")

</div>

Hi,  
I am trying to load tokenizer for llama2 using AutoTokenizer but I am facing this issue

“”“OSError: Can’t load tokenizer for ‘meta-llama/Llama-2-7b-hf’. If you were trying to load it from ‘[Models - Hugging Face](https://huggingface.co/models)’, make sure you don’t have a local directory with the same name. Otherwise, make sure ‘meta-llama/Llama-2-7b-hf’ is the correct path to a directory containing all relevant files for a LlamaTokenizer tokenizer.  
“””

any help is appreciated!

Thanks

---

<div class="post-metadata">

**Author:** ![linying111](https://avatars.discourse-cdn.com/v4/letter/l/a9adbd/32.png) [@linying111](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Flinying111)%5C
**Post date:** [October 21, 2023, 7:47am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcannot-load-tokenizer-for-llama2%2F55046%2F2 "2023-10-21T07:47:45Z")

</div>

Hello, have you solved this problem,I’m having the same issue too?

---

<div class="post-metadata">

**Author:** ![saireddy](https://avatars.discourse-cdn.com/v4/letter/s/c57346/32.png) [@saireddy](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsaireddy)%5C
**Post date:** [October 25, 2023, 7:56pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcannot-load-tokenizer-for-llama2%2F55046%2F3 "2023-10-25T19:56:26Z")

</div>

yes, we need to pass access\_token and proxy(if applicable) for tokenizers as well

---

<div class="post-metadata">

**Author:** ![Hemanth05](https://avatars.discourse-cdn.com/v4/letter/h/d2c977/32.png) [@Hemanth05](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FHemanth05)%5C
**Post date:** [February 2, 2024, 3:27pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcannot-load-tokenizer-for-llama2%2F55046%2F4 "2024-02-02T15:27:39Z")

</div>

hi, how did you do that?

---

<div class="post-metadata">

**Author:** ![saireddy](https://avatars.discourse-cdn.com/v4/letter/s/c57346/32.png) [@saireddy](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsaireddy)%5C
**Post date:** [March 20, 2024, 2:14pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcannot-load-tokenizer-for-llama2%2F55046%2F5 "2024-03-20T14:14:02Z")

</div>

tokenizer = AutoTokenizer.from\_pretrained(model\_id, token=access\_token)  
access\_token can be generated from huggingface UI

---

<div class="post-metadata">

**Author:** ![shinieaggarwal](https://avatars.discourse-cdn.com/v4/letter/s/ce73a5/32.png) [@shinieaggarwal](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fshinieaggarwal)%5C
**Post date:** [September 7, 2024, 7:17pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcannot-load-tokenizer-for-llama2%2F55046%2F6 "2024-09-07T19:17:27Z")

</div>

hi, what did you put in the variable model\_id?? i am trying it on colab and facing this same issue

---

<div class="post-metadata">

**Author:** ![saireddy](https://avatars.discourse-cdn.com/v4/letter/s/c57346/32.png) [@saireddy](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsaireddy)%5C
**Post date:** [September 13, 2024, 3:48pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcannot-load-tokenizer-for-llama2%2F55046%2F7 "2024-09-13T15:48:35Z")

</div>

model\_id should be the name from hf model repository  
for example : meta-llama/Meta-Llama-3-8B