# AutoTokenizer.from\_pretrained() suddenly raises an error

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fautotokenizer-from-pretrained-suddenly-raises-an-error%2F153809%3E%5C
**Category:** 🤗Transformers\
**Created:** [May 6, 2025, 7:41pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fautotokenizer-from-pretrained-suddenly-raises-an-error%2F153809 "2025-05-06T19:41:08Z")\
**Posts on this page:** 1\
**Showing post:** 4

<div class="post-metadata">

**Author:** ![smostafanejad](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/smostafanejad/32/34306_2.png) [@smostafanejad](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsmostafanejad)%5C
**Post date:** [May 7, 2025, 2:39pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fautotokenizer-from-pretrained-suddenly-raises-an-error%2F153809%2F4 "2025-05-07T14:39:35Z")

</div>

OK since this was an `EnvironmentError` I checked everything and I think I have found the culprit.  
In my bashrc, I had `export HF_HUB_ENABLE_HF_TRANSFER=1` set which means the problem might have something to do with an inconsistency with the **hf-transfer** package. Reading Hugging Face’s [Environment Variable documentation](https://huggingface.co/docs/huggingface_hub/v0.31.0/package_reference/environment_variables) gave the clue about the possibility of such incidents and undefined behavior

```plaintext
HF_HUB_ENABLE_HF_TRANSFER

Set to True to download files from the Hub using hf_transfer. It’s a Rust-based package that enables faster download (up to x2 speed-up). Be aware that this is still experimental so it might cause issues in your workflow. In particular, it does not support features such as progress bars, resume download, proxies or error handling.

Note: hf_transfer has to be installed separately from Pypi.

```

I have forced a reinstall and upgrade through pip and apparently that resolved the issues with both supercomputer and data center machines which had problems calling the `AutoTokenizer.from_pretrained()`.

---

_[View the full topic](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fautotokenizer-from-pretrained-suddenly-raises-an-error%2F153809)._