# Error loading Llama model

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ferror-loading-llama-model%2F67190%3E%5C
**Category:** Beginners\
**Created:** [December 27, 2023, 5:08pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ferror-loading-llama-model%2F67190 "2023-12-27T17:08:47Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![XIAOXIAOFROZEN](https://avatars.discourse-cdn.com/v4/letter/x/c68b51/32.png) [@XIAOXIAOFROZEN](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FXIAOXIAOFROZEN)%5C
**Post date:** [December 27, 2023, 5:08pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ferror-loading-llama-model%2F67190%2F1 "2023-12-27T17:08:47Z")

</div>

I got a problem loading the Llama-2-13b-hf model with the following code,

LlamaForCausalLM.from\_pretrained(base\_model, trust\_remote\_code=True, device\_map = “cuda:0”, load\_in\_8bit = True),

A error returned as  
ImportError: Using `load_in_8bit=True` requires Accelerate: `pip install accelerate` and the latest version of bitsandbytes `pip install -i https://test.pypi.org/simple/ bitsandbytes` or pip install bitsandbytes`.

But I am pretty sure the two package are installed with version bitsandbytes-0.41.3, accelerate 0.25.0. Anyone has encountered the issue before and know how to resolve it, thank you very much!

---

<div class="post-metadata">

**Author:** ![nielsr](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/nielsr/32/39617_2.png) [@nielsr](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fnielsr)%5C
**Post date:** [December 27, 2023, 6:38pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ferror-loading-llama-model%2F67190%2F2 "2023-12-27T18:38:52Z")

</div>

Hi,

Are you running in Google Colab? If yes, restarting the runtime may help.

---

<div class="post-metadata">

**Author:** ![XIAOXIAOFROZEN](https://avatars.discourse-cdn.com/v4/letter/x/c68b51/32.png) [@XIAOXIAOFROZEN](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FXIAOXIAOFROZEN)%5C
**Post date:** [December 27, 2023, 7:21pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ferror-loading-llama-model%2F67190%2F3 "2023-12-27T19:21:14Z")

</div>

Hi, nielsr, I am running it in AWS sagemaker, I found a solution in forum by downgrading transformers to 4.30.0, the error disappears but a new error emerge: SafetensorError: Error while deserializing header: HeaderTooLarge, wonder if you have any clue on that, thank you!

---

<div class="post-metadata">

**Author:** ![nielsr](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/nielsr/32/39617_2.png) [@nielsr](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fnielsr)%5C
**Post date:** [December 27, 2023, 7:35pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ferror-loading-llama-model%2F67190%2F4 "2023-12-27T19:35:18Z")

</div>

It should also work with the latest Transformers version. Could you try creating a new environment?

---

<div class="post-metadata">

**Author:** ![XIAOXIAOFROZEN](https://avatars.discourse-cdn.com/v4/letter/x/c68b51/32.png) [@XIAOXIAOFROZEN](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FXIAOXIAOFROZEN)%5C
**Post date:** [December 27, 2023, 8:14pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ferror-loading-llama-model%2F67190%2F5 "2023-12-27T20:14:31Z")

</div>

Yes, latest version of transformers also pass that error, thank you very much!

---

<div class="post-metadata">

**Author:** ![akshat-kumar-akight](https://avatars.discourse-cdn.com/v4/letter/a/858c86/32.png) [@akshat-kumar-akight](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fakshat-kumar-akight)%5C
**Post date:** [March 9, 2024, 8:14am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ferror-loading-llama-model%2F67190%2F6 "2024-03-09T08:14:43Z")

</div>

Hi, were you able to resolve the error?