# Flash Attention 2 Error on Mistral Based Model

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fflash-attention-2-error-on-mistral-based-model%2F66167%3E%5C
**Category:** Beginners\
**Created:** [December 18, 2023, 7:05am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fflash-attention-2-error-on-mistral-based-model%2F66167 "2023-12-18T07:05:26Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![luisfrentzen](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/luisfrentzen/32/22208_2.png) [@luisfrentzen](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fluisfrentzen)%5C
**Post date:** [December 18, 2023, 7:05am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fflash-attention-2-error-on-mistral-based-model%2F66167%2F1 "2023-12-18T07:05:26Z")

</div>

Hi, I am trying to enable flash attention 2 on a model yet I got this error:

```auto
ValueError: past key much have a shape of (`batch_size, num_heads, self.config.sliding_window-1, head_dim`), got torch.Size([4, 8, 3968, 128])

```

I am using openchat’s openchat\_3.5 7B model which I believe is based on mistral [openchat/openchat\_3.5 · Hugging Face](https://huggingface.co/openchat/openchat_3.5). I am loading the model as such:

```auto
model = AutoModelForCausalLM.from_pretrained(model_name, 
                             device_map="auto", 
                             torch_dtype=torch.bfloat16,
                             use_flash_attention_2="flash_attention_2",
                             low_cpu_mem_usage=True)

```

Can someone explain the error for me? Or point me to a resource which can help me understand the problem, thank you.