# Llama2 pad token for batched inference

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fllama2-pad-token-for-batched-inference%2F48020%3E%5C
**Category:** Models\
**Created:** [July 25, 2023, 7:07am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fllama2-pad-token-for-batched-inference%2F48020 "2023-07-25T07:07:06Z")\
**Posts on this page:** 1\
**Showing post:** 3

<div class="post-metadata">

**Author:** ![wztwzt](https://avatars.discourse-cdn.com/v4/letter/w/ea666f/32.png) [@wztwzt](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fwztwzt)%5C
**Post date:** [August 20, 2023, 3:12pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fllama2-pad-token-for-batched-inference%2F48020%2F3 "2023-08-20T15:12:15Z")

</div>

I find the following working very well:

```auto
tokenizer.pad_token = "[PAD]"
tokenizer.padding_side = "left"

```

I used to use what you had, but I found that doing batch inference with that inference gives different results compared to sequential inference, which is not supposed to happen.

---

_[View the full topic](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fllama2-pad-token-for-batched-inference%2F48020)._