# Padding side in instruction fine-tuning using SFTT

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fpadding-side-in-instruction-fine-tuning-using-sftt%2F113549%3E%5C
**Category:** 🤗Transformers\
**Created:** [October 23, 2024, 9:48am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fpadding-side-in-instruction-fine-tuning-using-sftt%2F113549 "2024-10-23T09:48:08Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![dkoterwa](https://avatars.discourse-cdn.com/v4/letter/d/e9c0ed/32.png) [@dkoterwa](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fdkoterwa)%5C
**Post date:** [October 23, 2024, 9:48am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fpadding-side-in-instruction-fine-tuning-using-sftt%2F113549%2F1 "2024-10-23T09:48:08Z")

</div>

I am a little bit confused about the correct padding side that should be utilized while instruction fine-tuning Large Language Models.

Supervised Fine-tuning Trainer comes from tlr library. Most of the tutorials concerning instruction fine-tuning (e.g. a tutorial provided by the Technical Lead at HF - Philipp Schmid: [LINK](https://www.philschmid.de/fine-tune-llms-in-2024-with-trl)) attach a line of code with the change of padding side:

```auto
tokenizer.padding_side = 'right'

```

When this is done, then we get a warning message from transformers library:

```auto
A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer.

```

Let’s assume that I am a polite student listetning to the warnings and I change the padding side to left, but after that another warning is thrown by tlr library:

```auto
UserWarning: You passed a tokenizer with `padding_side` not equal to `right` to the SFTTrainer. This might lead to some unexpected behaviour due to overflow issues when training a model in half-precision. You might consider adding `tokenizer.padding_side = 'right'` to your code.

```

This is super confusing to me because you basically cannot fine-tune without any warning, there is always something telling you that you should initialize the tokenizer in different way.

So my question is: what is the desired padding side for instruction fine-tuning and what are the use cases of left and right padding? I found multiple discussions on the Internet, however, the conclusions are contradictory (some state right padding, the other ones left padding). Even the HuggingFace website tells you to use left padding ([generation](https://huggingface.co/docs/transformers/llm_tutorial#wrong-padding-side)), so why a person from HF preparing a tutorial used the right padding instead?

---

<div class="post-metadata">

**Author:** ![cgr71ii](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/cgr71ii/32/26351_2.png) [@cgr71ii](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fcgr71ii)%5C
**Post date:** [December 9, 2024, 1:21pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fpadding-side-in-instruction-fine-tuning-using-sftt%2F113549%2F2 "2024-12-09T13:21:20Z")

</div>

I found a clear answer: “the padding side should be `left` when generating and `right` when training/tuning” ([setting of padding\_side in Llama tokenizers · Issue #34842 · huggingface/transformers · GitHub](https://github.com/huggingface/transformers/issues/34842#issuecomment-2490994584))

However, I asked for further explanations because I don’t totally understand why different padding side should be applied for training and inference ([setting of padding\_side in Llama tokenizers · Issue #34842 · huggingface/transformers · GitHub](https://github.com/huggingface/transformers/issues/34842#issuecomment-2527910988)).