# Purpose of padding and truncating
**URL:**
**Author:**  [@aclifton314](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Faclifton314)%5C
**Post date:** [July 21, 2020, 3:03pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fpurpose-of-padding-and-truncating%2F412%2F1 "2020-07-21T15:03:33Z")
I have read the [Preprocessing Data page](https://huggingface.co/transformers/preprocessing.html#base-use). I understand what padding and truncating are doing, but I’m not sure I understand the reason for doing either of them. Can anyone help me understand the purpose for doing them? Thanks in advance!
---
**Author:**  [@valhalla](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fvalhalla)%5C
**Post date:** [July 21, 2020, 3:24pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fpurpose-of-padding-and-truncating%2F412%2F2 "2020-07-21T15:24:28Z")
Hi @aclifton314,
padding :
Padding is used to make all examples same length so that you can pack them in batch, sequences with uneven length can’t be batched. So if a sequence is shorter, than your max length then padding is used to make that sequence longer. Also some model might expect fixed length input, so padding help there too.
truncation:
Most of the models have max\_lengths defined for them (there are exceptions, model with relative attention can take arbitrarily long sequences) for ex.for BERT max\_length is 512, so if one of your sequence is longer than that you can’t feed it directly, so you need to truncate (drop extra tokens) to make the sequence smaller.
Hope this helps 🙂
---
**Author:**  [@aclifton314](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Faclifton314)%5C
**Post date:** [July 21, 2020, 3:27pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fpurpose-of-padding-and-truncating%2F412%2F3 "2020-07-21T15:27:11Z")
@valhalla crystal clear. Thank you very much!
---
**Author:**  [@kasar3](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fkasar3)%5C
**Post date:** [August 3, 2020, 12:23pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fpurpose-of-padding-and-truncating%2F412%2F4 "2020-08-03T12:23:42Z")
So if I understand right, the sentence that is too long will see some of its tokens deleted, without using these extra tokens to train the future model?
Does these tokens are randomly selected or if the max\_lenght=100 then tokens \> 100 will be deleted?
Thank you 🙂
---
**Author:**  [@valhalla](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fvalhalla)%5C
**Post date:** [August 3, 2020, 12:44pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fpurpose-of-padding-and-truncating%2F412%2F5 "2020-08-03T12:44:48Z")
Hi @kasar3, if you enable `truncation` then the extra tokens will be deleted , they are not selected randomly. This [guide](https://huggingface.co/transformers/preprocessing.html#everything-you-always-wanted-to-know-about-padding-and-truncation) explains padding and truncation in detail.
---
**Author:**  [@kasar3](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fkasar3)%5C
**Post date:** [August 3, 2020, 1:25pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fpurpose-of-padding-and-truncating%2F412%2F6 "2020-08-03T13:25:41Z")
Alright thank you. 👍
I have an other question, don’t know of it’s the right place… For example, in the case of text classification using BERT model (512 tokens max): let’s say that I have a sentence of 1000 tokens, what should I do to tokenize it like BERT want and still being able to classife it later. Should I split it manually in sentences of length \<512 ?
For exemple split it in 2 sentences of 500 tokens while duplicate the text classification label of the original sentence?
---
**Author:**  [@valhalla](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fvalhalla)%5C
**Post date:** [August 3, 2020, 4:50pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fpurpose-of-padding-and-truncating%2F412%2F7 "2020-08-03T16:50:24Z")
I don’t have a good answer for this, but I have seen people breaking longer documents into chunks, getting the label for each chunk and then aggregate it.
Second approach is, now with recent models like longformer it’s possible to train models with large sequence length so that you won’t need to chunk the document.
@sgugger might have a better answer.
---
**Author:**  [@sgugger](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsgugger)%5C
**Post date:** [August 3, 2020, 5:01pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fpurpose-of-padding-and-truncating%2F412%2F8 "2020-08-03T17:01:43Z")
More than the labels, I’d get the pooler output of each chunk and then average them before using the classifier. This will require you to change the code of `BertForSequenceClassification` a little bit.