# When can we expect TPU Trainer?

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fwhen-can-we-expect-tpu-trainer%2F10353%3E%5C
**Category:** 🤗Transformers\
**Created:** [September 28, 2021, 10:09am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fwhen-can-we-expect-tpu-trainer%2F10353 "2021-09-28T10:09:44Z")\
**Posts on this page:** 1\
**Showing post:** 5

<div class="post-metadata">

**Author:** ![sgugger](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/sgugger/32/2291_2.png) [@sgugger](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsgugger)%5C
**Post date:** [December 23, 2021, 4:26pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fwhen-can-we-expect-tpu-trainer%2F10353%2F5 "2021-12-23T16:26:08Z")

</div>

You are not padding your inputs and targets to a fixed size in this example, but dynamically padding them to the longest input/target in each batch. This cause the TPU to recompile at each step, so it’s normal you see a very long training time compared to GPUs.

To properly train on TPU, you need to apply fixed padding in `tokenize_and_align_labels` to a given length of your choice, and pad the labels to that same length.

---

_[View the full topic](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fwhen-can-we-expect-tpu-trainer%2F10353)._