# How to train T5 with Tensorflow
**URL:**
**Author:**  [@patrickvonplaten](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fpatrickvonplaten)%5C
**Post date:** [August 8, 2020, 5:13pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-to-train-t5-with-tensorflow%2F641%2F1 "2020-08-08T17:13:28Z")
There are many examples showing how to train T5 in PyTorch, e.g.: [https://github.com/patil-suraj/exploring-T5/blob/master/T5\_on\_TPU.ipynb](https://github.com/patil-suraj/exploring-T5/blob/master/T5_on_TPU.ipynb) , but none so far for Tensorflow. Many people have been asking on 🤗transformers: [https://github.com/huggingface/transformers/issues/3626](https://github.com/huggingface/transformers/issues/3626) .
Does anybody have a good notebook showing how to train T5 in Tensorflow?
Otherwise, I will try to translate @valhalla 's great notebook: [https://github.com/patil-suraj/exploring-T5/blob/master/T5\_on\_TPU.ipynb](https://github.com/patil-suraj/exploring-T5/blob/master/T5_on_TPU.ipynb) to Tensorflow.
---
**Author:**  [@Jung](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FJung)%5C
**Post date:** [August 9, 2020, 3:17am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-to-train-t5-with-tensorflow%2F641%2F2 "2020-08-09T03:17:57Z")
Thanks very much Patrick @patrickvonplaten!
In Suraj Patil’s notebook, he employed Pytorch `Trainer` to train T5.
At first, I didn’t know that we can use `Trainer` with Seq2Seq problems (according to **“The Big Table of Tasks”** [https://huggingface.co/transformers/examples.html](https://huggingface.co/transformers/examples.html) which stated that `Trainer` does not yet support Translation / Summarization )
I will try to use `TFTrainer` for TF2 on Seq2Seq problems. If that doesn’t work, I think I will try to write custom loop in TF2.
---
**Author:**  [@valhalla](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fvalhalla)%5C
**Post date:** [August 9, 2020, 8:37am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-to-train-t5-with-tensorflow%2F641%2F3 "2020-08-09T08:37:19Z")
You can use trainer for seq2seq as well, you’ll just need to write a different data collator which will return the expected arguments to the model.
Few things to note about the that notebook,
I wrote it before v3.0.0, few things have changed after that
1. `DatCollator` is not a `class` anymore, so you won’t need to inherit from `DataCollator` when creating `T2TDataCollator`. Also `collate_batch` should be renamed to ` __call__ `.
2. `lm_lables` is now deprecated, use `labels` instead.
Let me know if you run into problems using the notebook.
---
**Author:**  [@HarrisDePerceptron](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FHarrisDePerceptron)%5C
**Post date:** [August 29, 2020, 12:10pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-to-train-t5-with-tensorflow%2F641%2F5 "2020-08-29T12:10:14Z")
Any luck on those Tensorflow T5 notebook?
---
**Author:**  [@patrickvonplaten](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fpatrickvonplaten)%5C
**Post date:** [September 1, 2020, 10:29am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-to-train-t5-with-tensorflow%2F641%2F6 "2020-09-01T10:29:30Z")
This might be related: [How to train TFT5ForConditionalGeneration model?](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-to-train-tft5forconditionalgeneration-model%2F888)
Okey, I will start working on a T5 TF notebook showing how T5 can be fine-tuned on CNN / Daily Mail using the TF Trainer this week.
---
**Author:**  [@HarrisDePerceptron](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FHarrisDePerceptron)%5C
**Post date:** [September 5, 2020, 12:49pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-to-train-t5-with-tensorflow%2F641%2F7 "2020-09-05T12:49:33Z")
hey @patrickvonplaten i want to contribute a fully working TF T5 training/finetuning notebook, how do i do that?
---
**Author:**  [@patrickvonplaten](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fpatrickvonplaten)%5C
**Post date:** [September 7, 2020, 6:15am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-to-train-t5-with-tensorflow%2F641%2F8 "2020-09-07T06:15:33Z")
Hey @HarrisDePerceptron,
That sounds awesome! Usually people create a google colab and add it under community notebooks here:
> **[transformers/notebooks at main · huggingface/transformers - Community notebooks:](https://github.com/huggingface/transformers/tree/main/notebooks#community-notebooks)**
>
> main/notebooks
Looking forward to your notebook 🙂
---
**Author:**  [@HarrisDePerceptron](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FHarrisDePerceptron)%5C
**Post date:** [October 2, 2020, 2:57pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-to-train-t5-with-tensorflow%2F641%2F9 "2020-10-02T14:57:38Z")
Hey everyone. We have recently contributed our [community notebook](https://colab.research.google.com/github/snapthat/TF-T5-text-to-text/blob/master/snapthatT5/notebooks/TF-T5-Datasets%20Training.ipynb) that lets us train T5 using pure tensorflow 2. do checkout it out !!! For any issue you can log them to our [offical repo](https://github.com/snapthat/TF-T5-text-to-text). 🤗🤗
---
**Author:**  [@RERobbins](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FRERobbins)%5C
**Post date:** [October 27, 2022, 12:07pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-to-train-t5-with-tensorflow%2F641%2F10 "2022-10-27T12:07:45Z")
Hello everyone.
I am struggling to find a working notebook to illustrate training T5 using keras.
@HarrisDePerceptron your notebook doesn’t seem to run at the moment.
@patrickvonplaten Can you point me to something useful?
This thread seemed to be leading in that direction, but the trail has gone cold.
Thanks!