# T5/mT5 model distillation

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ft5-mt5-model-distillation%2F30550%3E%5C
**Category:** 🤗Transformers\
**Created:** [January 27, 2023, 7:31am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ft5-mt5-model-distillation%2F30550 "2023-01-27T07:31:47Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![shivaays](https://avatars.discourse-cdn.com/v4/letter/s/53a042/32.png) [@shivaays](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fshivaays)%5C
**Post date:** [January 27, 2023, 7:31am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ft5-mt5-model-distillation%2F30550%2F1 "2023-01-27T07:31:47Z")

</div>

Hi Everyone,

I am trying to distill my T5 model. I am planning to use this script: [transformers/distillation.py at main · huggingface/transformers · GitHub](https://github.com/huggingface/transformers/blob/main/examples/research_projects/seq2seq-distillation/distillation.py)

Can anyone please guide me how to setup my codebase for this process? If anyone have any other better solution please guide me.

---

<div class="post-metadata">

**Author:** ![cyenjoylife](https://avatars.discourse-cdn.com/v4/letter/c/a88e4f/32.png) [@cyenjoylife](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fcyenjoylife)%5C
**Post date:** [December 25, 2023, 6:19am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ft5-mt5-model-distillation%2F30550%2F2 "2023-12-25T06:19:43Z")

</div>

I am also trying to distill from T5-xxl. What teacher model did you choose to use?