# How to modify loss function in a seq2seq trainer?

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-to-modify-loss-function-in-a-seq2seq-trainer%2F104680%3E%5C
**Category:** 🤗Transformers\
**Created:** [August 30, 2024, 3:15pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-to-modify-loss-function-in-a-seq2seq-trainer%2F104680 "2024-08-30T15:15:16Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![dmehta01](https://avatars.discourse-cdn.com/v4/letter/d/a6a055/32.png) [@dmehta01](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fdmehta01)%5C
**Post date:** [August 30, 2024, 3:15pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-to-modify-loss-function-in-a-seq2seq-trainer%2F104680%2F1 "2024-08-30T15:15:16Z")

</div>

I am trying to fine tune a whisper model using this source: [Fine-Tune Whisper For Multilingual ASR with 🤗 Transformers](https://huggingface.co/blog/fine-tune-whisper)

I want to modify the loss function used to fine tune it. For example, I would like to modify the loss function to be able to distill knowledge from another ASR model.

So how do I modify the loss function and how would I do the knowledge distillation part as well?

@sanchit-gandhi Would appreciate it if you can help!

---

<div class="post-metadata">

**Author:** ![nielsr](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/nielsr/32/39617_2.png) [@nielsr](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fnielsr)%5C
**Post date:** [August 31, 2024, 8:36am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-to-modify-loss-function-in-a-seq2seq-trainer%2F104680%2F2 "2024-08-31T08:36:57Z")

</div>

Hi,

You can overwrite the Seq2SeqTrainer’s compute\_loss method as shown [here](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fspecify-loss-for-trainer-trainingarguments%2F10481%2F2).