# How can I know what loss function I am using?

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-can-i-know-what-loss-function-i-am-using%2F156655%3E%5C
**Category:** Beginners\
**Created:** [May 24, 2025, 10:18pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-can-i-know-what-loss-function-i-am-using%2F156655 "2025-05-24T22:18:57Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![joaorr13](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/joaorr13/32/44878_2.png) [@joaorr13](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fjoaorr13)%5C
**Post date:** [May 24, 2025, 10:18pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-can-i-know-what-loss-function-i-am-using%2F156655%2F1 "2025-05-24T22:18:57Z")

</div>

Hello,  
I am finetunning my first model and I have to write a report on that, but I am having a little trouble finding what my loss function is.  
This is my setup for training:  
from transformers import DataCollatorForLanguageModeling

data\_collator = DataCollatorForLanguageModeling(tokenizer=tokenizer, mlm=False, seed=42)  
from transformers import EarlyStoppingCallback

early\_stopping\_callback = EarlyStoppingCallback(early\_stopping\_patience=3)

trainer = SFTTrainer(  
model=model,  
tokenizer=tokenizer,  
train\_dataset=train\_tokenized\_dataset,  
eval\_dataset=valid\_tokenized\_dataset,  
data\_collator=data\_collator,  
callbacks=[debug\_callback, early\_stopping\_callback],  
args=SFTConfig(  
output\_dir=“./output”,  
eval\_strategy=“epoch”,  
save\_strategy=“epoch”,  
learning\_rate=2e-4,  
num\_train\_epochs=200,  
per\_device\_train\_batch\_size=1800,  
per\_device\_eval\_batch\_size=1800,  
metric\_for\_best\_model=“eval\_loss”,  
bf16=True,  
),  
)

How can I get my loss function expression?

---

<div class="post-metadata">

**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FJohn6666)%5C
**Post date:** [May 25, 2025, 12:21am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-can-i-know-what-loss-function-i-am-using%2F156655%2F2 "2025-05-25T00:21:03Z")

</div>

Hmm… Maybe this?

> [@SFTTrainer Loss function](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fsfttrainer-loss-function%2F80549%2F3)%3A
>
> The loss function being used is the cross-entropy loss. It is defined within the model, e.g. [here](https://github.com/huggingface/transformers/blob/1082361a1978d30db5c3932d1ee08914d74d9697/src/transformers/models/llama/modeling_llama.py#L1202) for llama. In case you want to use your own custom loss function, you can overwrite the compute\_loss method of the Trainer as explained [here](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffinetuning-bart-using-custom-loss%2F4060%2F2). For training an LLM the loss function is computed on the whole concatenated prompts, how to alter this and make loss function only compute on the output prompts Indeed as recommended above, the [DataCollatorForCompletionOnlyLM](https://huggingface.co/docs/trl/en/sft_trainer#train-on-completions-only) can be used for this purpose…

[https://stackoverflow.com/questions/71581197/what-is-the-loss-function-used-in-trainer-from-the-transformers-library-of-huggi](https://stackoverflow.com/questions/71581197/what-is-the-loss-function-used-in-trainer-from-the-transformers-library-of-huggi)  
[https://stackoverflow.com/questions/72350835/how-to-plot-loss-when-using-hugginfaces-trainer](https://stackoverflow.com/questions/72350835/how-to-plot-loss-when-using-hugginfaces-trainer)

> **[Supervised Fine-tuning Trainer](https://huggingface.co/docs/trl/en/sft_trainer)**
>
> We’re on a journey to advance and democratize artificial intelligence through open source and open science.