# GPT-2 custom loss

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fgpt-2-custom-loss%2F20543%3E%5C
**Category:** Models\
**Created:** [July 18, 2022, 11:41pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fgpt-2-custom-loss%2F20543 "2022-07-18T23:41:39Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![adibm](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/adibm/32/5390_2.png) [@adibm](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fadibm)%5C
**Post date:** [July 18, 2022, 11:41pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fgpt-2-custom-loss%2F20543%2F1 "2022-07-18T23:41:39Z")

</div>

I am using a GPT-2 model to do generate text. During training I feed in the model a concatenation of context+target and during inference, I pass in the context, and the model predicts context+target. In training, I would like to modify the loss function to calculate the loss between the logits and the target only, i dont want to have the loss for the context predictions. Any tips on how I could do this? It would be great if you could point me to some resources that might help me do this.