# Running generate while evaluating test set?

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Frunning-generate-while-evaluating-test-set%2F49927%3E%5C
**Category:** 🤗Transformers\
**Created:** [August 8, 2023, 7:59am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Frunning-generate-while-evaluating-test-set%2F49927 "2023-08-08T07:59:43Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![shivance](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/shivance/32/18366_2.png) [@shivance](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fshivance)%5C
**Post date:** [August 8, 2023, 7:59am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Frunning-generate-while-evaluating-test-set%2F49927%2F1 "2023-08-08T07:59:43Z")

</div>

I’m trying to finetune a T5 model using Huggingface `Trainer` class which does simple evaluation and calculates the loss.

I want to run generations upto certain length at end of each evaluation on test set and store it in a file. I think I would need to override trainer to create a CustomTrainer , but I’m not sure how can I do it.

Could anyone help?