# Loading finetuned model to generate text

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Floading-finetuned-model-to-generate-text%2F892%3E%5C
**Category:** 🤗Transformers\
**Created:** [August 27, 2020, 5:08pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Floading-finetuned-model-to-generate-text%2F892 "2020-08-27T17:08:23Z")\
**Posts on this page:** 1\
**Showing post:** 9

<div class="post-metadata">

**Author:** ![rgwatwormhill](https://avatars.discourse-cdn.com/v4/letter/r/a9adbd/32.png) [@rgwatwormhill](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Frgwatwormhill)%5C
**Post date:** [October 19, 2020, 9:03am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Floading-finetuned-model-to-generate-text%2F892%2F9 "2020-10-19T09:03:45Z")

</div>

Hi @mathor,

what learning-rate did you use for the second lot of training?

Were you using similar training data or very different training data?

Did you save the optimizer state\_dict as well as the model state\_dict?

This post might help: [Checkpoint vs model weight](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcheckpoint-vs-model-weight%2F1434)

If you load your saved (fine-tuned) model, and do a validation check before you start any more training, what kind of validation accuracy do you get?

---

_[View the full topic](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Floading-finetuned-model-to-generate-text%2F892)._