# Loading finetuned model to generate text
**URL:**
**Author:**  [@aclifton314](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Faclifton314)%5C
**Post date:** [August 27, 2020, 5:08pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Floading-finetuned-model-to-generate-text%2F892%2F1 "2020-08-27T17:08:24Z")
This is a basic question. Suppose I finetune a pretrained GPT2 model on my particular corpus. How do I go about loading that new model and generating some text from that new model?
---
**Author:**  [@valhalla](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fvalhalla)%5C
**Post date:** [August 28, 2020, 3:31pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Floading-finetuned-model-to-generate-text%2F892%2F2 "2020-08-28T15:31:08Z")
Great [tutorial](https://huggingface.co/blog/how-to-generate) on `generate` method which lets you do greedy decoding, sampling, beam search. It has all the details you are looking for
---
**Author:**  [@aclifton314](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Faclifton314)%5C
**Post date:** [August 29, 2020, 8:51pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Floading-finetuned-model-to-generate-text%2F892%2F3 "2020-08-29T20:51:27Z")
@valhalla Yes that tutorial is fantastic. Thank you for linking it.
My finetuned model inherits from `GPT2LMHeadModel`. I wanted to check if the best way to load my finetuned model was via `from_pretrained()` and if my model will be able to generate text from the new corpus it eas finetuned on.
---
**Author:**  [@rgwatwormhill](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Frgwatwormhill)%5C
**Post date:** [August 30, 2020, 11:34am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Floading-finetuned-model-to-generate-text%2F892%2F4 "2020-08-30T11:34:54Z")
If your finetuned model is just tuned (and not extended with extra layers) then save\_pretrained and from\_pretrained should work, and may be the easiest.
If you have altered your model (with extra layers) then I think you need to use torch.save and torch.load, and redefine your model configuration before loading, eg
torch.save(model.state\_dict(),‘filepath’)
model = ATBertClass() # redefine using your custom model definition
model.load\_state\_dict(torch.load(‘filepath’))
---
**Author:**  [@aclifton314](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Faclifton314)%5C
**Post date:** [September 18, 2020, 7:17am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Floading-finetuned-model-to-generate-text%2F892%2F5 "2020-09-18T07:17:51Z")
@rgwatwormhill, I just fine tuned the model and did not alter it.
If the name of that model class was `MyGPT2Model` and I saved it using `Trainer`, to load it would simply be the line `MyGPT2Model.from_pretrained('/path/to/model/')`?
And since it inherits from `GPT2LMHeadModel` I can call the `generate` method to generate text like:
```python
import MyGPT2Model
from transformers import GPT2Tokenizer
prompt = 'Huggingface Transformers is fantastic!'
tokenizer = GPT2Tokenizer.from_pretrained('gpt2')
encoded_results = tokenizer(prompt, return_tensors='pt')
tok_ids = encoded_results['input_ids']
my_model = MyGPT2Model.from_pretrained('/path/to/model/')
gen_ids = my_model.generate(tok_ids)
decoded_str = tokenizer.decode(gen_ids)
```
Is this correct?
---
**Author:**  [@rgwatwormhill](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Frgwatwormhill)%5C
**Post date:** [September 18, 2020, 10:04am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Floading-finetuned-model-to-generate-text%2F892%2F6 "2020-09-18T10:04:28Z")
Hi aclifton314,
Short answer: I’ve no idea - what happens if you try that?
I’ve not tried using Trainer or GPT2 (I’ve been using Bert with native pytorch commands), so I could easily be wrong here. Please ignore anything where you know better.
In the Trainer documentation, it looks like from\_pretrained is the correct command to use.
Did you set output\_dir in your Trainer training arguments? Have you checked that your trained model exists in your /path/to/model/ directory?
I don’t think you can import your custom model class (I think you will need to include your code to define it).
If you are not altering the shape of the model, do you need a custom model class?
I think you might need to import GPT2LMHeadModel, whether you use it as-is or by inheriting.
I think you might need to use tokenizer.tokenize() or tokenizer.encode\_plus() or something like that, rather than just tokenizer()
---
**Author:**  [@aclifton314](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Faclifton314)%5C
**Post date:** [September 21, 2020, 6:30pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Floading-finetuned-model-to-generate-text%2F892%2F7 "2020-09-21T18:30:51Z")
> Short answer: I’ve no idea - what happens if you try that?
It looks like it’s working!
> Did you set output\_dir in your Trainer training arguments? Have you checked that your trained model exists in your /path/to/model/ directory?
Yes I did. The model does exist in the output directory set in the training arguments.
---
**Author:**  [@mathor](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fmathor)%5C
**Post date:** [October 17, 2020, 4:45am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Floading-finetuned-model-to-generate-text%2F892%2F8 "2020-10-17T04:45:48Z")
thank you for your answer, but i have a question for you
I validate the model as I train it, and save the model with the highest scores on the validation set using `torch.save(model.state_dict(), output_model_file)`. As shown in the figure below
Then I trained again and loaded the previously saved model instead of training from scratch, but it didn’t work well, which made me feel like it wasn’t saved or loaded successfully?
`model.load_state_dict(torch.load(output_model_file))`
My model code is [here](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-to-save-and-load-fine-tune-model%2F1595)
---
**Author:**  [@rgwatwormhill](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Frgwatwormhill)%5C
**Post date:** [October 19, 2020, 9:03am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Floading-finetuned-model-to-generate-text%2F892%2F9 "2020-10-19T09:03:45Z")
Hi @mathor,
what learning-rate did you use for the second lot of training?
Were you using similar training data or very different training data?
Did you save the optimizer state\_dict as well as the model state\_dict?
This post might help: [Checkpoint vs model weight](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcheckpoint-vs-model-weight%2F1434)
If you load your saved (fine-tuned) model, and do a validation check before you start any more training, what kind of validation accuracy do you get?
---
**Author:**  [@Pranavagrl](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FPranavagrl)%5C
**Post date:** [July 25, 2023, 1:18pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Floading-finetuned-model-to-generate-text%2F892%2F10 "2023-07-25T13:18:58Z")
Hey aclifton314, I am a beginner I was trying to fine-tune model to generate text i will be the great help if you will be able to provide your notebook to me
---
**Author:**  [@aclifton314](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Faclifton314)%5C
**Post date:** [July 25, 2023, 4:45pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Floading-finetuned-model-to-generate-text%2F892%2F11 "2023-07-25T16:45:00Z")
@Pranavagrl welcome to the HF Forums!
I’m happy to help out in any way I can. If you wouldn’t mind providing some more details, it would help me out to walk you through finetuning.
1. Are you wanting to use a model to generate text over a particular corpus that you have?
2. Which model would you prefer to use?
3. Have you any previous experience (either with coding or learning) with machine learning and/or NLP?
---
**Author:**  [@Pranavagrl](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FPranavagrl)%5C
**Post date:** [July 26, 2023, 7:29am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Floading-finetuned-model-to-generate-text%2F892%2F12 "2023-07-26T07:29:37Z")
Hello @aclitfton314
Yeah, I have a basic understanding of that.
1. Are you wanting to use a model to generate text over a particular corpus that you have? ==\> No
2. Which model would you prefer to use? ==\> Any Model Using This Architecture: LlamaForCausalLM
---
**Author:**  [@aclifton314](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Faclifton314)%5C
**Post date:** [August 7, 2023, 5:27pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Floading-finetuned-model-to-generate-text%2F892%2F13 "2023-08-07T17:27:11Z")
Hi @Pranavagrl,
Do you mind describing in more detail what you’re interested in doing?