# Fine-tuned pre-trained Roberta model on different labels

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffine-tuned-pre-trained-roberta-model-on-different-labels%2F16570%3E%5C
**Category:** 🤗Transformers\
**Created:** [April 7, 2022, 5:53am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffine-tuned-pre-trained-roberta-model-on-different-labels%2F16570 "2022-04-07T05:53:30Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![WJ210](https://avatars.discourse-cdn.com/v4/letter/w/bb73d2/32.png) [@WJ210](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FWJ210)%5C
**Post date:** [April 7, 2022, 5:53am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffine-tuned-pre-trained-roberta-model-on-different-labels%2F16570%2F1 "2022-04-07T05:53:30Z")

</div>

Hi,

I was wondering if its possible to save the model parameters of a fine-tuned roberta model on sentiment analysis of 2 classes and then load it into another model to solve the task of text classification of different number of classes, example 4.

Example :  
if i trained RobertaForSequenceClassification.from\_pretrained(‘roberta-base’, num\_labels = 2).  
I would like to save only the base model without the classification head and create a new class which takes in the saved model and insert a classification head of output = 4, while freezing the base model and only fine-tuning the head for the new task.

Would that work? and how would i go above saving and loading only the base model?