# Is it possible to train a new head?

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fis-it-possible-to-train-a-new-head%2F14197%3E%5C
**Category:** Beginners\
**Created:** [January 28, 2022, 5:51pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fis-it-possible-to-train-a-new-head%2F14197 "2022-01-28T17:51:37Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![JakeMSc](https://avatars.discourse-cdn.com/v4/letter/j/ecc23a/32.png) [@JakeMSc](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FJakeMSc)%5C
**Post date:** [January 28, 2022, 5:51pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fis-it-possible-to-train-a-new-head%2F14197%2F1 "2022-01-28T17:51:37Z")

</div>

I have two data sets, A and B. They both consist of sentences and named entity labels.

I’ve fine tuned distilbert for named entity recognition on data set A.

I would like to retain the encoding layer of this model, but remove the head that performs the named entity recognition.

I would then like to attach a new head for named entity recognition, and fine tune this on data set B.

So ultimately the head is trained only on data set B, but the encoding layer is trained on both data set A and B.

Is this possible? How can this be done?