# Custom heads with Trainer API

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcustom-heads-with-trainer-api%2F16719%3E%5C
**Category:** Beginners\
**Created:** [April 12, 2022, 9:03am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcustom-heads-with-trainer-api%2F16719 "2022-04-12T09:03:49Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![Kioto97](https://avatars.discourse-cdn.com/v4/letter/k/c77e96/32.png) [@Kioto97](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FKioto97)%5C
**Post date:** [April 12, 2022, 9:03am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcustom-heads-with-trainer-api%2F16719%2F1 "2022-04-12T09:03:49Z")

</div>

Hi, I’ve been using this guide for fine-tuning a BERT-based model with the Trainer API:  
[Guide](https://huggingface.co/docs/transformers/training#finetuning-in-pytorch-with-the-trainer-api)  
My doubt is: Can I build a custom head on top of a model built with Trainer? For example, if I want to build my head with several layers for SQuAD, can I do it with Trainer? Or am I bound to use only the pre-existing heads? If it’s possible, do you have a snippet of code that shows how to do it? I couldn’t find any example.  
Thanks.