# Struggling with fine tuning bert
**URL:**
**Author:**  [@mcaizpp2](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fmcaizpp2)%5C
**Post date:** [August 5, 2025, 6:22pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fstruggling-with-fine-tuning-bert%2F165766%2F1 "2025-08-05T18:22:50Z")
Hi all
im using bert for question and answering (reading from pdf) and want to fine tune its answers. The problem is the code i have isnt really fine tuninig the model.
Im trying to further fine tune a model (bert with squad) with a small dataset (for now)
If i type in what is the reinsured address? It returns the name of the reinsured not the address below it.
I want to tell bert that that answer is incorrect but i dont kn ow how to proceed. I tried feeding a question answer pair with startid as -1 so that it would “know” the reinsured name was not the correct answer.
But i feel no matter what i do the model is just not “learning”.
Im basically trying to create a question and answer model for reading and answering questions about insurance contracts.
Does anyone know of any good tutorial, videos, methods to achieve my goal?
Thank you
---
**Author:**  [@John6666](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FJohn6666)%5C
**Post date:** [August 5, 2025, 11:18pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fstruggling-with-fine-tuning-bert%2F165766%2F2 "2025-08-05T23:18:39Z")
> But i feel no matter what i do the model is just not “learning”.
If there is no change in the model output, it may be because [the model layers themselves are _frozen_ and prevented from learning](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fwhen-fine-tuning-a-object-detection-model-which-parameters-do-we-update%2F162494%2F2)%2C or because [the dataset label names are not in the expected format, effectively rendering the dataset “empty”](https://github.com/huggingface/datasets/issues/2585). In this case, it is probably the latter.
[For BERT fine-tuning, how about this video](https://huggingface.co/learn/llm-course/en/chapter7/7) for now?
---
**Author:**  [@mcaizpp2](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fmcaizpp2)%5C
**Post date:** [August 6, 2025, 2:57pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fstruggling-with-fine-tuning-bert%2F165766%2F3 "2025-08-06T14:57:49Z")
Thanks going to try this now
**But i feel no matter what i do the model is just not “learning”.**
Apologies I got a little frustrated when I wrote this post 🙂