# Error while training a custom pretrained model
**URL:**
**Author:**  [@Krzysztof](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FKrzysztof)%5C
**Post date:** [July 22, 2021, 1:18pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ferror-while-training-a-custom-pretrained-model%2F8588%2F1 "2021-07-22T13:18:51Z")
Hi,
I trained a model as follows:
checkpoint = “bert-base-uncased”
tokenizer = AutoTokenizer.from\_pretrained(checkpoint)
model = AutoModelForSequenceClassification.from\_pretrained(checkpoint, num\_labels=5)
After 3 epochs I wanted to finetune this model again with the following code (I tried all three versions):
#1
model = BertForSequenceClassification.from\_pretrained(“C:/Users/THINK/Dysk Google/\_Priv/\_Courses/Huggingface/Fine\_tuned\_models/rmp\_eval\_bert\_base\_uncased/”)
#2
model = BertModel.from\_pretrained(‘C:/Users/THINK/Dysk Google/\_Priv/\_Courses/Huggingface/Fine\_tuned\_models/rmp\_eval\_bert\_base\_uncased/’)
#3
config = BertConfig.from\_json\_file(‘C:/Users/THINK/Dysk Google/\_Priv/\_Courses/Huggingface/Fine\_tuned\_models/rmp\_eval\_bert\_base\_uncased/config.json’)
model = BertModel.from\_pretrained(‘C:/Users/THINK/Dysk Google/\_Priv/\_Courses/Huggingface/Fine\_tuned\_models/rmp\_eval\_bert\_base\_uncased/’, config=config)
But in each case I got the error:
TypeError: forward() got an unexpected keyword argument ‘labels’
The input format is the same as before.
The model was saved as:
output\_dir = ‘C:/Users/THINK/Dysk Google/\_Priv/\_Courses/Huggingface/Fine\_tuned\_models/rmp\_diff\_bert\_base\_uncased’
model.save\_pretrained(output\_dir)
Can you help me, what code I should use to load a custom pretrained bert\_base\_uncased model?
Thank you.
---
**Author:**  [@sgugger](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsgugger)%5C
**Post date:** [July 26, 2021, 1:04pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ferror-while-training-a-custom-pretrained-model%2F8588%2F2 "2021-07-26T13:04:37Z")
It’s logical you got that error for the last two lines since `BertModel` does not accept a `labels` argument. I don’t think you had that exact error for the first one as it does accept a `labels` argument.
---
**Author:**  [@Krzysztof](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FKrzysztof)%5C
**Post date:** [July 26, 2021, 2:46pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ferror-while-training-a-custom-pretrained-model%2F8588%2F3 "2021-07-26T14:46:21Z")
After your comment I tried to replicate the earlier error on another computer, and I could not. The additional fine-tuning of custom BertForSequenceClassification model saved earlier was done without any error. So you were right. Thank you.