# Fine-tuning a locally saved model on NER task

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffine-tuning-a-locally-saved-model-on-ner-task%2F8325%3E%5C
**Category:** 🤗Transformers\
**Created:** [July 13, 2021, 7:10pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffine-tuning-a-locally-saved-model-on-ner-task%2F8325 "2021-07-13T19:10:10Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![HosseinK](https://avatars.discourse-cdn.com/v4/letter/h/a3d4f5/32.png) [@HosseinK](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FHosseinK)%5C
**Post date:** [July 13, 2021, 7:10pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffine-tuning-a-locally-saved-model-on-ner-task%2F8325%2F1 "2021-07-13T19:10:10Z")

</div>

A few days ago I further prep-trained nlpaueb/legal-bert-base-uncased ([nlpaueb/legal-bert-base-uncased · Hugging Face](https://huggingface.co/nlpaueb/legal-bert-base-uncased)) model for masked token prediction task on a custom dataset using run\_mlm.py. After training being done, I saved the model in a local directory. I can see pytorch\_model.bin, config.json, all other required files in this directory.

Now I want to fine-tune this saved model on NER task using run\_ner.py (in transformers 4.6.0) with another custom training data using the following command

python3 run\_ner.py --model\_name\_or\_path path-to-the-saved-model --train\_file path-to-the-json-training-data --validation\_file path-to-the-json-validation-data --do\_train t --do\_eval t --output\_dir path-to-output-dir --overwrite\_output\_dir t --label\_all\_tokens t --return\_entity\_level\_metrics t &\> log-file.txt

log file shows the following error

Some weights of BertForTokenClassification were not initialized from the model checkpoint at /home/ubuntu/trained-model/legalbert\_further\_trained/ and are newly initialized: [‘classifier.bias’, ‘classifier.weight’]  
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.  
07/13/2021 18:17:44 - WARNING - datasets.arrow\_dataset - Loading cached processed dataset at /home/ubuntu/.cache/huggingface/datasets/json/default-e94e893514cb3fcb/0.0.0/83d5b3a2f62630efc6b5315f00f20209b4ad91a00ac586597caee3a4da0bef02/cache-361f604016d1c665.arrow  
07/13/2021 18:17:44 - WARNING - datasets.arrow\_dataset - Loading cached processed dataset at /home/ubuntu/.cache/huggingface/datasets/json/default-e94e893514cb3fcb/0.0.0/83d5b3a2f62630efc6b5315f00f20209b4ad91a00ac586597caee3a4da0bef02/cache-a77e5011b7db5bd5.arrow  
07/13/2021 18:17:44 - WARNING - datasets.arrow\_dataset - Loading cached processed dataset at /home/ubuntu/.cache/huggingface/datasets/json/default-e94e893514cb3fcb/0.0.0/83d5b3a2f62630efc6b5315f00f20209b4ad91a00ac586597caee3a4da0bef02/cache-5659054e40c94504.arrow  
07/13/2021 18:17:44 - WARNING - datasets.load - Using the latest cached version of the module from /home/ubuntu/.cache/huggingface/modules/datasets\_modules/metrics/seqeval/1fde2544ac1f3f7e54c639c73221d3a5e5377d2213b9b0fdb579b96980b84b2e (last modified on Fri Jul 2 16:39:09 2021) since it couldn’t be found locally at seqeval/seqeval.py or remotely (ConnectionError).  
[INFO|trainer.py:1047] 2021-07-13 18:17:48,569 \>\> Loading model from /home/ubuntu/trained-model/legalbert\_further\_trained/).  
Traceback (most recent call last):  
File “run\_ner.py”, line 533, in   
main()  
File “run\_ner.py”, line 478, in main  
train\_result = trainer.train(resume\_from\_checkpoint=checkpoint)  
File “/home/ubuntu/.local/lib/python3.6/site-packages/transformers/trainer.py”, line 1066, in train  
self.\_load\_state\_dict\_in\_model(state\_dict)  
File “/home/ubuntu/.local/lib/python3.6/site-packages/transformers/trainer.py”, line 1387, in \_load\_state\_dict\_in\_model  
if set(load\_result.missing\_keys) == set(self.model.\_keys\_to\_ignore\_on\_save):  
TypeError: ‘NoneType’ object is not iterable.

Note that if I replace [–model\_name\_or\_path path-to-the-saved-model] with [–model\_name\_or\_path nlpaueb/legalbert-base-uncased] (or any other existing language model that I haven’t further pre-trained), run\_ner.py runs without any issue.

---

<div class="post-metadata">

**Author:** ![DarshanAdiga](https://avatars.discourse-cdn.com/v4/letter/d/f475e1/32.png) [@DarshanAdiga](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FDarshanAdiga)%5C
**Post date:** [July 20, 2022, 10:56pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffine-tuning-a-locally-saved-model-on-ner-task%2F8325%2F2 "2022-07-20T22:56:26Z")

</div>

Hey, I am in a similar situation: Trying to fine-tune a BERT model that has been saved locally with additional tokens added to its tokenizer vocabulary. I am getting the same error as you have described.  
Was anyone able to identify the root cause of this issue?

---

<div class="post-metadata">

**Author:** ![DarshanAdiga](https://avatars.discourse-cdn.com/v4/letter/d/f475e1/32.png) [@DarshanAdiga](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FDarshanAdiga)%5C
**Post date:** [July 21, 2022, 12:31pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffine-tuning-a-locally-saved-model-on-ner-task%2F8325%2F3 "2022-07-21T12:31:17Z")

</div>

In case anyone else is facing a similar issue: I found that I was adding new tokens to a BERT model using AutoModel and then trying to fine-tune using AutoModelForSequenceClassification. So, the fix was to load the BERT model as a Sequence Classification model itself using AutoModelForSequenceClassification when adding new tokens to the tokenizer.