# Original and re-loaded model are not the same

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Foriginal-and-re-loaded-model-are-not-the-same%2F9243%3E%5C
**Category:** Beginners\
**Created:** [August 14, 2021, 8:51pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Foriginal-and-re-loaded-model-are-not-the-same%2F9243 "2021-08-14T20:51:19Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![ThomasG](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/thomasg/32/2445_2.png) [@ThomasG](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FThomasG)%5C
**Post date:** [August 14, 2021, 8:51pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Foriginal-and-re-loaded-model-are-not-the-same%2F9243%2F1 "2021-08-14T20:51:19Z")

</div>

Hi. I fine-tuned the `Wav2Vec2ForCTC` model using Common Voice’s Greek data, using the code below:

> training\_args\_bl3 = TrainingArguments(  
> output\_dir = ‘bl2-cv-HFTrainer-small-lr’,  
> group\_by\_length = True,  
> per\_device\_train\_batch\_size = 16, # batch size  
> gradient\_accumulation\_steps = 2,  
> evaluation\_strategy = ‘steps’,  
> num\_train\_epochs = 60,  
> fp16 = True,  
> save\_steps = 298,  
> eval\_steps = 298,  
> logging\_steps = 298,  
> learning\_rate = 3e-4,  
> warmup\_steps = 180,  
> save\_total\_limit = 1,  
> load\_best\_model\_at\_end = True,  
> metric\_for\_best\_model = ‘wer’,  
> greater\_is\_better = False)
> 
> trainer\_bl3 = Trainer(  
> model = bl2,  
> data\_collator = data\_collator,  
> args = training\_args\_bl3,  
> compute\_metrics = compute\_metrics,  
> train\_dataset = cv\_train,  
> eval\_dataset = cv\_val,  
> tokenizer = processor.feature\_extractor,  
> callbacks = [EarlyStoppingCallback(early\_stopping\_patience = 10)])
> 
> trainer\_bl3.train()

where the model `bl2` is loaded like this:

> bl2 = Wav2Vec2ForCTC.from\_pretrained(  
> ‘facebook/wav2vec2-large-xlsr-53’,  
> attention\_dropout = 0.1,  
> hidden\_dropout = 0.1,  
> feat\_proj\_dropout = 0.0,  
> mask\_time\_prob = 0.05,  
> layerdrop = 0.1,  
> gradient\_checkpointing = True,  
> ctc\_loss\_reduction = ‘mean’,  
> pad\_token\_id = processor.tokenizer.pad\_token\_id,  
> vocab\_size = len(processor.tokenizer),  
> cache\_dir = ‘/mnt/twohdd/.cache’)

After the training is completed, I proceed to save the model like this:

> trainer\_bl3.save\_model(‘FILE\_NAME’)

and then load it as follows

> test = Wav2Vec2ForCTC.from\_pretrained(‘FILE\_NAME’).to(‘cuda’)

After this, and because I noticed that the two models (`bl2`-fine-tuned & `test`-reloaded) yielded different error rates at my test set when I checked, I saw that all their parameters differ. I checked using a function posted in this post [Check if models have same weights - PyTorch Forums](https://discuss.pytorch.org/t/check-if-models-have-same-weights/4351), by aasharma90.

Am I doing something wrong? Can someone please help me, as it is very important to me to be able to load the exact same fine-tuned model at the future.

Thanks in advance.

EDIT: Just wanted to mention that their scores are very close, but not the same. Also I’d like to note that when making the test set prediction, I am setting both models to evaluation mode.

**EDIT2:** I just tried to save my model by doing `bl2.save_pretrained('FILENAME')`, instead of using `Trainer()`'s `save_model()`. After loading with the same code show above (`from_pretrained()`), and comparing the models using the function I linked here, they _match_.

This is good in the sense that now the models match, however I dont understand why this is happening. As far as I know, `Trainer()`'s `save_model()` uses the `save_pretrained()` method. What’s going wrong here?