# Checkpoint vs model weight
**URL:**
**Author:**  [@shainaraza](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fshainaraza)%5C
**Post date:** [October 8, 2020, 8:54pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcheckpoint-vs-model-weight%2F1434%2F1 "2020-10-08T20:54:09Z")
Please make me clear difference between checkpoint and saving the weights of the model,
which one can I use to load later?
Also I could not find my checkpoints (may be overwrite option at my end), so the same can done via these line of code
> trainer.save\_model(“/content/drive//results/distillbert/trainer”)
>
> tokenizer.save\_pretrained(“/content/drive/results/distillbert/tokenizer”)
---
**Author:**  [@rgwatwormhill](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Frgwatwormhill)%5C
**Post date:** [October 12, 2020, 4:06pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcheckpoint-vs-model-weight%2F1434%2F2 "2020-10-12T16:06:26Z")
I think a “checkpoint” is what we call a partial save during training.
To take a checkpoint during training, you can save the model’s state\_dict, which is a list of the current values of all the parameters that have been updated during this training run.
Note that this doesn’t save the non-variable parameters, and it doesn’t save the weights in any frozen layers.
To reload the model to that checkpoint state, you first of all have to load a complete model with the right configuration. You can do this either by initializing randomly with the config file, or by loading a suitable pre-trained model. Then you update that complete model with the saved state\_dict weights.
If you want to continue the training from the same point, you also need information about the scheduler and the optimizer. This can be saved and applied using the optimizer’s state\_dict.
I haven’t any examples of using save\_model or save\_pretrained, but here’s an example of saving a model and optimizer during training.
> filedt = datetime.datetime.now().strftime(“%Y%m%d-%H%M%S”)
> torch.save(model.state\_dict(),‘/content/drive/My Drive/ftregmod-’ + filedt)
> torch.save(optimizer.state\_dict(),‘/content/drive/My Drive/ftregopt-’ + filedt)
and then to reload and continue training:
> READFROMNAMEMODEL = ‘/content/drive/My Drive/ftregmod-20200911-014657’ ####
> READFROMNAMEOPT = ‘/content/drive/My Drive/ftregopt-20200911-014657’ ####
> model = BertForSequenceClassification.from\_pretrained(‘bert-base-uncased’,
> num\_labels=NCLASSES,
> output\_attentions=True)
> model.load\_state\_dict(torch.load(READFROMNAMEMODEL), strict=False)
> optimizer = AdamW(model.parameters(),
> lr = LEARNRATE, # default is 5e-5
> eps = 1e-8 # default is 1e-8.
> )
> optimizer.load\_state\_dict(torch.load(READFROMNAMEOPT))
---
**Author:**  [@shainaraza](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fshainaraza)%5C
**Post date:** [October 12, 2020, 6:53pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcheckpoint-vs-model-weight%2F1434%2F3 "2020-10-12T18:53:29Z")
thanks @rgwatwormhill