# Load weight from local ckpt file
**URL:**
**Author:**  [@Sniper](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FSniper)%5C
**Post date:** [February 23, 2021, 9:26am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fload-weight-from-local-ckpt-file%2F3854%2F1 "2021-02-23T09:26:20Z")
I have download a standard bert ckpt file ,
,but how can I load this weights into model?
I have read the document:
and try some method like:
`config = BertConfig.from_json_file('./bert_model/bert_config.json')`
`model =TFBertModel.from_pretrained('bert_model/bert_model.ckpt',config=config)`
or
`config = BertConfig.from_json_file('./bert_model/bert_config.json')`
`model = TFBertModel(config).load_weights('bert_model/bert_model.ckpt')`
but it seem like doesn’t work.
---
**Author:**  [@lewtun](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Flewtun)%5C
**Post date:** [February 23, 2021, 10:05am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fload-weight-from-local-ckpt-file%2F3854%2F2 "2021-02-23T10:05:59Z")
Hi @Sniper, I’m not very familiar with the TensorFlow API of `transformers` but I think the following should work:
```auto
config = BertConfig.from_pretrained("path/to/your/bert/directory")
model = TFBertModel.from_pretrained("path/to/your/bert/directory", config=config)
```
If that doesn’t work, can you share the error that you get?
---
**Author:**  [@Sniper](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FSniper)%5C
**Post date:** [February 23, 2021, 10:16am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fload-weight-from-local-ckpt-file%2F3854%2F3 "2021-02-23T10:16:26Z")
> [@lewtun](#):
>
> `from_pretrained`
thanks for your reply.
I still have some question here.
should `path/to/your/bert/directory` is `path/` or `path/model.ckpt`
If I use `path` here, there is an error:
`OSError: Error no file named ['pytorch_model.bin', 'tf_model.h5'] found in directory bert_model/ or `from\_pt` set to False`
If I use `path/model.ckpt` ,there is another error:
`**OSError: Unable to load weights from h5 file. If you tried to load a TF 2.0 model from a PyTorch checkpoint, please set from_pt=True. `
It seem like the code can’t recognize my ckpt file.
code is like this:
`config = BertConfig.from_json_file('./bert_model/bert_config.json')`
`model = TFBertModel.from_pretrained('bert_model/bert_model.ckpt', config=config)`
---
**Author:**  [@lewtun](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Flewtun)%5C
**Post date:** [February 23, 2021, 10:42am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fload-weight-from-local-ckpt-file%2F3854%2F4 "2021-02-23T10:42:31Z")
As far as I know, the `path/to/your/bert/directory` in the `from\_pretrained function should point to the root of the directory where the model / tokenizer files are stored, not individual files.
Can you share the contents of your directory? It seems there might be some files missing. Also, are you trying to load a model that was trained in PyTorch or TensorFlow?
---
**Author:**  [@Sniper](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FSniper)%5C
**Post date:** [February 23, 2021, 10:54am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fload-weight-from-local-ckpt-file%2F3854%2F5 "2021-02-23T10:54:57Z")
Hi,@lewtun
I show my directory in the first paragraph. I can show it again.(I download this file at [GitHub - google-research/bert: TensorFlow code and pre-trained models for BERT](https://github.com/google-research/bert))
It can be load by other framework like [bert4keras](https://github.com/bojone/bert4keras).
In fact,I have a runable model write by bert4keras and I try to rewrite the code to tensorflow2(bert4keras base on tf1).
---
**Author:**  [@lewtun](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Flewtun)%5C
**Post date:** [February 23, 2021, 11:10am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fload-weight-from-local-ckpt-file%2F3854%2F6 "2021-02-23T11:10:28Z")
Thanks for clarification - I see in the [docs](https://huggingface.co/transformers/main_classes/model.html#transformers.PreTrainedModel.from_pretrained) that one can indeed point `from_pretrained` a TF checkpoint file:
> A path or url to a tensorflow index checkpoint file (e.g, `./tf_model/model.ckpt.index` ). In this case, `from_tf` should be set to `True` and a configuration object should be provided as `config` argument. This loading path is slower than converting the TensorFlow checkpoint in a PyTorch model using the provided conversion scripts and loading the PyTorch model afterwards.
It seems that what you’re missing is the `from_tf=True` argument, so maybe something like the following works:
```auto
config = BertConfig.from_pretrained("path/to/your/bert/directory")
model = TFBertModel.from_pretrained("path/to/bert_model.ckpt.index", config=config, from_tf=True)
```
I’m not sure whether the config should be loaded with `from_pretrained` or `from_json_file` but maybe you can test both to see which one works 🙂
---
**Author:**  [@Sniper](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FSniper)%5C
**Post date:** [February 23, 2021, 11:22am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fload-weight-from-local-ckpt-file%2F3854%2F7 "2021-02-23T11:22:35Z")
I add `from_tf` in it.
But it have an error(the version of `transformers` is the lastest one)
`TypeError: ('Keyword argument not understood:', 'from_tf')`
It seem like parameter `from_tf` just use for pytorch,and tensorflow just have parameter `from_pt`.
docs are different in pytorch and tensorflow, tf version of [docs](https://huggingface.co/transformers/main_classes/model.html#transformers.TFPreTrainedModel) just have `from_pt`.
---
**Author:**  [@lewtun](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Flewtun)%5C
**Post date:** [February 23, 2021, 12:51pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fload-weight-from-local-ckpt-file%2F3854%2F8 "2021-02-23T12:51:37Z")
Ah now I understand the source of your `OSError`: it seems that `from_pretrained` expects your model to be serialised as `tf_model.h5` instead of the TensorFlow checkpoint format. You can see this in the source code here: [transformers/modeling\_tf\_utils.py at master · huggingface/transformers · GitHub](https://github.com/huggingface/transformers/blob/master/src/transformers/modeling_tf_utils.py#L1181)
If you’re happy to use the PyTorch API, it seems that it _might_ be possible to load `.ckpt` files with `from_pretrained`: [transformers/modeling\_utils.py at master · huggingface/transformers · GitHub](https://github.com/huggingface/transformers/blob/master/src/transformers/modeling_utils.py#L981)
In that case you could try something like
```auto
config = BertConfig.from_pretrained("path/to/your/bert/directory")
model = BertModel.from_pretrained("path/to/bert_model.ckpt.index", config=config, from_tf=True)
```
---
**Author:**  [@Sniper](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FSniper)%5C
**Post date:** [February 23, 2021, 1:03pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fload-weight-from-local-ckpt-file%2F3854%2F9 "2021-02-23T13:03:21Z")
Maybe I should use pytorch load `.ckpt` and save the model.Then load with tensorflow?
I think I can use albert instead ☹
Thanks a lot for your patience help. Happy Chinese New Year!
---
**Author:**  [@lewtun](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Flewtun)%5C
**Post date:** [February 23, 2021, 1:09pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fload-weight-from-local-ckpt-file%2F3854%2F10 "2021-02-23T13:09:12Z")
> [@Sniper](#):
>
> Maybe I should use pytorch load `.ckpt` and save the model.Then load with tensorflow?
Yes that might work - good luck and happy chinese new year to you too 🙂 !