# Unable to load TF2-checkpoints into Huggingface

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Funable-to-load-tf2-checkpoints-into-huggingface%2F257%3E%5C
**Category:** 🤗Transformers\
**Created:** [July 13, 2020, 6:34pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Funable-to-load-tf2-checkpoints-into-huggingface%2F257 "2020-07-13T18:34:13Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![pere](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/pere/32/40613_2.png) [@pere](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fpere)%5C
**Post date:** [July 13, 2020, 6:34pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Funable-to-load-tf2-checkpoints-into-huggingface%2F257%2F1 "2020-07-13T18:34:13Z")

</div>

I am unable to import a pretrained BERT checkpoints trained with Tensorflow 2.

I get the error “AttributeError: ‘BertForPreTraining’ object has no attribute ‘shape’”. I have tried both the “transformers-cli convert” and the “.from\_pretrained”, basically stopping at the same issue.

It also affects the official BERT models. I have no issues loading the official BERT-model trained for TF1.x, however trying to load the official TF2.0/Keras-model is giving me the same error.

This Colab Gist downloads the three official models hosted at gs://cloud-tpu-checkpoints and tries to load them into Huggingface:  
[https://colab.research.google.com/gist/peregilk/6b97384d3443ca9354a1898efd9819af/loading-tf-2-0-checkpoints-in-huggingface.ipynb](https://colab.research.google.com/gist/peregilk/6b97384d3443ca9354a1898efd9819af/loading-tf-2-0-checkpoints-in-huggingface.ipynb)

---

<div class="post-metadata">

**Author:** ![Shivangi05](https://avatars.discourse-cdn.com/v4/letter/s/b5a626/32.png) [@Shivangi05](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FShivangi05)%5C
**Post date:** [November 9, 2022, 4:31pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Funable-to-load-tf2-checkpoints-into-huggingface%2F257%2F2 "2022-11-09T16:31:45Z")

</div>

Hey Pere,

Were you able to solve this issue? I have the a similar error but with bias. That is the error says “AttributeError: ‘BertForPreTraining’ object has no attribute ‘bias’”.

---

<div class="post-metadata">

**Author:** ![pere](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/pere/32/40613_2.png) [@pere](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fpere)%5C
**Post date:** [November 9, 2022, 5:00pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Funable-to-load-tf2-checkpoints-into-huggingface%2F257%2F3 "2022-11-09T17:00:21Z")

</div>

Long ago. From what I remembered I gave up on this. One of the reasons I switched to using PyTorch and Flax.