# How pretrained models are trained?
**URL:**
**Author:**  [@al31415](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fal31415)%5C
**Post date:** [October 2, 2020, 7:06am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-pretrained-models-are-trained%2F1343%2F1 "2020-10-02T07:06:40Z")
Hello,
How pre-trained are obtained?
In the library there is always some pre-trained model downloaded from the server, but how is it originally trained? Is it also trained using transformers library?
---
**Author:**  [@BramVanroy](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FBramVanroy)%5C
**Post date:** [October 2, 2020, 8:10am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-pretrained-models-are-trained%2F1343%2F2 "2020-10-02T08:10:54Z")
In most if not all built-in cases (e.g. `bert-base-cased`), the original paper implementations are _ported_ to the `transformers` architecture. (you can have a look at conversion scripts, e.g. [https://github.com/huggingface/transformers/blob/77cd0e13d2d09f60d2f6d8fb8b08f493d7ca51fe/src/transformers/convert\_pytorch\_checkpoint\_to\_tf2.py](https://github.com/huggingface/transformers/blob/77cd0e13d2d09f60d2f6d8fb8b08f493d7ca51fe/src/transformers/convert_pytorch_checkpoint_to_tf2.py), [https://github.com/huggingface/transformers/blob/d155b38d6ea70fef3dec2e1f678269e713672bb7/src/transformers/commands/convert.py](https://github.com/huggingface/transformers/blob/d155b38d6ea70fef3dec2e1f678269e713672bb7/src/transformers/commands/convert.py))
User models (e.g. `username/mymodel-uncased`) may have been trained in other ways or ported to the `transformers` architecture manually, with custom scripts, or they are trained by using the `transformers` library directly.
---
**Author:**  [@al31415](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fal31415)%5C
**Post date:** [October 2, 2020, 8:14am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-pretrained-models-are-trained%2F1343%2F3 "2020-10-02T08:14:34Z")
Thanks for reply.
How can I train my own model from scratch using transformers?
---
**Author:**  [@BramVanroy](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FBramVanroy)%5C
**Post date:** [October 2, 2020, 8:30am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-pretrained-models-are-trained%2F1343%2F4 "2020-10-02T08:30:48Z")
You can have a look here: [https://colab.research.google.com/github/huggingface/blog/blob/master/notebooks/01\_how\_to\_train.ipynb](https://colab.research.google.com/github/huggingface/blog/blob/master/notebooks/01_how_to_train.ipynb)