# Using GPT's BPE tokenizer for BERT?

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fusing-gpts-bpe-tokenizer-for-bert%2F9998%3E%5C
**Category:** 🤗Transformers\
**Created:** [September 14, 2021, 9:41pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fusing-gpts-bpe-tokenizer-for-bert%2F9998 "2021-09-14T21:41:38Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![dchang](https://avatars.discourse-cdn.com/v4/letter/d/48db29/32.png) [@dchang](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fdchang)%5C
**Post date:** [September 14, 2021, 9:41pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fusing-gpts-bpe-tokenizer-for-bert%2F9998%2F1 "2021-09-14T21:41:38Z")

</div>

Hello,

I’m training a custom vocab to train a BERT from scratch, and I was wondering if it would make sense to train a GPT-style BPE tokenizer and use a BertModel.

Has anyone done this kind of training with mismatched tokenizer and model types?  
I’d appreciate any insights!