# Train wordpiece from scratch
**URL:**
**Author:**  [@Kamel](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FKamel)%5C
**Post date:** [September 9, 2021, 10:06am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftrain-wordpiece-from-scratch%2F9843%2F1 "2021-09-09T10:06:32Z")
Hi,
I am pre training a Bert model from scratch. For that I first need to train a wordpiece tokenizer, I am using BertWordPieceTokenizer for this.
My question:
Should I train the tokenizer on the whole corpus which is huge, or training it on a sample is enough?
Is there a way to tell the tokenizer to take train only on a sample?
Thanks.
---
**Author:**  [@nielsr](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fnielsr)%5C
**Post date:** [September 9, 2021, 11:41am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftrain-wordpiece-from-scratch%2F9843%2F2 "2021-09-09T11:41:14Z")
> [@Kamel](#):
>
> Should I train the tokenizer on the whole corpus which is huge
Yes. With [HuggingFace Tokenizers](https://github.com/huggingface/tokenizers), it takes seconds. From the README: “Takes less than 20 seconds to tokenize a GB of text on a server’s CPU”.
---
**Author:**  [@Kamel](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FKamel)%5C
**Post date:** [September 9, 2021, 11:56am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftrain-wordpiece-from-scratch%2F9843%2F3 "2021-09-09T11:56:35Z")
Thanks again Nielsr