# Load SentencePieceBPETokenizer in TF

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fload-sentencepiecebpetokenizer-in-tf%2F17257%3E%5C
**Category:** 🤗Tokenizers\
**Created:** [April 27, 2022, 9:07pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fload-sentencepiecebpetokenizer-in-tf%2F17257 "2022-04-27T21:07:36Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![jbmaxwell](https://avatars.discourse-cdn.com/v4/letter/j/5daacb/32.png) [@jbmaxwell](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fjbmaxwell)%5C
**Post date:** [April 27, 2022, 9:07pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fload-sentencepiecebpetokenizer-in-tf%2F17257%2F1 "2022-04-27T21:07:36Z")

</div>

I have a tokenizer trained using SentencePieceBPETokenizer from the tokenizers library.

```auto
tokenizer = SentencePieceBPETokenizer(add_prefix_space=True)

# Customize training
tokenizer.train(
    files=paths, 
    vocab_size=4000, 
    min_frequency=2, 
    show_progress=True, 
    special_tokens=special_tokens,
)

# Saving model
tokenizer.save("sp/tokenizer.json")

```

It’s the first one I’ve trained that really fits the task (in particular because it fills the requested `vocab_size` 🎉), so I’d like to use it everywhere I can.

I’m wondering whether it’s possible to load the `tokenizer.json` in the SentencepieceTokenizer from `tensorflow-text` (i.e., `text.SentencepieceTokenizer()`)? Or, can it be loaded with `PreTrainedTokenizerFast`, then converted to the TF format? Obviously the vocab and merges are the crucial parts I want to utilize in TF.

Thanks in advance.