# Pipelines without a tokenizer

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fpipelines-without-a-tokenizer%2F9301%3E%5C
**Category:** 🤗Transformers\
**Created:** [August 17, 2021, 4:46pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fpipelines-without-a-tokenizer%2F9301 "2021-08-17T16:46:53Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![brosand](https://avatars.discourse-cdn.com/v4/letter/b/3e96dc/32.png) [@brosand](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fbrosand)%5C
**Post date:** [August 17, 2021, 4:46pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fpipelines-without-a-tokenizer%2F9301%2F1 "2021-08-17T16:46:53Z")

</div>

Hello,

I am trying to use a pipeline for zero-shot classification, and due to my need to pretokenize the data in order to split large text, I have tokenized data I would like to feed into the pipeline. Is there any way to do this? I don’t see any options to run a pipeline without a tokenizer.

Is my only method to define some new tokenizer that doesn’t actually do anything?

Thanks

---

<div class="post-metadata">

**Author:** ![Illuminarchie](https://avatars.discourse-cdn.com/v4/letter/i/c5a1d2/32.png) [@Illuminarchie](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FIlluminarchie)%5C
**Post date:** [February 19, 2024, 12:27am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fpipelines-without-a-tokenizer%2F9301%2F2 "2024-02-19T00:27:25Z")

</div>

Hi,

Hate to bring this back up, but did you ever find a solution to this? I was thinking of trying to make a custom tokenizer which doesn’t do anything, but I was wondering if you found any good solution ! Thank you!