# Pipeline very slow

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fpipeline-very-slow%2F19474%3E%5C
**Category:** 🤗Transformers\
**Created:** [June 22, 2022, 10:40pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fpipeline-very-slow%2F19474 "2022-06-22T22:40:04Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![nprime496](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/nprime496/32/5055_2.png) [@nprime496](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fnprime496)%5C
**Post date:** [June 22, 2022, 10:40pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fpipeline-very-slow%2F19474%2F1 "2022-06-22T22:40:04Z")

</div>

I just trained a `BertForSequenceClassification` classifier but come on problems when trying to predict.

When I use the `predict` method of trainer on encodings I precomputed, I’m able to obtain predictions for ~350 samples from test set in less than 20 seconds.

```auto
trainer.predict(test_encodings)

```

However, when I load the model from storage and use a pipeline, the code runs for more than 10 mins, even adding batching doesn’t seem to help.

```auto
classifier = pipeline("text-classification", model=model,tokenizer=tokenizer,max_length=512, truncation=True)
classifier(X_test.Text.to_list(),batch_size=10)

```

What can explain this difference ?

---

<div class="post-metadata">

**Author:** ![anon84609851](https://avatars.discourse-cdn.com/v4/letter/a/2bfe46/32.png) [@anon84609851](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fanon84609851)%5C
**Post date:** [May 5, 2023, 1:52pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fpipeline-very-slow%2F19474%2F2 "2023-05-05T13:52:15Z")

</div>

Can’t offer any solutions, but I’m experiencing a similar problem training a model in the HuggingFace hub and trying to load it with the `from_pretrained` method to make predictions locally. I’m averaging 6s / observation - not something I can ever use in production.

This [SO post](https://stackoverflow.com/questions/76012523/huggingface-model-generate-is-extremely-slow-when-i-load-from-checkpoint) reports basically the same issue with `from_pretrained`…