# Batching on Vanilla CPU for Inference

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fbatching-on-vanilla-cpu-for-inference%2F47048%3E%5C
**Category:** 🤗Transformers\
**Created:** [July 17, 2023, 7:11am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fbatching-on-vanilla-cpu-for-inference%2F47048 "2023-07-17T07:11:48Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![Shamik](https://avatars.discourse-cdn.com/v4/letter/s/ea5d25/32.png) [@Shamik](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FShamik)%5C
**Post date:** [July 17, 2023, 7:11am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fbatching-on-vanilla-cpu-for-inference%2F47048%2F1 "2023-07-17T07:11:48Z")

</div>

Hello,

Infrastructure: Vanilla CPU(any intel based CPU without any specialised ASIC/TPUs or anything)

How does transformers pipeline class handle batching for a system only with CPU, e.g.

```python
 pipeline(task='token-classification',
                     model=model,
                     tokenizer=tokenizer,
                     batch_size = 128,
                     device='cpu',
                     )

```

If the above code(changing device to ‘cuda’) was working on a GPU then it would do a batch inferencing unless the batch size is too much and it encounters a OOM allocation error.

However, how would the same code for a CPU? How does the pipeline class natively handle it when the CPU isn’t capable of handling batch computing?

Thank you