# How to use transformers for batch inference

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-to-use-transformers-for-batch-inference%2F9366%3E%5C
**Category:** 🤗Transformers\
**Created:** [August 20, 2021, 7:37am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-to-use-transformers-for-batch-inference%2F9366 "2021-08-20T07:37:23Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![wangdong](https://avatars.discourse-cdn.com/v4/letter/w/73ab20/32.png) [@wangdong](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fwangdong)%5C
**Post date:** [August 20, 2021, 7:37am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-to-use-transformers-for-batch-inference%2F9366%2F1 "2021-08-20T07:37:23Z")

</div>

I use transformers to train text classification models，for a single text, it can be inferred normally. The code is as follows

```auto
from transformers import BertTokenizer, TFAlbertForSequenceClassification
text = 'This is a sentence'
model_path ='../albert_chinese_tiny'
tokenizer = BertTokenizer.from_pretrained(model_path)
model = TFAlbertForSequenceClassification.from_pretrained('../model_tf/20210818')
encoding = tokenizer(text, truncation=True, padding=True, max_length=30, return_tensors="tf")
result = model(encoding)

```

When I predict more than one text at a time, an error will be reported. The code is as follows

```auto
texts = ['This is a sentence', 'This is another sentence']
encodings = []
model_path ='../albert_chinese_tiny'
tokenizer = BertTokenizer.from_pretrained(model_path)
model = TFAlbertForSequenceClassification.from_pretrained('../model_tf/20210818')
for text in texts:
    encoding = tokenizer(text, truncation=True, padding=True, max_length=30, return_tensors="tf")
    encodings.append(encoding)
result = model(np.array(encodings))

```

The error information is as follows:  
tensorflow.python.framework.errors\_impl.InvalidArgumentError: Value for attr ‘Tindices’ of string is not in the list of allowed values: int32, int64  
; NodeDef: {{node ResourceGather}}; Op\<name=ResourceGather; signature=resource:resource, indices:Tindices → output:dtype; attr=batch\_dims:int,default=0; attr=validate\_indices:bool,default=true; attr=dtype:type; attr=Tindices:type,allowed=[DT\_INT32, DT\_INT64]; is\_stateful=true\> [Op:ResourceGather]

---

<div class="post-metadata">

**Author:** ![nielsr](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/nielsr/32/39617_2.png) [@nielsr](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fnielsr)%5C
**Post date:** [August 20, 2021, 12:03pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-to-use-transformers-for-batch-inference%2F9366%2F2 "2021-08-20T12:03:53Z")

</div>

This question has been answered [here](https://github.com/huggingface/transformers/issues/13199).