# I have trained my classifier, now how do I do predictions?
**URL:**
**Author:**  [@Abe](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FAbe)%5C
**Post date:** [February 13, 2021, 12:09pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fi-have-trained-my-classifier-now-how-do-i-do-predictions%2F3625%2F1 "2021-02-13T12:09:08Z")
Hi everybody and thank you in advance for anyone who can help my out. I am not a total beginner when it comes to huggingface libraries (I have already built a well functioning sentiment analyzer) however I have mostly taken tutorials and integrated their content without going too much into details of who each line of code does. Trying to learn more I have put together a document classifier using a couple of tutorials I’ve found online.
I have built the trainer and the validator and they work just fine. I started with a dataset that assigns 6 different labels to a text, with each text having 0, 1 or more than 1 label. I trained the model and saved it. My problem is: now what? I can’t understand exactly how to do the prediction part. Here is where I am:
```
def validation():
model = torch.load(destination_folder+'model.pt')
model.eval()
with torch.no_grad():
for _, data in enumerate(testing_loader, 0):
ids = data['ids'].to(device, dtype = torch.long)
mask = data['mask'].to(device, dtype = torch.long)
token_type_ids = data['token_type_ids'].to(device, dtype = torch.long)
preds = model(ids, mask,token_type_ids)
print(preds.argmax(1) + 1)
```
This is a snippet of the output of the print command:
```
tensor([1, 1, 1, 1])
tensor([6, 1, 1, 1])
tensor([1, 1, 1, 1])
tensor([1, 5, 2, 1])
```
I’ve done this using the validation data and by adapting the validation routine, while in reality I would need to do this for a single line of text, but regardless of the way the data is fed to the prediction function, how do I read the prediction data? How do I go from “This is the text of my document to be classified” to “This document is 75% label1, 15% label5, 2% label6”?
Again, thank you in advance for any help!
---
**Author:**  [@lewtun](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Flewtun)%5C
**Post date:** [February 13, 2021, 12:57pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fi-have-trained-my-classifier-now-how-do-i-do-predictions%2F3625%2F2 "2021-02-13T12:57:32Z")
Hi @Abe, if I understand correctly you’d like to go from an input string like “I love this movie!” to a set of predicted labels and their confidence scores (i.e. probabilities).
The simplest way to achieve that would be to wrap your model and tokenizer in a [`TextClassificationPipeline`](https://huggingface.co/transformers/main_classes/pipelines.html?highlight=textclassificationpipeline#transformers.TextClassificationPipeline) with `return_all_scores=True`:
```auto
from transformers import TextClassificationPipeline
model = ...
tokenizer = ...
pipe = TextClassificationPipeline(model=model, tokenizer=tokenizer, return_all_scores=True)
# outputs a list of dicts like [[{'label': 'NEGATIVE', 'score': 0.0001223755971295759}, {'label': 'POSITIVE', 'score': 0.9998776316642761}]]
pipe("I love this movie!")
```
The above also works for multiple inputs by feed a list of examples instead of a single string:
```auto
pipe(["I love this movie!", "I hate this movie!"])
```
If you want to have human-readable labels like “positive” and “negative” you can configure the `id2label` and `label2id` attributes of your model’s config class: [Change label names on inference API - #3 by lewtun](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fchange-label-names-on-inference-api%2F3063%2F3)
HTH!
---
**Author:**  [@Abe](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FAbe)%5C
**Post date:** [February 13, 2021, 1:52pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fi-have-trained-my-classifier-now-how-do-i-do-predictions%2F3625%2F3 "2021-02-13T13:52:35Z")
Thank you for the incredibly quick reply!
I now get
```
torch.nn.modules.module.ModuleAttributeError: 'BERTClass' object has no attribute 'config'
```
Which makes me think the class used for training is missing something that pipe needs. This is the class I used
```
class BERTClass(torch.nn.Module):
def __init__ (self):
super(BERTClass, self). __init__ ()
self.l1 = transformers.BertModel.from_pretrained('bert-base-uncased', return_dict=False)
self.l2 = torch.nn.Dropout(0.3)
self.l3 = torch.nn.Linear(768, 6)
def forward(self, ids, mask, token_type_ids):
_, output_1= self.l1(ids, attention_mask = mask, token_type_ids = token_type_ids)
output_2 = self.l2(output_1)
output = self.l3(output_2)
return output
```
It always has no config attribute/method. Am I right in thinking I have to retrain using a different class?
---
**Author:**  [@lewtun](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Flewtun)%5C
**Post date:** [February 13, 2021, 2:15pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fi-have-trained-my-classifier-now-how-do-i-do-predictions%2F3625%2F4 "2021-02-13T14:15:58Z")
Ah yes, if possible I think you’d be better off using the `BertForSequenceClassification` class together with `BertConfig` instead of the custom class you created.
For example, you might be able to make this work as follows:
```auto
config = ...
model = BertForSequenceClassification.from_pretrained(destination_folder+'model.pt', config=config)
```
and then passing `model` and the tokenizer to the pipeline as before. If not, then you’ll probably have to re-train the model or live with the default labels from the pipeline
---
**Author:**  [@laurb](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Flaurb)%5C
**Post date:** [February 13, 2021, 5:00pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fi-have-trained-my-classifier-now-how-do-i-do-predictions%2F3625%2F5 "2021-02-13T17:00:59Z")
Using 3.5.1
I am also trying to use the text classification pipeline. I trained my model using trainer and saved it to “path to saved model”. My issue is that when I try to use the pipeline to predict, the call to the tokenizer is not truncating the result to the “model\_max\_length” set in the configuration of my trained model/tokenizer. I initialize as below:
tokenizer = RobertaTokenizer.from\_pretrained(“path to saved model”)
model = RobertaForSequenceClassification.from\_pretrained(“path to saved model”)
classifier = pipeline(‘sentiment-analysis’, model=model,tokenizer=tokenizer)
Do I need to create my own pipeline subclassing the text classification one in order to force truncation?
Thanks
---
**Author:**  [@lewtun](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Flewtun)%5C
**Post date:** [February 13, 2021, 8:26pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fi-have-trained-my-classifier-now-how-do-i-do-predictions%2F3625%2F6 "2021-02-13T20:26:59Z")
Hi @laurb, I think you can specify the truncation length by passing `max_length` as part of `generate_kwargs` (e.g. 50 tokens in my example):
```auto
classifier = pipeline(‘sentiment-analysis’, model=model, tokenizer=tokenizer, generate_kwargs={"max_length":50})
```
As far as I know the `Pipeline` class (from which all other pipelines inherit) does not truncate the inputs by default: [transformers/base.py at master · huggingface/transformers · GitHub](https://github.com/huggingface/transformers/blob/master/src/transformers/pipelines/base.py#L581)
---
**Author:**  [@BramVanroy](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FBramVanroy)%5C
**Post date:** [February 13, 2021, 10:08pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fi-have-trained-my-classifier-now-how-do-i-do-predictions%2F3625%2F7 "2021-02-13T22:08:21Z")
This would have been a lot easier if you had simply used BertForSequenceClassification because then you could have easily used this with `save_pretrained` inside the trainer. Here is [one example](https://huggingface.co/transformers/training.html#fine-tuning-in-native-pytorch) but there are more on Github in the examples folder.
---
**Author:**  [@Abe](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FAbe)%5C
**Post date:** [February 14, 2021, 1:09am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fi-have-trained-my-classifier-now-how-do-i-do-predictions%2F3625%2F8 "2021-02-14T01:09:07Z")
I tried this example but I don’t think this works with multiple labels since I get this:
```
1D target tensor expected, multi-target not supported
```
from this block of code
```
model.train()
for _,data in enumerate(training_loader, 0):
ids = data['ids'].to(device, dtype = torch.long)
mask = data['mask'].to(device, dtype = torch.long)
token_type_ids = data['token_type_ids'].to(device, dtype = torch.long)
targets = data['targets'].to(device, dtype = torch.float)
outputs = model(ids, mask, token_type_ids)
optimizer.zero_grad()
#loss = loss_fn(outputs, targets)
**loss = F.cross_entropy(outputs.logits, targets)**...
```