# Distilbert-base-multilingual-cased'
**URL:**
**Author:**  [@natank](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fnatank)%5C
**Post date:** [June 22, 2021, 8:46am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdistilbert-base-multilingual-cased%2F7054%2F1 "2021-06-22T08:46:05Z")
Hello
I am running distilbert-base-multilingual-cased’ on Pytorch. My model has 4 classes in the target.
In the code in **models/distilbert/modeling\_distilbert.py**. I am reaching this state
**elif self.config.problem\_type == “multi\_label\_classification”:**
\*\* loss\_fct = BCEWithLogitsLoss()\*\*
\*\* loss = loss\_fct(logits, labels)\*\*
I have two questions:
1 BCEWithLogitsloss must receive the labels as one hot rather integers . must I take care of doing one\_hot?
2 If I wish to add regulation functions to the loss , what is the best practice for doing so?
Thanks
---
**Author:**  [@sgugger](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsgugger)%5C
**Post date:** [June 22, 2021, 1:21pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdistilbert-base-multilingual-cased%2F7054%2F2 "2021-06-22T13:21:08Z")
Note that `multi_label_classification` is only for problems where you can have multiple labels for one example, so you should use the default if your samples can only have one label.
If you are in a true multiple label problem, then it’s very likely your labels are already in a one-hot format.
For your second question, you should just output the logits of your model and then compute the loss manually with your penalty. If you’re using the Trainer API, you can subclass and write a `compute_loss` function with that, see [here](https://huggingface.co/transformers/main_classes/trainer.html) for an example.
---
**Author:**  [@natank](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fnatank)%5C
**Post date:** [June 22, 2021, 1:57pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdistilbert-base-multilingual-cased%2F7054%2F3 "2021-06-22T13:57:52Z")
Thanks for the quick answer
Q2 is pretty clear
Q1. - I realized that I gave labels of int64. As we begin to train , in data\_loader.py we have the block
**if “label” in first and first[“label”] is not None:**
\*\* label = first[“label”].item() if isinstance(first[“label”], torch.Tensor) else first[“label”]\*\*
\*\* dtype = torch.long if isinstance(label, int) else torch.float\*\*
Since is instance(label, int) for int64 is False , it converted the labels to Float. Which causes to multi\_label\_classifcation due to the following block
**elif self.num\_labels \> 1 and (labels.dtype == torch.long or labels.dtype == torch.int):**
\*\* self.config.problem\_type = “single\_label\_classification”\*\*
\*\* else:\*\*
\*\* self.config.problem\_type = “multi\_label\_classification”\*\*
_The corollary is that one has to give labels as int and not int64_
Thanks
Thanks
Natan