# Multi-label token classification: "-100" special label

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fmulti-label-token-classification-100-special-label%2F48092%3E%5C
**Category:** 🤗Transformers\
**Created:** [July 25, 2023, 2:23pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fmulti-label-token-classification-100-special-label%2F48092 "2023-07-25T14:23:58Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![dried](https://avatars.discourse-cdn.com/v4/letter/d/7feea3/32.png) [@dried](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fdried)%5C
**Post date:** [July 25, 2023, 2:23pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fmulti-label-token-classification-100-special-label%2F48092%2F1 "2023-07-25T14:23:59Z")

</div>

Hello!  
I am implementing a HF-based model augmented with native Pytorch-code to classify tokens (not the document!) into one or more classes.

Now, based on single labels and using the AutoTokenizer and after aligning subwords with their labels, I get the following output, where the labels -100 correspond to the [CLS] and [SEP] special tokens as well as subwords starting with ## (not seen below). Apparently, the -100 labels tell the model to ignore the token when calculating loss.

{‘input\_ids’: [2,  
4759,  
4683,  
3],  
‘token\_type\_ids’: [0,  
0,  
0,  
0],  
‘attention\_mask’: [1,  
1,  
1,  
1],  
‘labels’: [-100,  
3,  
4,  
-100]  
}

However, with multi-label classification, my labels are lists of one-hot encodings as required for calculating nn.CEWithLogitsLoss():

‘labels’: [XXX,  
[1, 0, 1],  
[1, 1, 0],  
[1, 0, 0],  
XXX]]

What should I put here for XXX instead of the “-100” special label to tell the model to ignore special tokens as well as subword tokens?

---

<div class="post-metadata">

**Author:** ![Fedoration](https://avatars.discourse-cdn.com/v4/letter/f/8edcca/32.png) [@Fedoration](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FFedoration)%5C
**Post date:** [September 18, 2023, 10:30am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fmulti-label-token-classification-100-special-label%2F48092%2F2 "2023-09-18T10:30:28Z")

</div>

Perhaps you can try to create a vector consisting of -100 values?  
So instead of XXX you can try [-100 -100 -100]

Let me know if you get some results.