# Shape mismatch between labels and logits

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fshape-mismatch-between-labels-and-logits%2F30607%3E%5C
**Category:** 🤗Transformers\
**Created:** [January 28, 2023, 4:42am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fshape-mismatch-between-labels-and-logits%2F30607 "2023-01-28T04:42:35Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Kamugg](https://avatars.discourse-cdn.com/v4/letter/k/91b2a8/32.png) [@Kamugg](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FKamugg)%5C
**Post date:** [January 28, 2023, 4:42am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fshape-mismatch-between-labels-and-logits%2F30607%2F1 "2023-01-28T04:42:35Z")

</div>

Hello everyone I really hope this is the correct category for this question. I’m using TFAutoModelForSequenceClassification to perform a multi labeling task on a dataset. This dataset has a text and 20 columns, one for each class. If an example has a 1 on a column it means that it belongs to that class, an example could be:

‘It’s cold today’ 0 0 1 1 0 1 0

I loaded the Dataframe into a HF dataset and I loaded the model and tokenizer with:

```auto
tokenizer = AutoTokenizer.from_pretrained("bert-base-uncased")
bert = TFAutoModelForSequenceClassification.from_pretrained('bert-base-uncased', num_labels=num_labels, problem_type='multi_label_classification', id2label=id2labels, label2id=labels2id)

```

I then proceeded to convert the dataset into:

‘Tokenized text’ | ‘Array of 0s and 1s’

To do this I wrote this code:

```auto
def tokenize_and_encode(val, tokenizer, max_length):
  tokenized = tokenizer(val['Premise'], truncation=True, padding='max_length', max_length=max_length)
  labels = []
  for index in id2labels.keys():
    # Convert the columns into a single array of zeros and ones
    labels.append(val[str(index)])
  return {'input_ids': tokenized['input_ids'],
          'attention_mask': tokenized['attention_mask'],
          'labels': labels}
  

train_dataset = Dataset.from_pandas(train_df)

train_dataset = train_dataset.map(lambda x: tokenize_and_encode(x, tokenizer, 200), remove_columns=train_dataset.column_names)

```

I then prepared the dataset and the model for the training phase:

```auto
batch_size = 16
num_epochs = 3
batches_per_epoch = len(train_dataset) // batch_size
total_train_steps = int(batches_per_epoch * num_epochs)
optimizer, schedule = create_optimizer(init_lr=2e-5, num_warmup_steps=0, num_train_steps=total_train_steps)

tf_train_set = bert.prepare_tf_dataset(
    train_dataset,
    shuffle=True,
    batch_size=batch_size
)

bert.compile(optimizer=optimizer)

```

Until now I have no problems in running the code. But when I call:

```auto
history = bert.fit(tf_train_set, epochs=5)

```

I receive a very big error, but I think the most important part is:

```auto
ValueError: `labels.shape` must equal `logits.shape` except for the last dimension. Received: labels.shape=(320,) and logits.shape=(16, 20)
    
    
    Call arguments received by layer "tf_bert_for_sequence_classification" (type TFBertForSequenceClassification):
      • self={'input_ids': 'tf.Tensor(shape=(16, 200), dtype=int64)', 'attention_mask': 'tf.Tensor(shape=(16, 200), dtype=int64)', 'labels': 'tf.Tensor(shape=(16, 20), dtype=int64)'}
      • input_ids=None
      • attention_mask=None
      • token_type_ids=None
      • position_ids=None
      • head_mask=None
      • inputs_embeds=None
      • output_attentions=None
      • output_hidden_states=None
      • return_dict=None
      • labels=None
      • training=True

```

The model states that it received labels of shape 320 when I should have provided a shape of (16, 20) and the line below the error states that I indeed provided a shape of (16, 20). It’s like my labels are being flattened? I can’t understand what’s happening.

Thank you very much to all of you.

---

<div class="post-metadata">

**Author:** ![wtt2077](https://avatars.discourse-cdn.com/v4/letter/w/dec6dc/32.png) [@wtt2077](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fwtt2077)%5C
**Post date:** [December 27, 2023, 8:03am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fshape-mismatch-between-labels-and-logits%2F30607%2F2 "2023-12-27T08:03:59Z")

</div>

model.compile(optimizer=optimizer, loss=‘categorical\_crossentropy’)