# Error in fine-tuning BERT

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ferror-in-fine-tuning-bert%2F3469%3E%5C
**Category:** Beginners\
**Created:** [February 1, 2021, 6:41pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ferror-in-fine-tuning-bert%2F3469 "2021-02-01T18:41:28Z")\
**Posts on this page:** 1\
**Showing post:** 8

<div class="post-metadata">

**Author:** ![Mo777](https://avatars.discourse-cdn.com/v4/letter/m/da6949/32.png) [@Mo777](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FMo777)%5C
**Post date:** [February 16, 2022, 3:53pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ferror-in-fine-tuning-bert%2F3469%2F8 "2022-02-16T15:53:15Z")

</div>

I have a similar problem!  
I am working on a summarization model with Bart.  
so far I can train the model and it is fine, but when I want to do trainer.evaluate() it returns this warning:  
“”"  
VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify ‘dtype=object’ when creating the ndarray.  
result = getattr(asarray(obj), method)(\*args, \*\*kwds)  
“”"  
and this error:  
“”"  
**ValueError** : could not broadcast input array from shape (50,32,50265) into shape (50,)  
“”""  
50 is the number of rows in my evaluation dataset  
32 is the max\_evaluation\_length  
50265 is BartForConditionalGeneration(  
(model): BartModel(  
(shared): Embedding(50265, 768, padding\_idx=1)

here is the code:  
from transformers import AutoTokenizer  
model\_checkpoint = “facebook/bart-base”

tokenizer = AutoTokenizer.from\_pretrained(model\_checkpoint)

max\_input\_length = 64  
max\_target\_length = 32

def preprocess\_function(example):

```
model_inputs = tokenizer(example['text'],max_length = max_input_length, padding=True , truncation= True) 

with tokenizer.as_target_tokenizer():

    labels = tokenizer(example['summary'], max_length =max_target_length ,padding=True , truncation = True ) 

model_inputs['labels'] = labels['input_ids']

return model_inputs

```

tokenized\_datasets = dataset.map(preprocess\_function, batched= True, remove\_columns = [‘text’,‘summary’])

from transformers import AutoModelForSeq2SeqLM

model = AutoModelForSeq2SeqLM.from\_pretrained(“facebook/bart-base”)

from transformers import TrainingArguments

training\_args = TrainingArguments(“test\_trainer”)

from transformers import Trainer

import numpy as np

from datasets import load\_metric

#metric = load\_metric(“accuracy”)

metric = load\_metric(“glue”, “mrpc”)

def compute\_metrics(eval\_pred):

```
logits, labels = eval_pred

predictions = np.argmax(logits, axis=-1)

return metric.compute(predictions=predictions, references=labels)

```

trainer = Trainer(model=model,

```
        args=training_args,

        train_dataset=full_train_dataset,

        eval_dataset=full_eval_dataset,

        compute_metrics=compute_metrics,

        tokenizer = tokenizer)

```

then 🙂  
I train the model and it works fine but trainer.evaluate() returns that error

but it works at the start like this:  
\*\*\*\*\* Running Evaluation \*\*\*\*\*  
Num examples = 50  
Batch size = 8

6%|▋ | 12/189 [13:07\<3:13:36, 65.63s/it]  
33%|███▎ | 62/189 [08:23\<17:12, 8.13s/it]

then error!

---

_[View the full topic](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ferror-in-fine-tuning-bert%2F3469)._