# TrOCR, CER metric error

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftrocr-cer-metric-error%2F12653%3E%5C
**Category:** 🤗Transformers\
**Created:** [December 9, 2021, 7:02am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftrocr-cer-metric-error%2F12653 "2021-12-09T07:02:29Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Kforcode](https://avatars.discourse-cdn.com/v4/letter/k/57b2e6/32.png) [@Kforcode](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FKforcode)%5C
**Post date:** [December 9, 2021, 7:02am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftrocr-cer-metric-error%2F12653%2F1 "2021-12-09T07:02:29Z")

</div>

I am finetuning TrOCR and using Character Error Rate from jiwer as the metric.

```nohighlight
def compute_cer(pred_ids, label_ids, processor):
    pred_str = processor.batch_decode(pred_ids, skip_special_tokens=True)
    label_ids[label_ids == -100] = processor.tokenizer.pad_token_id
    print(f"len of label_ids {len(label_ids)}")
    label_str = processor.batch_decode(label_ids, skip_special_tokens=True)
    print(f"len_pred_str={len(pred_str)}, len_label={len(label_str)}")
    cer = cer_metric.compute(predictions=pred_str, references=label_str)
    return cer 

```

Except for the print statements the code is a direct copy from @nielsr[tutorial](https://github.com/NielsRogge/Transformers-Tutorials/blob/master/TrOCR/Fine_tune_TrOCR_on_IAM_Handwriting_Database_using_native_PyTorch.ipynb) . Despite len(pred\_str) and len(label\_str) being the same,

I am getting

```nohighlight
ValueError: number of ground truth inputs (17) and hypothesis inputs (24) must match.

```

I have attached the screenshot of the same

 ![image](https://us1.discourse-cdn.com/hellohellohello/original/2X/d/db3285a36d2304adeb16c4b51b910b52f7084b92.png)

Please let me know, if you have any clue what might be causing the issue

---

<div class="post-metadata">

**Author:** ![nielsr](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/nielsr/32/39617_2.png) [@nielsr](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fnielsr)%5C
**Post date:** [December 9, 2021, 10:05am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftrocr-cer-metric-error%2F12653%2F2 "2021-12-09T10:05:58Z")

</div>

I believe this was a bug that has been fixed, see [Datasets.load\_metric("cer") does not work](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdatasets-load-metric-cer-does-not-work%2F11931)