# Unbelievable Error: Help ME!

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Funbelievable-error-help-me%2F129971%3E%5C
**Category:** 🤗Transformers\
**Created:** [December 8, 2024, 12:11pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Funbelievable-error-help-me%2F129971 "2024-12-08T12:11:11Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![richardlevinson3](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/richardlevinson3/32/37833_2.png) [@richardlevinson3](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Frichardlevinson3)%5C
**Post date:** [December 8, 2024, 12:11pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Funbelievable-error-help-me%2F129971%2F1 "2024-12-08T12:11:11Z")

</div>

Hey Guys, I got unbelievable Error.

ValueError: You have set `args.eval_strategy` to steps but you didn’t pass an `eval_dataset` to `Trainer`. Either set `args.eval_strategy` to `no` or pass an `eval_dataset`.

My training code is:

```auto
training_args = TrainingArguments(
    output_dir="/kaggle/working/twitter-sentiment-analysis-llm",
    report_to="wandb",
    learning_rate=2e-5,
    per_device_train_batch_size=16,
    per_device_eval_batch_size=16,
    num_train_epochs=3,
    weight_decay=0.01,
    eval_strategy="steps",
    save_strategy="steps",
    load_best_model_at_end=True,
    eval_steps=500,
    save_steps=500,
    logging_steps=10,
    fp16=True,
)

trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=train_ds['train'],
    eval_dataset=train_ds['test'],
    tokenizer=tokenizer,
    data_collator=data_collator,
    compute_metrics=compute_metrics,
)

trainer.train()

```

I checked eval\_dataset and passed it exactly. But I got an error after all training steps.

Guys, Please Help me!!

---

<div class="post-metadata">

**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FJohn6666)%5C
**Post date:** [December 8, 2024, 12:38pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Funbelievable-error-help-me%2F129971%2F2 "2024-12-08T12:38:18Z")

</div>

There seem to be several patterns… I hope it’s not a new bug.

> [@ValueError: Trainer: evaluation requires an eval\_dataset](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fvalueerror-trainer-evaluation-requires-an-eval-dataset%2F40670)%3A
>
> I’m trying to do a finetuning without an evaluation dataset. For that, I’m using the following code: training\_args = TrainingArguments( output\_dir=resume\_from\_checkpoint, evaluation\_strategy="epoch", per\_device\_train\_batch\_size=1, ) def compute\_metrics(pred: EvalPrediction): labels = pred.label\_ids preds = pred.predictions.argmax(-1) f1 = f1\_score(labels, preds, average="weighted") acc = accuracy\_score(labels, preds, average="…

> You can avoid this error by not specifying `evaluation_strategy` in `TrainingArguments`.

> <https://stackoverflow.com/questions/76310533/how-to-fix-trainer-evaluation-requires-an-eval-dataset-in-huggingface-transfo>

---

<div class="post-metadata">

**Author:** ![richardlevinson3](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/richardlevinson3/32/37833_2.png) [@richardlevinson3](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Frichardlevinson3)%5C
**Post date:** [December 9, 2024, 2:03am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Funbelievable-error-help-me%2F129971%2F3 "2024-12-09T02:03:44Z")

</div>

Thanks for your comment.  
You’re right, but I 'd like to use evaluation step.

---

<div class="post-metadata">

**Author:** ![ipzk201-kio](https://avatars.discourse-cdn.com/v4/letter/i/5daacb/32.png) [@ipzk201-kio](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fipzk201-kio)%5C
**Post date:** [December 11, 2024, 4:54pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Funbelievable-error-help-me%2F129971%2F4 "2024-12-11T16:54:24Z")

</div>

+1

Any workarounds yet?

UDP: In my case it was a problem with dataset separation. Try len() both of your train and validation dataset

---

<div class="post-metadata">

**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FJohn6666)%5C
**Post date:** [December 12, 2024, 12:53am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Funbelievable-error-help-me%2F129971%2F5 "2024-12-12T00:53:35Z")

</div>

Should we raise an issue on the github for the datasets library?😅  
It’s not clear whether it’s a bug or not, but at the very least the error message is not appropriate.

> **[Issues · huggingface/datasets](https://github.com/huggingface/datasets/issues)**
>
> 🤗 The largest hub of ready-to-use datasets for ML models with fast, easy-to-use and efficient data manipulation tools - Issues · huggingface/datasets

---

<div class="post-metadata">

**Author:** ![mettinski](https://avatars.discourse-cdn.com/v4/letter/m/a88e4f/32.png) [@mettinski](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fmettinski)%5C
**Post date:** [December 18, 2024, 12:06pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Funbelievable-error-help-me%2F129971%2F6 "2024-12-18T12:06:51Z")

</div>

Had the same issue, was mentioned [here](https://github.com/huggingface/transformers/issues/34701) and fixed [here](https://github.com/huggingface/transformers/commit/7b3d615bc23c40daa9f38dbae0abd979936f6414). `pip install transformers==4.47.1` fixed it for me

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/hellohellohello/original/2X/d/de4155eb4aa4108ecb32a1389d7cc37ae69f88b7.png) [@system](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsystem)%5C
**Post date:** [December 19, 2024, 9:06am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Funbelievable-error-help-me%2F129971%2F7 "2024-12-19T09:06:53Z")

</div>

This topic was automatically closed 12 hours after the last reply. New replies are no longer allowed.