# Model responses are random ignoring my dataset
**URL:**
**Author:**  [@DigioMatthy](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FDigioMatthy)%5C
**Post date:** [July 28, 2025, 9:12am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fmodel-responses-are-random-ignoring-my-dataset%2F164782%2F1 "2025-07-28T09:12:37Z")
I am using AutoTrain to finetune my Llama model with my custom data and the model give random responses ignoring my dataset. The thing is that on my dataset I have 145 rows in JSONL and when I start the fine-tuning with this dataset and I analyze logs I can see these rows:
So the dataset is recognized with 145 rows so from here I can understand that my dataset is well-structured and every row is a valid JSON object.
But right after the model shards are uploaded, it gives me this log:
```auto
Generating train split: 0 examples [00:00, ? examples/s]
Generating train split: 9 examples [00:00, ? examples/s]
```
So my question is: Why does it log Generating train split 0 examples and Generating train split 9 examples right below?
Is this a normal behaviour of AutoTrain?
Or thereās something that I have to adjust on my training dataset?
After the model is finetuned, obviously I can see it on my HuggingFace hub and I can also see the training statistics on TensorBoard but I see only a dot on the graphs and the training loss about 5.4, so yeah, everytime I try to ask him something about my dataset or anything else, he answers me randomly.
What can I do in order to finetune a model in the right way? Maybe I just have to expand my dataset because 145 rows are not enough and those logs are just normal?
---
**Author:**  [@John6666](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FJohn6666)%5C
**Post date:** [July 28, 2025, 10:01am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fmodel-responses-are-random-ignoring-my-dataset%2F164782%2F2 "2025-07-28T10:01:44Z")
> Why does it log Generating train split 0 examples and Generating train split 9 examples right below?
This error seems to occur when [Column Mapping](https://huggingface.co/docs/autotrain/col_map) is not set correctly.
---
**Author:**  [@DigioMatthy](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FDigioMatthy)%5C
**Post date:** [July 28, 2025, 10:18am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fmodel-responses-are-random-ignoring-my-dataset%2F164782%2F3 "2025-07-28T10:18:12Z")
My dataset is a jsonl format and has only one column ātextā.
In AutoTrain I set the Column Mapping like this:
And the chat template parameter is set to None
---
**Author:**  [@John6666](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FJohn6666)%5C
**Post date:** [July 28, 2025, 10:23am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fmodel-responses-are-random-ignoring-my-dataset%2F164782%2F4 "2025-07-28T10:23:53Z")
It appears to be correctā¦ Another possible factor is that [`packing` is enabled with the small dataset](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ferror-using-sfttrainer-make-sure-that-your-dataset-has-enough-samples-to-at-least-yield-one-packed-sequence%2F73731%2F7).
Also, unless there is a specific reason, I think itās safer to leave [Chat Template](https://huggingface.co/docs/transformers/v4.53.3/en/chat_templating) on automatic.
---
**Author:**  [@DigioMatthy](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FDigioMatthy)%5C
**Post date:** [July 28, 2025, 10:36am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fmodel-responses-are-random-ignoring-my-dataset%2F164782%2F5 "2025-07-28T10:36:43Z")
Following the general documentation on the Column Mapping in AutoTrain topic I tried to set the Column Mapping like this:
And it gives me error KeyError {ātextā: ātextā} is invalid. (even if Iām using SFT)
So now looking at the discussion they are talking about disabling the parameter packing but the thing is that even if I enable full parameter mode there is no packing parameter, anyway Iām using basic parameter mode because otherwise I donāt know what to tweak.
Maybe do I have to write manually parameters activating JSON parameters first and doing so I can write like `packing=false` and try with other parameters?
Or maybe itās just my dataset too small and I have to expand it?
---
**Author:**  [@John6666](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FJohn6666)%5C
**Post date:** [July 28, 2025, 1:03pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fmodel-responses-are-random-ignoring-my-dataset%2F164782%2F6 "2025-07-28T13:03:22Z")
There is no doubt that the dataset is too small, but I donāt think itās absolutely impossible with that amount of dataā¦
If there is a publicly available dataset that can reproduce the symptoms, it would be possible to investigateā¦
If there are no settings for packing, [it will be difficult with SFT](https://huggingface.co/docs/trl/en/sft_trainer#packing-dataset) with small datasetā¦
---
**Author:**  [@DigioMatthy](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FDigioMatthy)%5C
**Post date:** [July 28, 2025, 1:22pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fmodel-responses-are-random-ignoring-my-dataset%2F164782%2F7 "2025-07-28T13:22:04Z")
Ok it was predictable that the dataset was too small for a real fine-tuning actually, Iāll create a bigger one and Iāll try launch a finetuning and weāll see if I will have the same problem, but I donāt think so š .
Last question, what do you think the minimal amount of examples a dataset should have in order to make a really good and successful fine-tuning?
---
**Author:**  [@DigioMatthy](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FDigioMatthy)%5C
**Post date:** [July 28, 2025, 1:26pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fmodel-responses-are-random-ignoring-my-dataset%2F164782%2F8 "2025-07-28T13:26:55Z")
Ah I forgot to say, maybe the issue could be that AutoTrain GUI doesnāt permit to set a value to a packing parameter because behind itās a default set and it canāt be handled, so if someone wants to train their own model, the dataset has to be large
---
**Author:**  [@John6666](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FJohn6666)%5C
**Post date:** [July 28, 2025, 1:49pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fmodel-responses-are-random-ignoring-my-dataset%2F164782%2F9 "2025-07-28T13:49:26Z")
Hmm, I think you should ask someone who knows more about LLM fine-tuning than I do, but what I sometimes hear is that ā500 to 1000 samples are sufficient for LoRAā, ā[data diversity is more important than quantity](https://arxiv.org/pdf/2305.11206)ā, etc.
Since [it is difficult to manually create a dataset from scratch, many people choose to use existing AI tools to create dataset](https://huggingface.co/posts/CultriX/959128360368232). Also, [the online documents like this](https://huggingface.co/blog/tegridydev/llm-dataset-formats-101-hugging-face) may be useful references regarding formatting.
---
**Author:**  [@John6666](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FJohn6666)%5C
**Post date:** [July 28, 2025, 1:55pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fmodel-responses-are-random-ignoring-my-dataset%2F164782%2F10 "2025-07-28T13:55:12Z")
There are people who know more about AI than I do who say things like, _āAsk AI about AI.ā_ Commercial AI systems like Gemini and ChatGPT have been trained on a lot of AI-related information, so when you ask them about AI itself, they often provide fairly reliable answers. Since they have a solid foundation of knowledge, even just enabling search can help you gather reasonably up-to-date information.
---
**Author:**  [@DigioMatthy](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FDigioMatthy)%5C
**Post date:** [July 28, 2025, 1:55pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fmodel-responses-are-random-ignoring-my-dataset%2F164782%2F11 "2025-07-28T13:55:18Z")
Ok, I think these documentations you pinged me are enough to solve the dataset problem.
Thank you so much for your time and support!! šŖ
---
**Author:**  [@DigioMatthy](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FDigioMatthy)%5C
**Post date:** [July 28, 2025, 1:56pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fmodel-responses-are-random-ignoring-my-dataset%2F164782%2F12 "2025-07-28T13:56:32Z")
Wow, didnāt know that. Ok will try it then! Ty!! šŖ šŖ
---
**Author:**  [@system](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsystem)%5C
**Post date:** [July 29, 2025, 1:56am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fmodel-responses-are-random-ignoring-my-dataset%2F164782%2F13 "2025-07-29T01:56:48Z")
Closed on July 29, 2025, 1:56am UTC
This topic was automatically closed 12 hours after the last reply. New replies are no longer allowed.