# How can I evaluate a fine tuned LLM?
**URL:**
**Author:**  [@mertoguzhan](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fmertoguzhan)%5C
**Post date:** [January 6, 2025, 8:35pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-can-i-evaluate-a-fine-tuned-llm%2F134538%2F1 "2025-01-06T20:35:11Z")
Hi there! I am curious about how fine tuned large language models are evaluated? I mean, I fine tuned a model for specific task like a question-answer chatbot for my business but how can I decide to my language model is good, bad or terrible? Is there any ways to evaluate fine tuned model such as token similarity, semantic similarity…
---
**Author:**  [@John6666](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FJohn6666)%5C
**Post date:** [January 7, 2025, 1:45am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-can-i-evaluate-a-fine-tuned-llm%2F134538%2F2 "2025-01-07T01:45:45Z")
I’m not very familiar with it, but is it possible to use the evaluation method used in the leaderboard?
> **[Spaces - Hugging Face](https://huggingface.co/spaces?sort=trending&search=leaderboard)**
>
> Discover amazing ML apps made by the community
---
**Author:**  [@Alanturner2](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FAlanturner2)%5C
**Post date:** [January 7, 2025, 2:08am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-can-i-evaluate-a-fine-tuned-llm%2F134538%2F3 "2025-01-07T02:08:29Z")
Hi!
Evaluating a fine-tuned language model, especially for tasks like a question-answer chatbot, involves a mix of quantitative metrics and qualitative evaluation. Here are some common methods you can use to assess your model’s performance:
1. **Task-Specific Metrics** :
2. **Semantic Similarity** :
3. **Human Evaluation** :
You should follow @John6666 's advice.
4. **Real-World Testing** :
You should follow @John6666 's advice. This is very effective way to test your project using hugging face space. I’ve also used this space and this was helpful.
By combining these approaches, you can get a holistic view of your model’s quality and identify areas that need improvement. Task-specific metrics like F1 score and EM are great for initial evaluations, while human evaluation and real-world testing can offer deeper insights.
Hope this helps!
---
**Author:**  [@mertoguzhan](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fmertoguzhan)%5C
**Post date:** [January 7, 2025, 3:34am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-can-i-evaluate-a-fine-tuned-llm%2F134538%2F4 "2025-01-07T03:34:07Z")
Thanks a lot. I believe that semantic similarity will be good decision but I will try all of them.
---
**Author:**  [@system](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsystem)%5C
**Post date:** [January 7, 2025, 3:34pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-can-i-evaluate-a-fine-tuned-llm%2F134538%2F5 "2025-01-07T15:34:12Z")
Closed on January 7, 2025, 3:34pm UTC
This topic was automatically closed 12 hours after the last reply. New replies are no longer allowed.