I have model which I want to evaluate how and where can i do this . like I want to use some visual ques answering and some math benchmarks also. thanks for help in advance

Like this? [image] How can I evaluate a fine tuned LLM? Intermediate Hi there! I am curious about how fine tuned large language models are evaluated? I mean, I fine tuned a model for specific task like a question-answer chatbot for my business but how can I decide to my language model is good, bad or terrible? Is there any ways to evaluate fine tuned model such as token similarity, semantic similarity… [image] Let's talk about LLM evaluation A Blog post by Clémentine Fourrier on Hugging Face

Ohh but how dp people evaluate their models on standard benchmarks like MMLU, docVQA, etc. which we see in model releases and papers.

You could probably run each benchmark individually, but if you’re thinking about comparing it to other models, it might be more reliable and easier to ask the model to evaluate the leaderboard. [image] What's going on with the Open LLM Leaderboard? We’re on a journey to advance and democratize artificial intelligence through open source and open science. [image] open-llm-leaderboard (Open LLM Leaderboard) Evaluating open LLMs

I’m New to huggingface.co. I know this wrong area where I’m posting this but there’s no one to answer my question, I hope you guys will tolerate me. I’m running facefusion… and had processed a video which is saved in "/tmp " But I don’t know how to access that directory any help? The video was still processing when I logged out of my hugging face account and when I came back in it was showing completed in terminal But I can’t access that processed file. I have posted this question in beginners Category but its been 2 days and its still pending. I don’t know where to move forward.

Here. I don’t understand the situation well. There are multiple Face Fusions… [image] How to access /tmp files Beginners I’m running facefusion… and had processed a video which is saved in "/tmp " But I don’t know how to access that directory any help?

try this: [image] GitHub - EleutherAI/lm-evaluation-harness: A framework for few-shot evaluation of language... A framework for few-shot evaluation of language models.

Evaluating my own model

Intermediate

John6666 February 20, 2025, 5:26am 2

Like this?

Topic		Replies	Views
How to Evaluate Fine-Tuned LLMs? Models	3	614	November 27, 2025
What's your method for benchmarking? Models	3	240	October 6, 2026
How can I evaluate a fine tuned LLM? Intermediate	3	3076	January 7, 2025
Open-LLM-Leaderboard for dummies Intermediate	2	598	December 30, 2024
Evaluate fine-tuned LLM for question answering Beginners	1	202	May 2, 2025

Evaluating my own model

Related topics