# Reasoning LLM Benchmarking
**URL:**
**Author:**  [@rgb2gbr](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Frgb2gbr)%5C
**Post date:** [March 24, 2025, 11:11am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Freasoning-llm-benchmarking%2F147171%2F1 "2025-03-24T11:11:19Z")
I’m training/fine-tuning a [Qwen](https://huggingface.co/Qwen/Qwen2.5-3B-Instruct)model using [GRPO](https://huggingface.co/docs/trl/main/en/grpo_trainer)(Group Relative Policy Optimisation) it performs well and answers nearly all the [GSM8K](https://huggingface.co/datasets/openai/gsm8k)test questions correctly.
But for the benchmarking, traditional ways/scripts wont work, since it just looks for numeric value but the output is verbose, a reasoning step by step result, thus the benchmark identifies the output incorrectly.
how they’d do the benchmarking because a script only goes through the answer and looking for a numeric value, it cannot understand the correct answer because most of the time its a reasoning or verbose output
is there a legit way to measure/evaluate and benchmark the the reasoning models?
---
**Author:**  [@John6666](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FJohn6666)%5C
**Post date:** [March 24, 2025, 11:48am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Freasoning-llm-benchmarking%2F147171%2F2 "2025-03-24T11:48:41Z")
Hmm, for example, could we use the benchmark for the reasoning model used in these leaderboards?
> **[Open LMM Reasoning Leaderboard - a Hugging Face Space by opencompass](https://huggingface.co/spaces/opencompass/Open_LMM_Reasoning_Leaderboard)**
>
> A Leaderboard that demonstrates LMM reasoning capabilities
> **[Open Multilingual Reasoning Leaderboard - a Hugging Face Space by kevinpro](https://huggingface.co/spaces/kevinpro/Open-Multilingual-Reasoning-Leaderboard)**
>
> Discover amazing ML apps made by the community
> **[NPHardEval Leaderboard: Unveiling the Reasoning Abilities of Large Language...](https://huggingface.co/blog/leaderboard-nphardeval)**
>
> We’re on a journey to advance and democratize artificial intelligence through open source and open science.
> **[ZebraLogic: Benchmarking the Logical Reasoning Ability of Language Models](https://huggingface.co/blog/yuchenlin/zebra-logic)**
>
> A Blog post by Bill Yuchen Lin on Hugging Face
---
**Author:**  [@rgb2gbr](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Frgb2gbr)%5C
**Post date:** [March 24, 2025, 12:16pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Freasoning-llm-benchmarking%2F147171%2F3 "2025-03-24T12:16:34Z")
thanks for the suggestion, the thing is need to regularly run multiple benchmark to see performance and improvement and i also need to demonstrate a well-known method