Correct Samplers Order?
The Readme recommends:
- temperature = 1.0
- top_p = 0.95
- top_k = 64
But HF Transformers applies them in different order:
- temperature
- top_k
- top_p
Llama.cpp applies them in yet another different order unless --samplers argument is used (which no one does):
- top_k
- top_p
- temperature
All that changes the model behavior.
Which order is correct?
Thanks.
With the README's settings the order doesn't actually matter. At temperature 1.0 the temperature step is a no-op, and both orders apply top-k before top-p. HF's temp → top_k → top_p and llama.cpp's top_k → top_p → temp therefore keep the same candidates with the same probabilities.
Order only matters once temperature ≠ 1. HF tempers first, so the 0.95 nucleus is computed on the sharpened (T < 1) distribution and keeps fewer tokens. llama.cpp picks the nucleus at T = 1 and tempers afterwards, so it keeps more.
Two settings do differ in practice:
- llama.cpp also applies min_p 0.05 by default. Pass
--min-p 0to match HF. - For tool calls, keep repeat_penalty at 1.0. A penalty above 1 acts on the schema tokens a call must repeat. In our testing it turned sampled tool calls into garbage, which is why Xyntetik Runner's default is now 1.0.
One data point on sampling for agentic work: on a 14B student distilled from Glimmer, sampling inside the reasoning turn (temp 0.6, top_p 0.9, min_p 0.05, top_k 20) solved 49 and 47 of 60 held-out tool tasks on two seeds, against 51 greedy. Measured on our student, not on Glimmer itself.