# Sampling: what's the secret sauce?
**URL:**
**Author:**  [@chrisdoyle](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fchrisdoyle)%5C
**Post date:** [August 8, 2022, 4:00pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fsampling-whats-the-secret-sauce%2F21384%2F1 "2022-08-08T16:00:27Z")
Just a practical question, `np.choices` is very slow to return a sample when one tries to sample from a large distribution - say, for example, a 52K token vocabulary.
How do HuggingFace’s implementations of sampling methods actually sample? Nucleus sampling makes sense to me because we only sample from the low entropy parts of the distribution, but for beam search which tries to find the max likelihood sequence over several samples, mightn’t we need to sample from close to the full distribution at any given step? How do we do this in practise??
---
**Author:**  [@joaogante](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fjoaogante)%5C
**Post date:** [August 19, 2022, 2:49pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fsampling-whats-the-secret-sauce%2F21384%2F2 "2022-08-19T14:49:40Z")
Hey @chrisdoyle 👋
We use PyTorch/TensorFlow/JAX sampling operations, which are optimized for GPU usage. See here, for example: [transformers/generation\_utils.py at e54a1b49aa6268c484625c6374f952f318914743 · huggingface/transformers · GitHub](https://github.com/huggingface/transformers/blob/e54a1b49aa6268c484625c6374f952f318914743/src/transformers/generation_utils.py#L2005)
Beam search does no sampling – it takes the beams/tokens with the highest score at each iteration, which is deterministic after running the model forward pass 🙂
---
**Author:**  [@chrisdoyle](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fchrisdoyle)%5C
**Post date:** [August 22, 2022, 8:52am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fsampling-whats-the-secret-sauce%2F21384%2F3 "2022-08-22T08:52:50Z")
Thanks @joaogante, I think I’ll find the answer I’m looking for inside the torch.multinomial implementation, many thanks!