# Constrained Beam Search - Very Slow

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fconstrained-beam-search-very-slow%2F40004%3E%5C
**Category:** 🤗Transformers\
**Created:** [May 15, 2023, 10:48pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fconstrained-beam-search-very-slow%2F40004 "2023-05-15T22:48:52Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![agneet](https://avatars.discourse-cdn.com/v4/letter/a/958977/32.png) [@agneet](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fagneet)%5C
**Post date:** [May 15, 2023, 10:48pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fconstrained-beam-search-very-slow%2F40004%2F1 "2023-05-15T22:48:52Z")

</div>

Hello,  
I am trying out generation in the following manner –

```auto
with torch.no_grad():
            generated_tokens = model.generate(
            desc_tokens.to('cuda'),
            max_new_tokens=256,
            min_new_tokens=256,
            force_words_ids = constraint_tokens,
            num_beams=2
        )

```

This is unexpectedly taking a lot of time. Is there a known issue?

---

<div class="post-metadata">

**Author:** ![Linke](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/linke/32/28838_2.png) [@Linke](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FLinke)%5C
**Post date:** [June 30, 2024, 1:00pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fconstrained-beam-search-very-slow%2F40004%2F2 "2024-06-30T13:00:08Z")

</div>

In my case, the generation speed is highly effected by the number of constraint words, i.e., `len(constraint_tokens )`. You can try to remove the constraints or reduce the size of it and see if the speed is improved to check this factor.  
As for the reason, you can refer to this great [post](https://huggingface.co/blog/constrained-beam-search) about constrained beam search. In brief, large number of constraints means much larger beam size, and thus causes larger delay.