# Prohibit GPT-2 from generating some words on a condition
**URL:**
**Author:**  [@hfnlpmb](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fhfnlpmb)%5C
**Post date:** [March 21, 2021, 1:12pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fprohibit-gpt-2-from-generating-some-words-on-a-condition%2F4823%2F1 "2021-03-21T13:12:07Z")
Hello,
I have added to the GPT-2 vocabulary two special tokens (`[ss]`, `[se]`) and use it to generate sequences.
I want to prohibit GPT-2 from generating some words after having generated `[ss]` until it generates `[se]` . For example in the following sequence
> some words of the sequence [ss] here I want some words not to be generated [se] other words of the sequence.
in other words, I want between `[ss]` … `[se]` to exclude some words from the generation. As @deathcrush answered [here](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fexclude-words-from-gpt-2-generate%2F4741%2F2) it is possible to prohibit GPT-2 from generating some vocabulary ids in general, but is it possible to do it on condition? (only between those tokens `[ss]` and `[se]`)
---
**Author:**  [@patrickvonplaten](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fpatrickvonplaten)%5C
**Post date:** [March 21, 2021, 2:04pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fprohibit-gpt-2-from-generating-some-words-on-a-condition%2F4823%2F2 "2021-03-21T14:04:05Z")
Could you check out this function for contained prefix generation: [Models — transformers 4.4.2 documentation](https://huggingface.co/transformers/main_classes/model.html?highlight=prefix_allowed_tokens_fn)
---
**Author:**  [@hfnlpmb](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fhfnlpmb)%5C
**Post date:** [March 21, 2021, 4:03pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fprohibit-gpt-2-from-generating-some-words-on-a-condition%2F4823%2F3 "2021-03-21T16:03:21Z")
Thank you @patrickvonplaten for your fast reply.
The link you provided is for the models page. Could you point me to the function you are talking about?
---
**Author:**  [@hfnlpmb](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fhfnlpmb)%5C
**Post date:** [March 22, 2021, 7:38pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fprohibit-gpt-2-from-generating-some-words-on-a-condition%2F4823%2F4 "2021-03-22T19:38:29Z")
@patrickvonplaten I saw that `generate` function has a `prefix_allowed_tokens_fn`. Were you referring to this?
---
**Author:**  [@prajjwal1](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fprajjwal1)%5C
**Post date:** [March 23, 2021, 2:44am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fprohibit-gpt-2-from-generating-some-words-on-a-condition%2F4823%2F5 "2021-03-23T02:44:56Z")
Yes, that is what he meant.
---
**Author:**  [@hfnlpmb](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fhfnlpmb)%5C
**Post date:** [March 23, 2021, 9:06pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fprohibit-gpt-2-from-generating-some-words-on-a-condition%2F4823%2F6 "2021-03-23T21:06:01Z")
This function is used to constrain _beam search_. I want to generate text with _sampling_ and specifically with `top_p`. Is there a way to utilise this function?
---
**Author:**  [@hfnlpmb](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fhfnlpmb)%5C
**Post date:** [March 29, 2021, 5:47pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fprohibit-gpt-2-from-generating-some-words-on-a-condition%2F4823%2F7 "2021-03-29T17:47:06Z")
@patrickvonplaten in the documentation of [generate](https://huggingface.co/transformers/main_classes/model.html?highlight=prefix_allowed_tokens_fn#transformers.generation_utils.GenerationMixin.generate) function the description of `prefix_allowed_tokens_fn` argument says
> If provided, this function constraints the beam search to allowed tokens only at each step.
Can this function be applied with sampling as well?
---
**Author:**  [@patrickvonplaten](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fpatrickvonplaten)%5C
**Post date:** [April 25, 2021, 6:03pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fprohibit-gpt-2-from-generating-some-words-on-a-condition%2F4823%2F8 "2021-04-25T18:03:02Z")
> [@hfnlpmb](#):
>
> t `generate` function has a `prefix_allowed_tokens_fn` . Were you referring to this?
Hey @hfnlpmb,
yes the function can be used in combination with `do_sample=True`