# What does the \`use\_cache\` in \`generate\` actually do?

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fwhat-does-the-use-cache-in-generate-actually-do%2F38855%3E%5C
**Category:** 🤗Transformers\
**Created:** [May 6, 2023, 3:05am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fwhat-does-the-use-cache-in-generate-actually-do%2F38855 "2023-05-06T03:05:16Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![nasheed](https://avatars.discourse-cdn.com/v4/letter/n/ea666f/32.png) [@nasheed](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fnasheed)%5C
**Post date:** [May 6, 2023, 3:05am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fwhat-does-the-use-cache-in-generate-actually-do%2F38855%2F1 "2023-05-06T03:05:16Z")

</div>

This is an offshoot from [Why past\_key\_values is not in GreedySearchDecoderOnlyOutput? - 🤗Transformers - Hugging Face Forums](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fwhy-past-key-values-is-not-in-greedysearchdecoderonlyoutput%2F23843)

What does the `use_cache` in `generate` actually do? Since there is no way to provide the `past_key_values` to the `generate` function, does that mean that generate internally stores `past_key_values` between runs (say I run `generate` in a loop for a chatbot)?

---

<div class="post-metadata">

**Author:** ![joaogante](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/joaogante/32/20106_2.png) [@joaogante](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fjoaogante)%5C
**Post date:** [May 9, 2024, 3:28pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fwhat-does-the-use-cache-in-generate-actually-do%2F38855%2F2 "2024-05-09T15:28:33Z")

</div>

@nasheed

You can find the answer [here](https://huggingface.co/docs/transformers/v4.40.2/en/llm_tutorial_optimization#321-multi-round-conversation) 🤗

TL;DR: pass `return_dict_in_generate=True` to `generate` → `generate` returns `past_key_values` → pass `past_key_values=past_key_values` to `generate`