Generate tokens given hidden states

zekeZZ August 28, 2023, 10:59pm 1

Is there a way to let HF generate tokens based on the last hidden states?

Topic		Replies	Views
Get each generated token last layer hidden state 🤗Transformers	3	120	March 16, 2025
Avoid recalculating hidden states between generate calls? 🤗Transformers	3	1267	March 30, 2023
Streaming token output from models like T5 🤗Transformers	7	12340	June 7, 2023
BLIP2 GreedySearchDecoderOnlyOutput, how can I extract the activations of a certain hidden layer? 🤗Transformers	0	153	July 5, 2023
T5.generate() cannot get hidden states although output_hidden_states=True 🤗Transformers	0	576	July 9, 2022