# Model.generate generates way too long outputs

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fmodel-generate-generates-way-too-long-outputs%2F54224%3E%5C
**Category:** 🤗Transformers\
**Created:** [September 9, 2023, 1:46pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fmodel-generate-generates-way-too-long-outputs%2F54224 "2023-09-09T13:46:47Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![Luan77777](https://avatars.discourse-cdn.com/v4/letter/l/ecc23a/32.png) [@Luan77777](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FLuan77777)%5C
**Post date:** [September 9, 2023, 1:46pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fmodel-generate-generates-way-too-long-outputs%2F54224%2F1 "2023-09-09T13:46:47Z")

</div>

Hi I trained a SpeechT5 model from scratch on an ASR task and i now want to use it for inference withe model.generate but somehow my generates ids are way longer than the labeld output… When i ran some metrics during the training with compute metrics, the predicted output length of the text by the model was nearly always the same for the labeled output, which worked very fine. But however with model.generate i actually only get putputs that are way too long, here is an example for when i use model.generate:

predicted output: [‘tempo:120 s1:f0 s2:f3 s3:f2 s4:f0 s5:f0 480 s1:f0 s2:f3 s3:f2 s4:f0 s5:f0 480 s1:f0 s2:f3 s3:f0 s4:f0 s5:f0 480 s1:f0 s2:f3 s3:f0 s4:f0 s5:f0 480 s1:f0 s2:f3 s3:f0 s4:f0 s5:f0 480 s1:f0 s2:f3 s3:f0 s4:f0 s5:f0 480 s1:f0 s2:f3 s3:f0 s4:f0 s5:f0 480 s1:f0 s2:f3 s3:f0 s4:f0 s5:f0 480 s1:f0 s2:f3 s3:f0 s4:f0 s5:f0 480 s1:f0 s2:f3 s3:f0 s4:f0 s5:f0 480 s1:f0 s2:f3 s3:f0 s4:f0 s5:f0 480 s1:f0 s2:f3 s3:f0 s4:f0 s5:f0 480 s1:f0 s2:f3 s3:f0 s4:f0 s5:f0 480 s1:f0 s2:f3 s3:f0 s4:f0 s5:f0 480 s1:f0 s2:f3 s3:f0 s4:f0 s5:f0 480 s1:f0 s2:f3 s3:f0 s4:f0 s5:f0 480 s1:f0 s2:f3 s3:f0 s4:f0 s5:f0 480 s1:f0 s2:f3 s3:f0 s4:f0 s5:f0 480 s1:f0 s2:f3 s3:f0 s4:f0 s5:f0 480 s1:f0 s2:f3 s3:f0 s4:f0 s5:f0 480 s1:f0 s2:f3 s3:f0 s4:f0 s5:f0 480 s1:f0 s2:f3 s3:f0 s4:f0 s5:f0 480 s1:f0 s2:f3 s3:f0 s4:f0 s5:f0 480 s1:f0 s2:f3 s3:f0 s4:f0 s5:f0 480 s1:f0 s2:f3 s3:f0 s4:f0 s5:f0 480 s1:f0 s2:f0 s3:f0 s4:f0 s5:f0 480 s1:f0 s2:f0 s3:f0 s4:f0 s5:f0 480 s1:f0 s2:f0 s3:f0 s4:f0 s5:f0 480 s1:f0 s2:f0 s3:f0 s4:f0 s5:f0 480 s1:f0 s2:f0 s3:f0 s4:f0 s5:f0 480 s1:f0 s2:f0 s3:f0 s4:f0 s5:f0 480 s1:f0 s2:f0 s3:f0 s4:f0 s5:f0 480 s1:f0 s2:f0 s3:f0 s4:f0 s5:f0 480 s1:f0 s2:f0 s3:f0 s4:f0 s5:f0 480 s1:f0 s2:f0 s3:f0 s4:f0 s5:f0 480 s1:f0 s2:f0 s3:f0 s4:f0 s5:f0 480 s1:f0 s2:f0 s3:f0 s4:f0 s5:f0 480 s1:f0 s2:f0 s3:f0 s4:f0 s5:f0 480 s1:f0 s2:f0 s3:f0 s4:f0 s5:f0 480 s1:f0 s2:f0 s3:f0 s4:f0 s5:f0 480 s1:f0 s2:f0 s3:f0 s4:f0 s5:f0 480 s1:f0 s2:f0 s3:f0 s4:f0 s5:f0 480 s1:f0 s2:f0 s3:f0 s4:f0 s5:f0 480 s1:f0 s2:f0 s3:f0 s4:f0 s5:f0 480 s1:f0 s2:f0 s3:f0 s4:f0 s5:f0 480 s1:f0 s2:f0 s3:f0 s4:f0 s5:f0 480 s1:f0 s2:f0 s3:f0 s4:f0 s5:f0 480 s1:f0 s2:f0 s3:f0 s4:f0 s5:f0 480 s1:f0 s2:f0 s3:f0 s4:f0 s5:f0 480 s1:f0 s2:f0 s3:f0 s4:f0 s5:f0 480 s1:f0 s2:f0 s3:f0 s4:f0 s5:f0 480 s1:f0 s2:f0 s3:f0 s4:f0 s5:f0 480 s1:f0 s2:f0 s3:f0 s4:f0 s5:f0 480 s1:f0 s2:f0 s3:f0 s4:f0 s5:f0 480 s1:f0 s2:f0 s3:f0 s4:f0 s5:f0 480 s1:f0 s2:f0 s3:f0 s4:f0 s5:f0 480 s1:f0 s2:f0 s3:f0 s4:f0 s5:f0 480 s1:f0 s2:f0 s3:f0 s4:f0 s5:f0 480 s1:f0 s2:f0 s3:f0 s4:f0 s5:f0 480 s1:f0 s2:f0 s3:f0 s4:f0 s5:f0 480 s1:f0 s2:f0 s3:f0 s4:f0 s5:f0 480 s1:f0 s2:f0 s3:f0 s4:f0 s5:f0 480 s1:f0 s2:f0 s3:f0 s4:f0 s5:f0 480 s1:f0 s2:f0 s3:f0 s4:f0 s5:f0 480 s1:f0 s2:f0 s3:f0 s4:f0 s5:f0 480 s1:f0 s2:f0 s3:f0 s4:f0 s5:f2 480 s1:f0 s2:f0 s3:f0 s4:f0 s5:f0 480 s1:f0 s2:f0 s3:f0 s4:f0 s5:f0 480 s1:f0 s2:f0 s3:f0 s4:f0 s5:f3 480 s1:f0 s2:f0 s3:f0 s4:f0 s5:f3 480 s1:f0 s2:f0 s3:f0 s4:f0 s5:f0 480 s1:f0 s2:f0 s3:f0 s4:f0 s5:f0 480 s1:f0 s2:f0 s3:f0 s4:f0 s5:f0 480 s1:f0 s2:f0 s3:f0 s4:f0 s5:f3 480 s1:f0 s2:f0 s3:f0 s4:f0 s5:f3 480 s1:f0 s2:f0 s3:f0 s4:f0 s5:f3 480 s1:f0 s2:f0 s3:f0 s4:f0 s5:f3 480 s1:f0 s2:f0 s3:f0 s4:f0 s5:f0 480 s1:f0 s2:f0 s3:f0 s4:f0 s5:f3 480 s1:f0 s2:f1 s3:f0 s4:f0 s5:f3 480 s1:f0 s2:f1 s3:f0 s4:f0 s5:f3 480 s1:f0 s2:f1 s3:f0 s4:f0 s5:f3 480 s1:f0 s2:f0 s3:f0 s4:f0 s5:f3 480’],

labeled output: tempo:120 s2:f3 s3:f2 s4:f0 960 s4:f3 480 s4:f4 480 s2:f3 s3:f2 s4:f0 1920 s2:f0 s3:f0 s4:f0 3840 s6:f3 480 s4:f0 480 s3:f0 480 s5:f0 960 s3:f0 480 s5:f2 480 s3:f0 480 s5:f3 480 s3:f0 480 s3:f0 480 s5:f2 960 s3:f0 480 s6:f3 480 s3:f0 480 s3:f2 s4:f0 960 s3:f2 s4:f0 480 s3:f4 s4:f0 480 rest 480 s3:f2 s4:f0 480 rest 480 s3:f4 s4:f0 480

As you can see the predicted text by the model is much longer than the output it should have predicted (labeled output), during training the length was nearly always the same for every output… This is my code that I use for inference:

import sys  
from datasets import load\_dataset\_builder, load\_dataset, dataset\_dict, Dataset, load\_metric  
from transformers import Seq2SeqTrainer, Speech2TextForConditionalGeneration, Speech2TextConfig, Speech2TextProcessor,   
Speech2TextTokenizer, Speech2TextFeatureExtractor, trainer\_utils,   
Trainer, DataCollatorWithPadding, Speech2TextModel, TrainingArguments, Seq2SeqTrainingArguments, DataCollatorForSeq2Seq,   
SpeechT5Processor, SpeechT5ForSpeechToText, Seq2SeqTrainingArguments, Seq2SeqTrainer, SpeechT5Config, SpeechT5FeatureExtractor, SpeechT5Tokenizer,   
SequenceFeatureExtractor  
import torch  
import sentencepiece as spm  
import numpy as np  
import tensorboard  
from jiwer import wer  
from sklearn.metrics import f1\_score  
import psutil  
from dataclasses import dataclass, field  
from typing import Any, Dict, List, Optional, Union  
from datasets import Audio  
import evaluate

tokenizer = SpeechT5Tokenizer(vocab\_file=r"/home/ec2-user/SageMaker/Tokenizer/word\_nur\_Zahlen.model")

feature\_extractor = SpeechT5FeatureExtractor(return\_attention\_mask=False, do\_normalize=True)

processor = SpeechT5Processor(feature\_extractor=feature\_extractor, tokenizer=tokenizer)

test\_dataset = load\_dataset(‘audiofolder’, data\_dir=r"/home/ec2-user/SageMaker/Datasets/Dataset Folder 2 Splitted/Test")  
test\_dataset = test\_dataset[‘train’]  
print(“Test Data loaded:”, test\_dataset)

test\_dataset = test\_dataset.cast\_column(“audio”, Audio(sampling\_rate=16000))  
print(“Test Data features:”, test\_dataset.features)

model = SpeechT5ForSpeechToText.from\_pretrained(r"/home/ec2-user/SageMaker/SpeechT5/Model 3 Word 10Sek 16Ep/Model Save")

print(model.can\_generate)

input\_ids = processor(audio=test\_dataset[“audio”][0][“array”], sampling\_rate=16000, return\_tensors=“pt”)

print(len(test\_dataset[“audio”][0][“array”]))

print(input\_ids)

predicted\_ids = model.generate(\*\*input\_ids, max\_length=500)

print(predicted\_ids)

transcription = processor.batch\_decode(predicted\_ids, skip\_special\_tokens=True)

print(transcription)

print(test\_dataset[0][‘transcription’])

Any help would be really apprecaited!