# BART generate() output not related to input
**URL:**
**Author:**  [@furunkel](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Ffurunkel)%5C
**Post date:** [June 30, 2021, 2:02pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fbart-generate-output-not-related-to-input%2F7744%2F1 "2021-06-30T14:02:56Z")
Hi,
I’m in need of a model that can fill in multi-token masks, and it seems like BART is the best choice at the moment.
I’ve trained a BART model from scratch using a custom collator that masks out sequential groups of tokens.
Unfortunately, when using the `generate()` method on my `BartForConditionalGeneration` model, the model output is completely unrelated to the input (with or without mask tokens). The output clearly stems from the dataset distribution, but is not at all related to the input, let alone has filled masks.
Now, if I obtain the logits directly (i.e., `model(input_ids).logits`), the predictions for the token in the input look perfectly fine, so the model seems to have learned something.
The model was trained as follows:
Input: `This is `
Label: `This is some input`
I let the library generate the decoder input, it should look something like: `This is some input`.
I wonder if I’m missing something obvious here?
---
**Author:**  [@jbmaxwell](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fjbmaxwell)%5C
**Post date:** [February 17, 2022, 3:38am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fbart-generate-output-not-related-to-input%2F7744%2F2 "2022-02-17T03:38:01Z")
Did you make any progress with this? I’m also looking to pretrain BART from scratch for infill generation.