# Fine tune GPT2/LLaMA in seq2seq manner
**URL:**
**Author:**  [@cardcounter](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fcardcounter)%5C
**Post date:** [October 23, 2023, 8:21am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffine-tune-gpt2-llama-in-seq2seq-manner%2F59522%2F1 "2023-10-23T08:21:21Z")
With GPT2/LLaMA, by default, we need to input the [prompt label] the whole sentence `model([prompt label])` in fine-tuning and caculate the CrossEntropy on the label part, and the model output the model().logits.
**Are there any ways to input the prompt only and do the fine-tuning in the seq2seq manner ?** (`model(prompt)`), this way we minimize the loss of log p(y|x).
---
**Author:**  [@as3eem](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fas3eem)%5C
**Post date:** [January 14, 2024, 11:29am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffine-tune-gpt2-llama-in-seq2seq-manner%2F59522%2F2 "2024-01-14T11:29:51Z")
Commenting to follow this thread.
---
**Author:**  [@nielsr](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fnielsr)%5C
**Post date:** [January 14, 2024, 7:17pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffine-tune-gpt2-llama-in-seq2seq-manner%2F59522%2F3 "2024-01-14T19:17:50Z")
Hi,
I assume you mean you want to only train the model on the completions, rather than the instructions (prompts)? This is supported by the [DataCollatorForCompletionOnlyLM](https://huggingface.co/docs/trl/sft_trainer#train-on-completions-only) collator in the TRL library. You can use it in combination with the `SFTTrainer` in order only train the model on the completions.