# Slower train with collator for completion only

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fslower-train-with-collator-for-completion-only%2F56600%3E%5C
**Category:** 🤗Transformers\
**Created:** [September 27, 2023, 4:10pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fslower-train-with-collator-for-completion-only%2F56600 "2023-09-27T16:10:12Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![miguelwon](https://avatars.discourse-cdn.com/v4/letter/m/858c86/32.png) [@miguelwon](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fmiguelwon)%5C
**Post date:** [September 27, 2023, 4:10pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fslower-train-with-collator-for-completion-only%2F56600%2F1 "2023-09-27T16:10:12Z")

</div>

Hi,

I’m trying to fine tune Llama 2 and I don’t want to fine tune in full (instructions + completion), only the completion. Because of that I’m using the `DataCollatorForCompletionOnlyLM` collator.

The completion of by training dataset is a very small sentence and so I was expecting a faster training, because the training would be resumed to predict the next token of completion. Is this what is happing when we use `DataCollatorForCompletionOnlyLM` collator? The problem is that I’m not measuring any improvement in speed. Actually, I think it got slower.

Any comment on this would very appreciated.  
Thanks

---

<div class="post-metadata">

**Author:** ![silcowitz](https://avatars.discourse-cdn.com/v4/letter/s/ebca7d/32.png) [@silcowitz](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsilcowitz)%5C
**Post date:** [April 7, 2024, 6:42pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fslower-train-with-collator-for-completion-only%2F56600%2F2 "2024-04-07T18:42:11Z")

</div>

Did you find an answer to this?