# Kosmos-2 Fine tuning

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fkosmos-2-fine-tuning%2F75691%3E%5C
**Category:** 🤗Transformers\
**Created:** [March 3, 2024, 12:54pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fkosmos-2-fine-tuning%2F75691 "2024-03-03T12:54:01Z")\
**Posts on this page:** 1\
**Showing post:** 5

<div class="post-metadata">

**Author:** ![Mit1208](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/mit1208/32/32368_2.png) [@Mit1208](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FMit1208)%5C
**Post date:** [March 7, 2024, 2:16am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fkosmos-2-fine-tuning%2F75691%2F5 "2024-03-07T02:16:47Z")

</div>

based on [Finetune BLIP on customer dataset #20893 - #2 by dxlong2000](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffinetune-blip-on-customer-dataset-20893%2F28446%2F2). I set labels as input\_ids in model  
e.g

```auto
class CustomTrainer(Trainer):
    def compute_loss(self, model, inputs, return_outputs=False):

        # outputs = model(**inputs)
        outputs = model(
            input_ids=inputs["input_ids"], pixel_values=inputs["pixel_values"],
            attention_mask=inputs["attention_mask"], image_embeds_position_mask=inputs["image_embeds_position_mask"],
            labels = inputs["input_ids"]
        )
        logits = outputs.logits
        # Ensure that 'eval_loss' is present in the metrics dictionary
        metrics = {'eval_loss': outputs.loss.item()}
        return (outputs.loss, outputs) if return_outputs else outputs.loss

```

---

_[View the full topic](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fkosmos-2-fine-tuning%2F75691)._