# Issue on Kosmos-2 model training on new dataset
**URL:**
**Author:**  [@Mit1208](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FMit1208)%5C
**Post date:** [February 10, 2024, 3:39pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fissue-on-kosmos-2-model-training-on-new-dataset%2F72653%2F1 "2024-02-10T15:39:55Z")
Hi @ydshieh,
I hope you are well. I am trying to train Kosmos model on DoclayNet data. I have prepared data. I am getting an error in Training.
My process for training is like this:
1. Convert data into kosmos-2 format.
2. Convert it to number using processor using below code:
```auto
inputs = processor(images = test2_df['image'].to_list(), text = test2_df['text'].to_list(), bboxes = test2_df['float_val'].to_list(),padding=True, return_tensors="pt").to(device)
dataset = Dataset.from_dict(inputs)
```
1. Split dataset into train and test and using Trainer for train like this:
```auto
Trainer(
model=model,
args=training_args,
train_dataset=train_dataset,
eval_dataset=test_dataset,
tokenizer=processor,
data_collator=default_data_collator,
)
```
1. Train model
I am getting error like this:
```auto
/usr/local/lib/python3.10/dist-packages/transformers/models/kosmos2/modeling_kosmos2.py in forward_embedding(self, input_ids, inputs_embeds, image_embeds, img_input_mask, past_key_values_length, position_ids)
1150 print(image_embeds)
1151 if image_embeds is not None:
-> 1152 inputs_embeds[img_input_mask.to(dtype=torch.bool)] = image_embeds.to(inputs_embeds.device).view(
1153 -1, image_embeds.size(-1)
1154 )
RuntimeError: Index put requires the source and destination dtypes match, got Float for the destination and Half for the source.
```
---
**Author:**  [@Mit1208](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FMit1208)%5C
**Post date:** [February 25, 2024, 2:27pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fissue-on-kosmos-2-model-training-on-new-dataset%2F72653%2F2 "2024-02-25T14:27:10Z")
Hi @ydshieh
Can you please advice to resolve this error?
Thanks.
---
**Author:**  [@Mit1208](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FMit1208)%5C
**Post date:** [February 25, 2024, 10:27pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fissue-on-kosmos-2-model-training-on-new-dataset%2F72653%2F3 "2024-02-25T22:27:49Z")
The problem was in my TrainingArguments(). I set `fp16=True`, which was messing up tensors dtypes. I removed it from the argument and it worked.
---
**Author:**  [@system](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsystem)%5C
**Post date:** [February 26, 2024, 10:28am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fissue-on-kosmos-2-model-training-on-new-dataset%2F72653%2F4 "2024-02-26T10:28:17Z")
Closed on February 26, 2024, 10:28am UTC
This topic was automatically closed 12 hours after the last reply. New replies are no longer allowed.