microsoft

/

Phi-3-vision-128k-instruct

like 971

Follow

Microsoft 20.3k

https://github.com/2U1/Phi3-Vision-Finetune/blob/main/src/merge_lora_weights.py

You can use this code for merging it.

https://github.com/2U1/Phi3-Vision-Finetune/blob/main/src/merge_lora_weights.py

You can use this code for merging it.

Thank you. But I made fine-tuning with Microsoft fine-tuning script and unfortunately you merging script fails on my lora_config.json from fine-tuning output dir.

https://github.com/2U1/Phi3-Vision-Finetune/blob/main/src/merge_lora_weights.py

You can use this code for merging it.

Thank you. But I made fine-tuning with Microsoft fine-tuning script and unfortunately you merging script fails on my lora_config.json from fine-tuning output dir.

Oh I think the loading script should be fix for that one. Can I see the file list that is made from the training script? I'll try to fix and post the code here.

Oh I think the loading script should be fix for that one. Can I see the file list that is made from the training script? I'll try to fix and post the code here.

I have these files after fine-tuning:

adapter_config.json
image_embedding_phi3_v.py
special_tokens_map.json
adapter_model.safetensors
image_processing_phi3_v.py
tokenizer_config.json
configuration_phi3_v.py
modeling_phi3_v.py
tokenizer.json
eval_after.json
preprocessor_config.json
training_args.bin
eval_before.json
processing_phi3_v.py
generation_config.json
processor_config.json

Thanks in advance.

In your case you could just use the Automodel class.

import torch
from transformers import AutoModelForCausalLM, AutoProcessor
from peft import PeftModel
from accelerate import Accelerator

model = AutoModelForCausalLM.from_pretrained('microsoft/Phi-3-vision-128k-instruct', low_cpu_mem_usage=True, trust_remote_code=True, torch_dtype=torch.float16)
processor = AutoProcessor.from_pretrained('microsoft/Phi-3-vision-128k-instruct', trust_remote_code=True)

print('Loading LoRA weights...')
model = PeftModel.from_pretrained(model, model_path)

print('Merging LoRA weights...')
model = model.merge_and_unload()

print('Model Loaded!!!')


accel = Accelerator()
# You could set the shard size whatever you want
accel.save_model(model, save_model_path, max_shard_size = '5GB')
model.config.save_pretrained(save_model_path)
processor.save_pretrained(save_model_path)

You could use this like this. I changed a littlebit and tested in my code and it works. But I've tested in my structure of my directory (my repo), so I think you should change a little bit like some arguments for loading the model or model_path.

You could use this like this. I changed a littlebit and tested in my code and it works. But I've tested in my structure of my directory (my repo), so I think you should change a little bit like some arguments for loading the model or model_path.

I just added some missing .py files from base model and it's working!
Thank you very much.