# kohya\_SS (Output Interpretation)
**URL:**
**Author:**  [@deicool](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fdeicool)%5C
**Post date:** [February 20, 2025, 9:29am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fkohya-ss-output-interpretation%2F141979%2F1 "2025-02-20T09:29:55Z")
Hello
I have trained the kohya\_ss model (stabilityai/stable-diffusion-xl-base-1.0) with 10 images. I was wondering where the output comes from (from the base model or my customized training).
How much % is the final output composed of ?
Eg:
(Base Model:60%, Customized Training:40%)
(Base Model:70%, Customized Training:30%)
For example:
The prompt is: DNA has to be shown in the background with a Indain-Woman-with-Mouth-Cancer in the Foreground
And the image created by the program is:
The program is:
```auto
from diffusers import AutoPipelineForText2Image, AutoencoderKL
import torch
import os
import numpy as np
from PIL import Image
print("vae")
# Clear GPU memory before starting
torch.cuda.empty_cache()
# Set seed for reproducibility
#torch.manual_seed(6666666)
#np.random.seed(6666666)
# Define the path to the directory containing your model and LoRA weights
print("Define the path to the directory containing your model and LoRA weights")
model_dir = "D:\\Ganu\\AIImage\\huggingface\\kohya_ss\\kohya_ss\\trained-model\\model\\"
lora_weights_path = os.path.join(model_dir, "last.safetensors")
# Load the base model using StableDiffusionPipeline
print("Load the base model using StableDiffusionPipeline")
model_id = "stabilityai/stable-diffusion-xl-base-1.0"
adapter_id = "wangfuyun/PCM_SDXL_LoRAs"
#vae = AutoencoderKL.from_pretrained("madebyollin/sdxl-vae-fp16-fix", torch_dtype=torch.float16)
pipeline = AutoPipelineForText2Image.from_pretrained(model_id, torch_dtype=torch.float32, variant="fp16").to("cpu")
pipeline.enable_sequential_cpu_offload()
pipeline.enable_attention_slicing("max")
# Load the LoRA weights
print("Load the LoRA weights")
try:
pipeline.load_lora_weights(lora_weights_path, weight_name="last.safetensors")
except ValueError as e:
print("Invalid LoRA checkpoint. Please check the compatibility and format of the weights file.")
raise e
# Generate an image from a text prompt
print("Generate an image from a text prompt")
text_prompt = "DNA has to be shown in the background with a Indain-Woman-with-Mouth-Cancer in the Foreground"
generated_image = pipeline(prompt=text_prompt).images[0]
generated_image.save("generated_image.png")
generated_image.show()
```
---
**Author:**  [@John6666](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FJohn6666)%5C
**Post date:** [February 20, 2025, 1:46pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fkohya-ss-output-interpretation%2F141979%2F2 "2025-02-20T13:46:49Z")
Good evening. That question is essentially impossible to answer…😅
The answer would be something like “it depends on the base model”, “it depends on what you want to express with LoRA (if it’s something like the characteristics of a person or a character, then LoRA will have a big impact)”, or “it can’t be expressed as a percentage in the first place”.
This is because the base model and LoRA are **fused together** when inference is executed. The mixed neural network is no longer suitable for being expressed as a percentage.
LoRA is not the same as full fine tuning, but it is one of the methods for training models, and there are various LoRA algorithms, each with their own strengths and weaknesses. (I am not familiar with each algorithm.)
> **[LoRA](https://huggingface.co/docs/peft/main/en/conceptual_guides/lora)**
>
> We’re on a journey to advance and democratize artificial intelligence through open source and open science.
> **[LoRA vs Full Fine-tuning: An Illusion of Equivalence](https://arxiv.org/abs/2410.21228)**
>
> Fine-tuning is a crucial paradigm for adapting pre-trained large language models to downstream tasks. Recently, methods like Low-Rank Adaptation (LoRA) have been shown to match the performance of fully fine-tuned models on various tasks with an...
---
**Author:**  [@deicool](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fdeicool)%5C
**Post date:** [February 21, 2025, 7:22am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fkohya-ss-output-interpretation%2F141979%2F3 "2025-02-21T07:22:13Z")
Hello
Can I get the last.safetensors weights file (for the model: stabilityai/stable-diffusion-xl-base-1.0) without my customized training (the original one)? So I can check the difference from my customized training?
---
**Author:**  [@John6666](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FJohn6666)%5C
**Post date:** [February 21, 2025, 8:31am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fkohya-ss-output-interpretation%2F141979%2F4 "2025-02-21T08:31:11Z")
Hmmm? How do you want it to be?🤔
---
**Author:**  [@deicool](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fdeicool)%5C
**Post date:** [February 21, 2025, 8:32am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fkohya-ss-output-interpretation%2F141979%2F5 "2025-02-21T08:32:50Z")
Sorry, didn’t get your question?
---
**Author:**  [@John6666](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FJohn6666)%5C
**Post date:** [February 21, 2025, 8:38am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fkohya-ss-output-interpretation%2F141979%2F6 "2025-02-21T08:38:18Z")
Yea. I didn’t understand it very well. I think you want to do something for comparison…
---
**Author:**  [@deicool](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fdeicool)%5C
**Post date:** [February 21, 2025, 8:42am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fkohya-ss-output-interpretation%2F141979%2F7 "2025-02-21T08:42:17Z")
When I do training with kohya\_ss (LORA), it generates a (last.safetensors) file which I use for image generation.
What I want is a original file (last.safetensors) without the changes done due to my training?
---
**Author:**  [@deicool](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fdeicool)%5C
**Post date:** [February 21, 2025, 9:01am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fkohya-ss-output-interpretation%2F141979%2F8 "2025-02-21T09:01:34Z")
For example, the following code:
```auto
from diffusers import AutoPipelineForText2Image, AutoencoderKL
import torch
import os
import numpy as np
from PIL import Image
print("vae")
# Clear GPU memory before starting
torch.cuda.empty_cache()
# Set seed for reproducibility
#torch.manual_seed(6666666)
#np.random.seed(6666666)
# Define the path to the directory containing your model and LoRA weights
print("Define the path to the directory containing your model and LoRA weights")
model_dir = "D:\\Ganu\\AIImage\\huggingface\\kohya_ss\\kohya_ss\\trained-model\\model\\"
lora_weights_path = os.path.join(model_dir, "last.safetensors")
# Load the base model using StableDiffusionPipeline
print("Load the base model using StableDiffusionPipeline")
model_id = "stabilityai/stable-diffusion-xl-base-1.0"
adapter_id = "wangfuyun/PCM_SDXL_LoRAs"
#vae = AutoencoderKL.from_pretrained("madebyollin/sdxl-vae-fp16-fix", torch_dtype=torch.float16)
pipeline = AutoPipelineForText2Image.from_pretrained(model_id, torch_dtype=torch.float32, variant="fp16").to("cpu")
pipeline.enable_sequential_cpu_offload()
pipeline.enable_attention_slicing("max")
# Load the LoRA weights
print("Load the LoRA weights")
try:
pipeline.load_lora_weights(lora_weights_path, weight_name="last.safetensors")
except ValueError as e:
print("Invalid LoRA checkpoint. Please check the compatibility and format of the weights file.")
raise e
# Generate an image from a text prompt
print("Generate an image from a text prompt")
text_prompt = "DNA has to be shown in the background, and a Indain Woman with Skin Disease in the Foreground"
generated_image = pipeline(prompt=text_prompt).images[0]
generated_image.save("generated_image.png")
generated_image.show()
```
generates the image:
Whereas the following code:
```auto
from diffusers import AutoPipelineForText2Image, AutoencoderKL
import torch
import os
import numpy as np
from PIL import Image
print("vae")
# Clear GPU memory before starting
torch.cuda.empty_cache()
# Set seed for reproducibility
#torch.manual_seed(6666666)
#np.random.seed(6666666)
# Load the base model using StableDiffusionPipeline
print("Load the base model using StableDiffusionPipeline")
model_id = "stabilityai/stable-diffusion-xl-base-1.0"
adapter_id = "wangfuyun/PCM_SDXL_LoRAs"
#vae = AutoencoderKL.from_pretrained("madebyollin/sdxl-vae-fp16-fix", torch_dtype=torch.float16)
pipeline = AutoPipelineForText2Image.from_pretrained(model_id, torch_dtype=torch.float32, variant="fp16").to("cpu")
pipeline.enable_sequential_cpu_offload()
pipeline.enable_attention_slicing("max")
# Generate an image from a text prompt
print("Generate an image from a text prompt")
text_prompt = "DNA has to be shown in the background, and a Indain Woman with Skin Disease in the Foreground"
generated_image = pipeline(prompt=text_prompt).images[0]
generated_image.save("generated_image.png")
generated_image.show()
```
generates the following image:
The two images generated are very different.
I was wondering why…
---
**Author:**  [@John6666](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FJohn6666)%5C
**Post date:** [February 21, 2025, 10:10am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fkohya-ss-output-interpretation%2F141979%2F9 "2025-02-21T10:10:49Z")
> The two images generated are very different.
I think this is because the latter code does not apply last.safetensors (LoRA). Also, if you want to keep both the pre-training and post-training models in KohyaSS, you need to specify an option…
>
>
> Supposed I have done the Lora training, but the result wasn't as expected, a bit… of under-fitting. My question is how can I continue the training basing on current Lora result without starting it over from beginning. BTW, I saved Lora as safetensor format. Should I use --resume or sth?
>
> Another similar question for classic fine tune, regarding the same challenge, should I just change model.safetensors to diffuser format then point "pretrained\_model\_name\_or\_path" to the directory of diffusers files, and continue my training?
>
> Thanks for help.
---
**Author:**  [@deicool](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fdeicool)%5C
**Post date:** [March 1, 2025, 6:18am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fkohya-ss-output-interpretation%2F141979%2F10 "2025-03-01T06:18:15Z")
Hello,
I am getting great images from the program **without LORA**. So if I want to retain the core design (without LORA) and then apply my LORA fine-tuning on it to apply cosmetic changes (all in one go!), how can I achieve that?
Please advise. Thank You.
---
**Author:**  [@John6666](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FJohn6666)%5C
**Post date:** [March 1, 2025, 9:09am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fkohya-ss-output-interpretation%2F141979%2F11 "2025-03-01T09:09:16Z")
Good evening.😀
I see. You want to train and apply LoRA to the extent that it doesn’t erase the goodness of the base model.
One way to do this is to lower the weight (scale) below 1.0 when applying LoRA that has already been trained.
Another way is to specify, using parameters, how much of the training data to include in the training using LoRA. In the case of KohyaSS, the parameters are as follows.
### When applying LoRA
[https://huggingface.co/docs/diffusers/main/en/tutorials/using\_peft\_for\_inference#merge-adapters](https://huggingface.co/docs/diffusers/main/en/tutorials/using_peft_for_inference#merge-adapters)
### When training LoRA
[https://github.com/kohya-ss/sd-scripts/pull/545](https://github.com/kohya-ss/sd-scripts/pull/545)
> **[LoRA training parameters](https://github.com/bmaltais/kohya_ss/wiki/LoRA-training-parameters)**
>
> Contribute to bmaltais/kohya\_ss development by creating an account on GitHub.
> **[Essential to Advanced Guide to training a LoRA | Civitai](https://civitai.com/articles/3105/essential-to-advanced-guide-to-training-a-lora)**
>
> 1. Introduction "Fear the curses that hide in your training" - Disclaimer: I won't teach you to make images like this one, don't worry. This aims t...
---
**Author:**  [@deicool](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fdeicool)%5C
**Post date:** [March 4, 2025, 4:51am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fkohya-ss-output-interpretation%2F141979%2F12 "2025-03-04T04:51:41Z")
Hi John6666,
There are a lot of “Training Parameters”. Is there a default value for all of them, or will I have to do a lot of “trial and errors” with each of them?
---
**Author:**  [@John6666](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FJohn6666)%5C
**Post date:** [March 4, 2025, 4:58am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fkohya-ss-output-interpretation%2F141979%2F13 "2025-03-04T04:58:02Z")
> Is there a default value for all of them,
Here.
> **[LoRA training parameters](https://github.com/bmaltais/kohya_ss/wiki/LoRA-training-parameters)**
>
> Contribute to bmaltais/kohya\_ss development by creating an account on GitHub.
> or will I have to do a lot of “trial and errors” with each of them
Or search parameters for similar use-case?😅
---
**Author:**  [@deicool](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fdeicool)%5C
**Post date:** [March 6, 2025, 5:52am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fkohya-ss-output-interpretation%2F141979%2F14 "2025-03-06T05:52:56Z")
Automated hyperparameter optimization (Optuna)?
---
**Author:**  [@John6666](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FJohn6666)%5C
**Post date:** [March 6, 2025, 5:58am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fkohya-ss-output-interpretation%2F141979%2F15 "2025-03-06T05:58:53Z")
Existing semi-automatic training scripts such as Kohya SS and OneTrainer use parameters that are within a certain range of acceptability from the start.
So it would probably be faster to search for know-how on how to create LoRA for similar use cases and borrow the detailed parameters.😅
I think that Optuna and other tools are more like frameworks for finding parameters when fine-tuning models fully manually.
---
**Author:**  [@deicool](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fdeicool)%5C
**Post date:** [March 6, 2025, 6:24am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fkohya-ss-output-interpretation%2F141979%2F16 "2025-03-06T06:24:14Z")
Would this be a good start?
[How to Train a Highly Convincing Real-Life LoRA Model - MyAIForce](https://myaiforce.com/real-life-lora-training/#:~:text=Training%20a%20LoRA%20model%20involves,settings%20within%20the%20Kohya%20trainer).
---
**Author:**  [@system](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsystem)%5C
**Post date:** [March 12, 2025, 9:36am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fkohya-ss-output-interpretation%2F141979%2F17 "2025-03-12T09:36:15Z")
This topic was automatically closed 12 hours after the last reply. New replies are no longer allowed.