# LORA - how to determine what module\_to\_save
**URL:**
**Author:**  [@SuperBowser](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FSuperBowser)%5C
**Post date:** [August 18, 2025, 7:38pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Flora-how-to-determine-what-module-to-save%2F167206%2F1 "2025-08-18T19:38:10Z")
I am reading through LORA [tutorial](https://huggingface.co/docs/peft/main/en/task_guides/semantic_segmentation_lora) and one of the options in LoraConfig is modue\_to\_save. In the example its value is ‘decode-head’. I would like to use LORA with SequenceClassification model and I not sure what module I need to save.
Any thoughts?
---
**Author:**  [@John6666](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FJohn6666)%5C
**Post date:** [August 19, 2025, 5:35am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Flora-how-to-determine-what-module-to-save%2F167206%2F2 "2025-08-19T05:35:21Z")
If you [specify `task_type`](https://huggingface.co/docs/peft/en/package_reference/peft_types#peft.TaskType), [PEFT will automatically set `module_to_save` to an appropriate value](https://huggingface.co/docs/peft/v0.17.0/en/developer_guides/troubleshooting#randomly-initialized-layers). If you want to manually search for [the head module to save](https://github.com/huggingface/peft/issues/876), it would look something like this.
```py
from transformers import AutoModelForSequenceClassification, AutoConfig
import torch.nn as nn
HEAD_CANDIDATES = ("classifier", "score", "logits_proj", "classification_head")
def find_cls_head_name(model):
present = [n for n, _ in model.named_modules() if n.split(".")[-1] in HEAD_CANDIDATES]
if present: return present[0], present
num_labels = getattr(getattr(model, "config", object()), "num_labels", None)
hits = []
for parent_name, module in model.named_modules():
for child_name, child in module.named_children():
if isinstance(child, nn.Linear) and getattr(child, "out_features", None) == num_labels:
hits.append(child_name if parent_name == "" else f"{parent_name}.{child_name}")
return (hits[0] if hits else None), hits
def print_head_name(model_name):
cfg = AutoConfig.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name, config=cfg)
best, all_hits = find_cls_head_name(model)
print("Model name:", model_name)
print("All candidate heads:", all_hits)
print("Suggested modules_to_save:", [best] if best else None)
print_head_name("distilbert-base-uncased-finetuned-sst-2-english")
#Model name: distilbert-base-uncased-finetuned-sst-2-english
#All candidate heads: ['classifier']
#Suggested modules_to_save: ['classifier']
print_head_name("HuggingFaceTB/SmolLM-135M")
#Model name: HuggingFaceTB/SmolLM-135M
#All candidate heads: ['score']
#Suggested modules_to_save: ['score']
```
---
**Author:**  [@system](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsystem)%5C
**Post date:** [August 20, 2025, 7:27pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Flora-how-to-determine-what-module-to-save%2F167206%2F3 "2025-08-20T19:27:47Z")
Closed on August 20, 2025, 7:27pm UTC
This topic was automatically closed 12 hours after the last reply. New replies are no longer allowed.