# SFTTrainer class - and Training arguements
**URL:**
**Author:**  [@AnishIISC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FAnishIISC)%5C
**Post date:** [May 9, 2024, 2:07pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fsfttrainer-class-and-training-arguements%2F85976%2F1 "2024-05-09T14:07:24Z")
I an trying to fine Tune Gemma 7B IT for a dataset. The code is being run in kaggle. both an existing run and current fails with this error
‘TrainingArguments’ object has no attribute ‘model\_init\_kwargs’
My install block is
!pip install -q -U torch --index-url [https://download.pytorch.org/whl/cu117](https://download.pytorch.org/whl/cu117)
!pip install -q -U transformers==“4.38.2”
!pip install -q accelerate
!pip install -q -i [Simple index](https://pypi.org/simple/) bitsandbytes
!pip install -q -U datasets
!pip install -q -U git+https://github.com/huggingface/trl
!pip install -q -U git+https://github.com/huggingface/peft
parameter block is below (The exact same code was running two months ago !)
peft\_config = LoraConfig(
lora\_alpha=16,
lora\_dropout=0,
r=64,
bias=“none”,
task\_type=“CAUSAL\_LM”,
target\_modules=[“q\_proj”, “k\_proj”, “v\_proj”, “o\_proj”,
“gate\_proj”, “up\_proj”, “down\_proj”,],
)
training\_arguments = TrainingArguments(
output\_dir=“logs”,
num\_train\_epochs=5,
gradient\_checkpointing=True,
per\_device\_train\_batch\_size=1,
gradient\_accumulation\_steps=8,
optim=“paged\_adamw\_32bit”,
save\_steps=0,
logging\_steps=25,
learning\_rate=2e-4,
weight\_decay=0.001,
fp16=True,
bf16=False,
max\_grad\_norm=0.3,
max\_steps=-1,
warmup\_ratio=0.03,
group\_by\_length=False,
evaluation\_strategy=‘steps’,
eval\_steps = 112,
eval\_accumulation\_steps=1,
lr\_scheduler\_type=“cosine”,
report\_to=“tensorboard”,
)
trainer = SFTTrainer(
model=model,
train\_dataset=train\_data,
eval\_dataset=eval\_data,
peft\_config=peft\_config,
dataset\_text\_field=“text”,
tokenizer=tokenizer,
max\_seq\_length=max\_seq\_length,
args=training\_arguments,
packing=False,
)
---
**Author:**  [@AnishIISC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FAnishIISC)%5C
**Post date:** [May 9, 2024, 6:07pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fsfttrainer-class-and-training-arguements%2F85976%2F2 "2024-05-09T18:07:33Z")
used the below to resolve
# Install Pytorch & other libraries
!pip install “torch==2.1.2” tensorboard
# Install Hugging Face libraries
!pip install --upgrade
“transformers==4.38.2”
“datasets==2.16.1”
“accelerate==0.26.1”
“evaluate==0.4.1”
“bitsandbytes==0.42.0”
“trl==0.7.11”
“peft==0.8.2”
---
**Author:**  [@system](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsystem)%5C
**Post date:** [May 10, 2024, 6:07am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fsfttrainer-class-and-training-arguements%2F85976%2F3 "2024-05-10T06:07:43Z")
Closed on May 10, 2024, 6:07am UTC
This topic was automatically closed 12 hours after the last reply. New replies are no longer allowed.