# Tokenizer deprecating in ORPO

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftokenizer-deprecating-in-orpo%2F113928%3E%5C
**Category:** Intermediate\
**Created:** [October 25, 2024, 7:56am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftokenizer-deprecating-in-orpo%2F113928 "2024-10-25T07:56:54Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![QuestforAIEd](https://avatars.discourse-cdn.com/v4/letter/q/67e7ee/32.png) [@QuestforAIEd](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FQuestforAIEd)%5C
**Post date:** [October 25, 2024, 7:56am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftokenizer-deprecating-in-orpo%2F113928%2F1 "2024-10-25T07:56:54Z")

</div>

Dear all,  
I was training an LLM using ORPO based on the [guides by Maxime Labonne](https://mlabonne.github.io/blog/posts/2024-04-19_Fine_tune_Llama_3_with_ORPO.html).

Unfortunately, I encountered the following error on the code below:  
Trainer.tokenizer is now deprecated. You should use Trainer.processing\_class instead.

Trying to resolve it by using the processing\_class in the way suggested by Gemini/ChatGPT did not resolve the problem. Does anyone have an idea how to resolve this?

Kind regards,  
Ben

```auto
orpo_args = ORPOConfig(
    learning_rate=8e-6,
    beta=0.1,
    lr_scheduler_type="linear",
    max_length=1024,
    max_prompt_length=512,
    per_device_train_batch_size=2,
    per_device_eval_batch_size=2,
    gradient_accumulation_steps=4,
    optim="paged_adamw_8bit",
    #Ideally train 3-5 epochs
    num_train_epochs=1,
    evaluation_strategy="steps",
    eval_steps=0.2,
    logging_steps=1,
    warmup_steps=10,
    report_to="wandb",
    output_dir="./results/",
)

trainer = ORPOTrainer(
    model=model,
    args=orpo_args,
    train_dataset=dataset["train"],
    eval_dataset=dataset["test"],
    peft_config=peft_config,
    tokenizer=tokenizer,
)
trainer.train()
trainer.save_model(new_model)

```

---

<div class="post-metadata">

**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FJohn6666)%5C
**Post date:** [October 25, 2024, 8:47am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftokenizer-deprecating-in-orpo%2F113928%2F2 "2024-10-25T08:47:34Z")

</div>

It appears that an issue has been issued on github and fixed. The date is yesterday, so I’m not sure if it has been reflected yet.

> <https://github.com/unslothai/unsloth/issues/1156>
>
> I can use orpo without issues with transformers 4.45.2 and orpo\_trainer.train(),… the old way of starting the training loop.
> 
> When I use the new unsloth\_train(orpo\_trainer) for gradient\_accumulation fix the following error shows up:
> 
> \`\`\`
> {
> "name": "KeyError",
> "message": "'labels'",
> "stack": "---------------------------------------------------------------------------
> KeyError Traceback (most recent call last)
> Cell In\[13\], line 3
> 1 from unsloth import unsloth\_train
> \----\> 3 trainer\_stats = unsloth\_train(orpo\_trainer)
> 5 # orpo\_trainer.train()
> 7 model.save\_pretrained(\\"lora\_model\_final\\") # Local saving
> 
> File c:\\\\work\\\\ModelTraining\\\\LocalUnsloth\\\\venv\_LocalUnsloth\\\\lib\\\\site-packages\\\\unsloth\\\\trainer.py:41, in unsloth\_train(trainer)
> 35 def unsloth\_train(trainer):
> 36 print(
> 37 \\"Unsloth: Using our custom gradient accumulation fixed trainer, which is not feature complete.\\
> \\"\\\\
> 38 \\"If you want to use our fix inside of HF, please update \`transformers\` to the latest version via:\\
> \\"\\\\
> 39 '\`pip uninstall transformers -y && pip install --upgrade --no-cache-dir \\"git+https://github.com/huggingface/transformers.git\\"\`'
> 40 )
> \---\> 41 return \_unsloth\_train(trainer)
> 
> File c:\\\\work\\\\ModelTraining\\\\LocalUnsloth\\\\venv\_LocalUnsloth\\\\lib\\\\site-packages\\\\unsloth\_zoo\\\\training\_utils.py:258, in unsloth\_train(trainer)
> 255 batches = \[next(train\_dataloader\_iterator) for j in range(n\_batches)\]
> 257 # Count non zeros before loss calc
> \--\> 258 n\_items = torch.stack(\[
> 259 torch.count\_nonzero(x\[\\"labels\\"\]\[..., 1:\] != -100) for x in batches
> 260 \]).sum()
> 262 # Gradient accumulation
> 263 for batch in batches:
> 
> File c:\\\\work\\\\ModelTraining\\\\LocalUnsloth\\\\venv\_LocalUnsloth\\\\lib\\\\site-packages\\\\unsloth\_zoo\\\\training\_utils.py:259, in \<listcomp\>(.0)
> 255 batches = \[next(train\_dataloader\_iterator) for j in range(n\_batches)\]
> 257 # Count non zeros before loss calc
> 258 n\_items = torch.stack(\[
> \--\> 259 torch.count\_nonzero(x\[\\"labels\\"\]\[..., 1:\] != -100) for x in batches
> 260 \]).sum()
> 262 # Gradient accumulation
> 263 for batch in batches:
> 
> KeyError: 'labels'"
> }
> \`\`\`
> 
> 
> 
> 
> When I upgrade transformers to the newest version as recommended 4.46.0.dev0
> 
> I get these errors on both ways of starting the training:
> \`\`\`
> {
> "name": "AttributeError",
> "message": "'generator' object has no attribute 'generate'",
> "stack": "---------------------------------------------------------------------------
> AttributeError Traceback (most recent call last)
> Cell In\[10\], line 3
> 1 from unsloth import unsloth\_train
> \----\> 3 trainer\_stats = unsloth\_train(orpo\_trainer)
> 5 # orpo\_trainer.train()
> 7 model.save\_pretrained(\\"lora\_model\_final\\") # Local saving
> 
> File c:\\\\work\\\\ModelTraining\\\\LocalUnsloth\\\\venv\_LocalUnsloth\\\\lib\\\\site-packages\\\\unsloth\\\\trainer.py:32, in unsloth\_train(trainer)
> 31 def unsloth\_train(trainer):
> \---\> 32 return trainer.train()
> 
> File c:\\\\work\\\\ModelTraining\\\\LocalUnsloth\\\\venv\_LocalUnsloth\\\\lib\\\\site-packages\\\\transformers\\\\trainer.py:2112, in Trainer.train(self, resume\_from\_checkpoint, trial, ignore\_keys\_for\_eval, \*\*kwargs)
> 2110 hf\_hub\_utils.enable\_progress\_bars()
> 2111 else:
> \-\> 2112 return inner\_training\_loop(
> 2113 args=args,
> 2114 resume\_from\_checkpoint=resume\_from\_checkpoint,
> 2115 trial=trial,
> 2116 ignore\_keys\_for\_eval=ignore\_keys\_for\_eval,
> 2117 )
> 
> File \<string\>:326, in \_fast\_inner\_training\_loop(self, batch\_size, args, resume\_from\_checkpoint, trial, ignore\_keys\_for\_eval)
> 
> File c:\\\\work\\\\ModelTraining\\\\LocalUnsloth\\\\venv\_LocalUnsloth\\\\lib\\\\site-packages\\\\trl\\\\trainer\\\\orpo\_trainer.py:852, in ORPOTrainer.get\_batch\_samples(self, model, batch)
> 849 generate\_context\_manager = amp.autocast(\\"cuda\\") if self.\_peft\_has\_been\_casted\_to\_bf16 else nullcontext()
> 851 with generate\_context\_manager:
> \--\> 852 policy\_output = model.generate(
> 853 input\_ids=batch\[\\"prompt\_input\_ids\\"\],
> 854 attention\_mask=batch\[\\"prompt\_attention\_mask\\"\],
> 855 max\_length=self.max\_length,
> 856 do\_sample=True,
> 857 pad\_token\_id=self.tokenizer.pad\_token\_id,
> 858 )
> 860 policy\_output = pad\_to\_length(policy\_output, self.max\_length, self.tokenizer.pad\_token\_id)
> 861 policy\_output\_decoded = self.tokenizer.batch\_decode(policy\_output, skip\_special\_tokens=True)
> 
> AttributeError: 'generator' object has no attribute 'generate'"
> }
> \`\`\`
> 
> My local env is:
> 🦥 Unsloth: Will patch your computer to enable 2x faster free finetuning.
> ==((====))== Unsloth 2024.10.3: Fast Qwen2 patching. Transformers = 4.46.0.dev0.
> \\\\ /| GPU: NVIDIA GeForce RTX 3090. Max memory: 23.999 GB. Platform = Windows.
> O^O/ \\\_/ \\ Pytorch: 2.4.0+cu121. CUDA = 8.6. CUDA Toolkit = 12.1.
> \\ / Bfloat16 = TRUE. FA \[Xformers = 0.0.27.post2. FA2 = False\]
> "-\_\_\_\_-" Free Apache license: http://github.com/unslothai/unsloth
> 
> But I also tried it with a linux cloud station running cuda 9 and 12.4 + torch 2.4.0
> The errors are the same.
> 
> Tried to use qwen 14B and llama 3.2 3B.
> Errors are the same.
> 
> 
> And on a second note when trying to prepare the dataset the Orpo-Trainer throws this warning:
> \`Trainer.tokenizer is now deprecated. You should use Trainer.processing\_class instead.\`
> 
> While it still works the massiv wall of prints is eating up cpu power and slowing down the data-prep.
> 
> You can surpress this with this code:
> \`\`\`
> import warnings
> from transformers import logging as transformers\_logging
> warnings.filterwarnings("ignore")
> transformers\_logging.set\_verbosity\_error()
> \`\`\`
> 
> But a clear updated code how to setup the trainer might be the cleaner solution.
> 
> Thanks for looking into this ^^

---

<div class="post-metadata">

**Author:** ![QuestforAIEd](https://avatars.discourse-cdn.com/v4/letter/q/67e7ee/32.png) [@QuestforAIEd](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FQuestforAIEd)%5C
**Post date:** [October 25, 2024, 9:06am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftokenizer-deprecating-in-orpo%2F113928%2F3 "2024-10-25T09:06:13Z")

</div>

Hi John,  
thanks for highlighting this - apparently it hasn’t been reflected yet. Will probably keep trying in the following days.

---

<div class="post-metadata">

**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FJohn6666)%5C
**Post date:** [October 25, 2024, 9:21am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftokenizer-deprecating-in-orpo%2F113928%2F4 "2024-10-25T09:21:36Z")

</div>

In a local environment, this change alone would be a workaround, but it would be best to wait until it is fixed for the latest version.

```auto
pip uninstall transformers
pip install transformers==4.45.2

```

---

<div class="post-metadata">

**Author:** ![QuestforAIEd](https://avatars.discourse-cdn.com/v4/letter/q/67e7ee/32.png) [@QuestforAIEd](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FQuestforAIEd)%5C
**Post date:** [October 25, 2024, 10:24am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftokenizer-deprecating-in-orpo%2F113928%2F5 "2024-10-25T10:24:24Z")

</div>

Dear John,  
thanks for helping a newbie – that literally fixed all my problems and stupid me didn’t think of just using the previous transformers version.

---

<div class="post-metadata">

**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FJohn6666)%5C
**Post date:** [October 25, 2024, 10:26am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftokenizer-deprecating-in-orpo%2F113928%2F6 "2024-10-25T10:26:15Z")

</div>

It’s not a solution, it’s a workaround, but, well, better to have it work than not to have it work!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/hellohellohello/original/2X/d/de4155eb4aa4108ecb32a1389d7cc37ae69f88b7.png) [@system](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsystem)%5C
**Post date:** [October 25, 2024, 10:26pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftokenizer-deprecating-in-orpo%2F113928%2F7 "2024-10-25T22:26:38Z")

</div>

Closed on October 25, 2024, 10:26pm UTC

This topic was automatically closed 12 hours after the last reply. New replies are no longer allowed.