# Fine tune Transformers for text generation
**URL:**
**Author:**  [@mwitiderrick](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fmwitiderrick)%5C
**Post date:** [April 4, 2022, 8:44am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffine-tune-transformers-for-text-generation%2F16444%2F1 "2022-04-04T08:44:16Z")
Hello
Is there an example like this one ([Fine-tune a pretrained model](https://huggingface.co/docs/transformers/training)) for fine tuning HF transformers for text generation?
---
**Author:**  [@merve](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fmerve)%5C
**Post date:** [April 4, 2022, 10:10am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffine-tune-transformers-for-text-generation%2F16444%2F2 "2022-04-04T10:10:26Z")
@mwitiderrick Hello 🤗
You can check out [this link](https://github.com/huggingface/transformers/tree/main/notebooks) for all example notebooks.
---
**Author:**  [@mwitiderrick](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fmwitiderrick)%5C
**Post date:** [April 4, 2022, 3:01pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffine-tune-transformers-for-text-generation%2F16444%2F3 "2022-04-04T15:01:51Z")
Hello @merve, thanks for the response, indeed I found the notebooks very useful. One follow-up question.
When I run predictions like this for a binary problem
```auto
import tensorflow as tf
predicted_class_id = int(tf.math.argmax(logits, axis=-1)[0])
bert.config.id2label[predicted_class_id]
```
I get the result as `LABEL_1`, How do I know if this is the prediction for class 0 or 1.
Thanks.
---
**Author:**  [@merve](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fmerve)%5C
**Post date:** [April 4, 2022, 3:46pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffine-tune-transformers-for-text-generation%2F16444%2F4 "2022-04-04T15:46:45Z")
Hello Derrick,
Can you send me the model repo so that I can see the config file?
---
**Author:**  [@mwitiderrick](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fmwitiderrick)%5C
**Post date:** [April 4, 2022, 5:01pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffine-tune-transformers-for-text-generation%2F16444%2F5 "2022-04-04T17:01:15Z")
Hello @merve
Not sure about the repo but the model is
```auto
from transformers import TFAutoModelForSequenceClassification
model = TFAutoModelForSequenceClassification.from_pretrained("bert-base-uncased", num_labels=2)
```
---
**Author:**  [@merve](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fmerve)%5C
**Post date:** [April 5, 2022, 8:13am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffine-tune-transformers-for-text-generation%2F16444%2F6 "2022-04-05T08:13:14Z")
Hello Derrick,
Sorry it’s my fault, label is already a label 😅 I meant, which dataset is the model fine-tuned on?
---
**Author:**  [@mwitiderrick](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fmwitiderrick)%5C
**Post date:** [April 5, 2022, 8:46am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffine-tune-transformers-for-text-generation%2F16444%2F7 "2022-04-05T08:46:22Z")
Hello @merve
It’s `imdb`
```auto
dataset = load_dataset("imdb")
```
---
**Author:**  [@mwitiderrick](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fmwitiderrick)%5C
**Post date:** [April 6, 2022, 5:10am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffine-tune-transformers-for-text-generation%2F16444%2F8 "2022-04-06T05:10:23Z")
Any update on this @merve ?
---
**Author:**  [@merve](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fmerve)%5C
**Post date:** [April 6, 2022, 10:16am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffine-tune-transformers-for-text-generation%2F16444%2F9 "2022-04-06T10:16:15Z")
@mwitiderrick In the [page of the dataset](https://huggingface.co/datasets/imdb) you can see the label for 1 is positive.
---
**Author:**  [@mwitiderrick](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fmwitiderrick)%5C
**Post date:** [April 6, 2022, 12:01pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffine-tune-transformers-for-text-generation%2F16444%2F10 "2022-04-06T12:01:40Z")
> [@merve](#):
>
> page of the dataset
Wanted to clarify that `LABEL_1` means label 1 and not 0
---
**Author:**  [@tbomez](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Ftbomez)%5C
**Post date:** [May 22, 2023, 5:11pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffine-tune-transformers-for-text-generation%2F16444%2F11 "2023-05-22T17:11:10Z")
hi @mwitiderrick, which HF notebook did you use for fine-tuning a model for text generation? thank you, Tom
---
**Author:**  [@Pranavagrl](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FPranavagrl)%5C
**Post date:** [July 27, 2023, 6:08am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffine-tune-transformers-for-text-generation%2F16444%2F12 "2023-07-27T06:08:09Z")
Hi @mwitiderrick I am a beginner I was trying to fine-tune the model to generate text I will be a great help if you will be able to provide your notebook to me.
I am ready to use any model of LlamaForCasualLM architecture any dataset will be fine for me I just want to understand the Concept and practical Implementation Behind that.