# Data format in run\_lm\_fine\_tuning.py

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdata-format-in-run-lm-fine-tuning-py%2F1011%3E%5C
**Category:** Beginners\
**Created:** [September 7, 2020, 1:57am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdata-format-in-run-lm-fine-tuning-py%2F1011 "2020-09-07T01:57:53Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![kintaro](https://avatars.discourse-cdn.com/v4/letter/k/e56c9b/32.png) [@kintaro](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fkintaro)%5C
**Post date:** [September 7, 2020, 1:57am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdata-format-in-run-lm-fine-tuning-py%2F1011%2F1 "2020-09-07T01:57:53Z")

</div>

Hello everyone, I would like to ask for help with the following:

I want to fine-tuning of a language model for text generation, for this I will use `run_lm_fine_tuning.py`

I would like to know what is the optimal way to train the model. Can it be trained with a pure csv file, or should the file have some kind of pre-processing?

Or is it necessary to train it with an instance of the `Dataset` class ?

Thanks for the help

---

<div class="post-metadata">

**Author:** ![valhalla](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/valhalla/32/4179_2.png) [@valhalla](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fvalhalla)%5C
**Post date:** [September 7, 2020, 4:24am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdata-format-in-run-lm-fine-tuning-py%2F1011%2F2 "2020-09-07T04:24:53Z")

</div>

Hi @kintaro,

`run_lm_fine_tuning.py` is now renamed to `run_language_modeling.py` , you can fine it [here](https://github.com/huggingface/transformers/tree/master/examples/language-modeling).

The data file can be in one of the two formats

1. Line by line: which means each example is on it’s own line seperated by `\n`. Set the `--line_by_line` command line argument
2. Or just a plain text file from which examples will be sampled .

This [notebook](https://colab.research.google.com/github/huggingface/blog/blob/master/notebooks/01_how_to_train.ipynb) also walks through how a language model can be trained.

---

<div class="post-metadata">

**Author:** ![kintaro](https://avatars.discourse-cdn.com/v4/letter/k/e56c9b/32.png) [@kintaro](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fkintaro)%5C
**Post date:** [September 8, 2020, 6:31pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdata-format-in-run-lm-fine-tuning-py%2F1011%2F3 "2020-09-08T18:31:21Z")

</div>

@valhalla Thanks a lot for your answer. Let me ask:

1.With line\_by\_line is there any limit in the max number of tokens per line? maybe 512?

it seems like the easy way is with the plain text file.