# Available dataset to train run\_translations.py examples

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Favailable-dataset-to-train-run-translations-py-examples%2F26596%3E%5C
**Category:** 🤗Datasets\
**Created:** [November 23, 2022, 4:25pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Favailable-dataset-to-train-run-translations-py-examples%2F26596 "2022-11-23T16:25:59Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![alvations](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/alvations/32/97_2.png) [@alvations](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Falvations)%5C
**Post date:** [November 23, 2022, 4:25pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Favailable-dataset-to-train-run-translations-py-examples%2F26596%2F1 "2022-11-23T16:25:59Z")

</div>

When trying to train a sagemaker, there’s this example from [https://huggingface.co/google/mt5-small:](https://huggingface.co/google/mt5-small:)

```auto
import sagemaker
from sagemaker.huggingface import HuggingFace

# gets role for executing training job
role = sagemaker.get_execution_role()
hyperparameters = {
	'model_name_or_path':'google/mt5-small',
	'output_dir':'/opt/ml/model'
	# add your remaining hyperparameters
	# more info here https://github.com/huggingface/transformers/tree/v4.17.0/examples/pytorch/seq2seq
}

# git configuration to download our fine-tuning script
git_config = {'repo': 'https://github.com/huggingface/transformers.git','branch': 'v4.17.0'}

# creates Hugging Face estimator
huggingface_estimator = HuggingFace(
	entry_point='run_translation.py',
	source_dir='./examples/pytorch/seq2seq',
	instance_type='ml.p3.2xlarge',
	instance_count=1,
	role=role,
	git_config=git_config,
	transformers_version='4.17.0',
	pytorch_version='1.10.2',
	py_version='py38',
	hyperparameters = hyperparameters
)

# starting the train job
huggingface_estimator.fit()

```

From the code, [transformers/examples/pytorch/translation/run\_translation.py at v4.17.0 · huggingface/transformers · GitHub](https://github.com/huggingface/transformers/blob/v4.17.0/examples/pytorch/translation/run_translation.py), there is a requirement to give (i) a `dataset_name` or (ii) `train_file` and `validation_file`.

### Are there examples of available Huggingface datasets that we can use to train the model?

### Is there an example of the jsonline file format for the `train_file` and `validation_file` arguments?

---

<div class="post-metadata">

**Author:** ![mariosasko](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/mariosasko/32/31548_2.png) [@mariosasko](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fmariosasko)%5C
**Post date:** [November 25, 2022, 12:58pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Favailable-dataset-to-train-run-translations-py-examples%2F26596%2F2 "2022-11-25T12:58:06Z")

</div>

Hi! You can find the required JSON structure [here](https://github.com/huggingface/transformers/blob/main/examples/pytorch/translation/README.md), and for the list of available translation datasets, visit [this page](https://huggingface.co/datasets?task_categories=task_categories:translation&sort=downloads).