# Donut Pre-Train on new Language
**URL:**
**Author:**  [@ChrisDelClea](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FChrisDelClea)%5C
**Post date:** [April 12, 2023, 3:32pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdonut-pre-train-on-new-language%2F36354%2F1 "2023-04-12T15:32:24Z")
Hi Folks,
thank you for offering the Donut model through Huggingface!
I followed the great blogpost from Philipp ( [Document AI: Fine-tuning Donut for document-parsing using Hugging Face Transformers (philschmid.de)](https://www.philschmid.de/fine-tuning-donut)) for the fine tuning.
However, I would like to train Donut on a new language before I fine-tune the model. As I have read, there is little difference between the initial pre-training of Donut and the fine-tuning. However, I wonder why so far only the training via the command line is specified on the official repo: `python train.py --config config/base.yaml --exp_version "base"`.
My question: can I change the code from the blogpost so that I can re-train / further fine-tune the model on a unseen language (i.e. the task of predicting the words on an image) with the `{"file_name": "image_138.jpg", "ground_truth": "{\"gt_parse\": {\"text_sequence\": \"... \"}}"}` format?
If this is possible, what do I need to change? Some code snippets or hints would be super helpful!
@amyeroberts Philipp told me you might help me with this question?
---
**Author:**  [@amyeroberts](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Famyeroberts)%5C
**Post date:** [April 13, 2023, 5:18pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdonut-pre-train-on-new-language%2F36354%2F2 "2023-04-13T17:18:23Z")
Hi @ChrisDelClea, thanks for your question! I’m not super familiar with the finetuning code, but I can try and help 🙂
It’s certainly possible to fine tune the Donut model on a new language. With regards to the question:
```auto
can I change the code from the blogpost so that I can re-train / further fine-tune the model on a unseen language (i.e. the task of predicting the words on an image) with the {"file_name": "image_138.jpg", "ground_truth": "{\"gt_parse\": {\"text_sequence\": \"... \"}}"} format?
```
are you asking what changes are required to take a dataset shown in the [official repo](https://github.com/clovaai/donut/blob/a0e94bf145d81cfc934eae8848f1269ae9ca46a2/README.md?plain=1#L122) to use it in the fine-tuning blog post?
In addition to getting the code to run with a new dataset, there might be other considerations too. For example, the tokenizer used might have had its vocabulary and tokens based on only English inputs.
---
**Author:**  [@ChrisDelClea](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FChrisDelClea)%5C
**Post date:** [April 14, 2023, 7:02am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdonut-pre-train-on-new-language%2F36354%2F3 "2023-04-14T07:02:17Z")
Hi @amyeroberts!
Yes exactly that dataset format for the `reading task`:
```auto
#### For (Pseudo) Text Reading Task
The `gt_parse` looks like `{"text_sequence" : "word1 word2 word3 ... "}`
- This task is also a pre-training task of Donut model.
```
How would i have to change the code from the blogpost to train on that format?
Regarding the tokenizer: Is there a way to change the used tokenizer inside DeonutProcessor in my fine-tuning code on new language?
I want to replace the following tokenizer to a german one:
[donut/model.py at e6623ad56c0e9f12a426dab2d8b2d65a39d64689 · clovaai/donut (github.com)](https://github.com/clovaai/donut/blob/e6623ad56c0e9f12a426dab2d8b2d65a39d64689/donut/model.py#L159-L161)
Does that make sense or is it not necessary at all? Sorry for the silly questions. Would help me a lot if you can tell me!
---
**Author:**  [@oual99](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Foual99)%5C
**Post date:** [November 3, 2023, 8:15pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdonut-pre-train-on-new-language%2F36354%2F4 "2023-11-03T20:15:11Z")
Hi @ChrisDelClea
I have the same objective, which is pre-train Donut on a new language (Arabic in my case), I don’t know if u succeded to do so.
If u have any instructions/code on this matter I’ll be very happy to know.
---
**Author:**  [@Jaw00](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FJaw00)%5C
**Post date:** [July 1, 2025, 9:16am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdonut-pre-train-on-new-language%2F36354%2F5 "2025-07-01T09:16:22Z")
hi @oual99
i’am currently working on the same objective: pre-train Donut on Arabic .
If you have succeeded to do so can you please share any tips /code that you have used?thank you.