# Question answeirng Fine tuning

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fquestion-answeirng-fine-tuning%2F89555%3E%5C
**Category:** Models\
**Created:** [May 31, 2024, 7:57am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fquestion-answeirng-fine-tuning%2F89555 "2024-05-31T07:57:20Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Berny93](https://avatars.discourse-cdn.com/v4/letter/b/a87d85/32.png) [@Berny93](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FBerny93)%5C
**Post date:** [May 31, 2024, 7:57am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fquestion-answeirng-fine-tuning%2F89555%2F1 "2024-05-31T07:57:20Z")

</div>

Hello. I need to do fine tuning to extract some fields from invoices (text extracted from invoices); some pretrained models such as distilled-bert require a dataset with the content, questions, the answers and the start\_id of the answer. I have a dataset containing only text extracted (the content) and the answers but not the start\_id. How can I overcome this limitation? Do you know other ways or models requiring only quesions and answers as dataset?

---

<div class="post-metadata">

**Author:** ![nielsr](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/nielsr/32/39617_2.png) [@nielsr](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fnielsr)%5C
**Post date:** [May 31, 2024, 1:03pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fquestion-answeirng-fine-tuning%2F89555%2F2 "2024-05-31T13:03:35Z")

</div>

There are multiple ways to solve this, as you’re working with invoices I’d assume a vision-language model to perform better than a text-only one.

See our blog post on document AI for an overview: [Accelerating Document AI](https://huggingface.co/blog/document-ai). Models like LayoutLM are better than text-only models like DistilBERT.

Nowadays there are also a lot of generative document AI models including PaliGemma, Idefics2, LLaVa,… besides Donut, Pix2Struct, UDOP.

You can find demo notebooks for all of those here: [GitHub - NielsRogge/Transformers-Tutorials: This repository contains demos I made with the Transformers library by HuggingFace.](https://github.com/NielsRogge/Transformers-Tutorials/tree/master).

Another option is to fine-tune a text-only LLM on OCR-ed text as I explained here: [Fine tune LLMs on PDF Documents - #9 by nielsr](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffine-tune-llms-on-pdf-documents%2F71374%2F9)

---

<div class="post-metadata">

**Author:** ![Berny93](https://avatars.discourse-cdn.com/v4/letter/b/a87d85/32.png) [@Berny93](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FBerny93)%5C
**Post date:** [June 3, 2024, 10:20am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fquestion-answeirng-fine-tuning%2F89555%2F3 "2024-06-03T10:20:59Z")

</div>

Thank you very much for the answer! I will surely have a look