# Domain-specific word similarity problem

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdomain-specific-word-similarity-problem%2F29071%3E%5C
**Category:** Awesome paper\
**Created:** [January 6, 2023, 12:56pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdomain-specific-word-similarity-problem%2F29071 "2023-01-06T12:56:33Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![vikasssss](https://avatars.discourse-cdn.com/v4/letter/v/b19c9b/32.png) [@vikasssss](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fvikasssss)%5C
**Post date:** [January 6, 2023, 12:56pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdomain-specific-word-similarity-problem%2F29071%2F1 "2023-01-06T12:56:33Z")

</div>

I am trying to create a chat-bot like application (inspired by chatGPT). The bot or application should be able to answer questions about our software on basis of help documents.  
I have tried to finetune QuestionAnswering models like distilbert\_base\_uncased on less than 100 annotated samples. But my model performance is not great. Can anyone suggest alternative approaches?

---

<div class="post-metadata">

**Author:** ![jmassot](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/jmassot/32/32864_2.png) [@jmassot](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fjmassot)%5C
**Post date:** [January 6, 2023, 7:13pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdomain-specific-word-similarity-problem%2F29071%2F2 "2023-01-06T19:13:42Z")

</div>

Hi Vikassss,

Are you talking about the performance of the Q&A engine applied on a test dataset or more generally after deployment?

In the second case, the low performance could be originated in different parts of the pipeline, not only the model. For example:  
1- what are you using as the retriever?  
2- what is your ranking strategy for the context?  
3- same question about the reader?

If your fine-tuned model is “forced” to find answers in non-optimal ranked contexts, it will fail.  
Could you please tell us more about your evaluation methodology?

Thanks

Best Regards

Jerome

---

<div class="post-metadata">

**Author:** ![sarmadq](https://avatars.discourse-cdn.com/v4/letter/s/ac8455/32.png) [@sarmadq](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsarmadq)%5C
**Post date:** [July 19, 2023, 12:59am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdomain-specific-word-similarity-problem%2F29071%2F3 "2023-07-19T00:59:55Z")

</div>

The most concrete suggestion I have would be to fine-tune the embeddings model on larger samples. For domain-specific use cases it’ll be really important to give as much of the domain-specific context as possible. Also, for my learning, what service are you using to fine-tune distilbert\_base\_uncased?