# Image captioning for Indonesia with pre-trained vision
**URL:**
**Author:**  [@munggok](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fmunggok)%5C
**Post date:** [June 24, 2021, 6:40pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fimage-captioning-for-indonesia-with-pre-trained-vision%2F7285%2F1 "2021-06-24T18:40:23Z")
## Image captioning for Indonesia with pre-trained vision and text model
For this project, a pre-trained image model like `ViT` can be used as an encoder, and a pre-trained text model like BERT and/or GPT2 can be used as a decoder.
### Model
Pre-trained [ViT](https://huggingface.co/models?filter=vit), [BERT](https://huggingface.co/bert-base-multilingual-cased) models can be found on the model hub. We could use multi-lingual BERT/ROERTa models for the Indonesian language.
### Datasets
The [WIT](https://github.com/google-research-datasets/wit) dataset can be used for this task. It has almost over 200K image-text pairs for Indonesia.
The [GEM](https://github.com/microsoft/GEM#get-dataset) dataset can also be used for the task
### Available training scripts
As this will be a Seq2Seq model, the [run\_summarization\_flax.py](https://github.com/huggingface/transformers/tree/master/examples/flax/summarization) script can be used for training this model with some modifications.
### (Optional) Desired project outcome
The desired outcome is to see if pre-trained vision and text models can be leveraged for image captioning and also train captioning models in for the Indonesia language. This can be showcased with a streamlit or gradio app.
### (Optional) Challenges
-This model will require some modifications to the existing text models.
-Data Processing
---
**Author:**  [@Galuh](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FGaluh)%5C
**Post date:** [June 24, 2021, 11:16pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fimage-captioning-for-indonesia-with-pre-trained-vision%2F7285%2F2 "2021-06-24T23:16:53Z")
Count me in!
---
**Author:**  [@cahya](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fcahya)%5C
**Post date:** [June 25, 2021, 7:05pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fimage-captioning-for-indonesia-with-pre-trained-vision%2F7285%2F3 "2021-06-25T19:05:56Z")
I am also in
---
**Author:**  [@ayameRushia](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FayameRushia)%5C
**Post date:** [June 29, 2021, 9:28am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fimage-captioning-for-indonesia-with-pre-trained-vision%2F7285%2F4 "2021-06-29T09:28:41Z")
I am also in
---
**Author:**  [@patrickvonplaten](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fpatrickvonplaten)%5C
**Post date:** [June 29, 2021, 3:16pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fimage-captioning-for-indonesia-with-pre-trained-vision%2F7285%2F5 "2021-06-29T15:16:39Z")
Great! Let’s define the project 🙂 cc @valhalla