# Further pretrain BART

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffurther-pretrain-bart%2F12791%3E%5C
**Category:** Models\
**Created:** [December 14, 2021, 9:24am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffurther-pretrain-bart%2F12791 "2021-12-14T09:24:12Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![atwaism](https://avatars.discourse-cdn.com/v4/letter/a/8baadc/32.png) [@atwaism](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fatwaism)%5C
**Post date:** [December 14, 2021, 9:24am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffurther-pretrain-bart%2F12791%2F1 "2021-12-14T09:24:12Z")

</div>

I’m looking for a script like [this](https://github.com/huggingface/transformers/blob/master/examples/pytorch/language-modeling/run_mlm.py) to further pretrain BART on my data , following the training tasks described in the paper.