# Pytorch Language Modeling Example for Seq2Seq Models

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fpytorch-language-modeling-example-for-seq2seq-models%2F163333%3E%5C
**Category:** 🤗Transformers\
**Created:** [July 16, 2025, 7:17pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fpytorch-language-modeling-example-for-seq2seq-models%2F163333 "2025-07-16T19:17:27Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![hrezaei](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/hrezaei/32/31681_2.png) [@hrezaei](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fhrezaei)%5C
**Post date:** [July 16, 2025, 7:17pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fpytorch-language-modeling-example-for-seq2seq-models%2F163333%2F1 "2025-07-16T19:17:27Z")

</div>

I customized the run\_clm.py example to be able to pretrain seq2seq models like T5, hence named [run\_slm.py](https://github.com/HRezaei/transformers/commit/195ab5bc640f98efb82abe58c03030dad7453c84). I think it might save time for other people interested in something similar. If you agree, please let me know to create a PR. Thanks! 🤗

@ArthurZ