# Seq2Seq Distillation: train\_distilbart\_xsum error

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fseq2seq-distillation-train-distilbart-xsum-error%2F1927%3E%5C
**Category:** 🤗Transformers\
**Created:** [November 7, 2020, 1:56pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fseq2seq-distillation-train-distilbart-xsum-error%2F1927 "2020-11-07T13:56:38Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![anon18297155](https://avatars.discourse-cdn.com/v4/letter/a/b3f665/32.png) [@anon18297155](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fanon18297155)%5C
**Post date:** [November 7, 2020, 1:56pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fseq2seq-distillation-train-distilbart-xsum-error%2F1927%2F1 "2020-11-07T13:56:38Z")

</div>

@sshleifer  
Hi,  
I would like to run the method command lines of Direct Knowledge Distillation (KD) which you mentioned at [https://github.com/huggingface/transformers/tree/master/examples/seq2seq](https://github.com/huggingface/transformers/tree/master/examples/seq2seq)  
" ./train\_distilbart\_xsum.sh --logger\_name wandb --gpus 1"  
and meet an error,  
“OSError: Model name ‘distilbart\_xsum\_12\_6/student’ was not found in tokenizers model name list (facebook/bart-base, facebook/bart-large, facebook/bart-large-mnli, facebook/bart-large-cnn, facebook/bart-large-xsum, yjernite/bart\_eli5). We assumed ‘distilbart\_xsum\_12\_6/student’ was a path, a model identifier, or url to a directory containing vocabulary files named [‘vocab.json’, ‘merges.txt’] but couldn’t find such vocabulary files at this path or url.”  
I try to fix it by add the code to distillation.py 46 line that is  
“hparams.tokenizer\_name = hparams.teacher # Use teacher’s tokenizer”  
It seems to fix it but I am not very sure.

---

<div class="post-metadata">

**Author:** ![sshleifer](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/sshleifer/32/3304_2.png) [@sshleifer](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsshleifer)%5C
**Post date:** [November 8, 2020, 9:53pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fseq2seq-distillation-train-distilbart-xsum-error%2F1927%2F2 "2020-11-08T21:53:19Z")

</div>

yes, your fix is perfect! Wil fix, thanks for reporting this!

---

<div class="post-metadata">

**Author:** ![anon18297155](https://avatars.discourse-cdn.com/v4/letter/a/b3f665/32.png) [@anon18297155](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fanon18297155)%5C
**Post date:** [November 9, 2020, 3:27am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fseq2seq-distillation-train-distilbart-xsum-error%2F1927%2F3 "2020-11-09T03:27:37Z")

</div>

Thanks very much!  
but this code with another issue:  
I cannot run the code with --fp16 --fp16\_opt\_level=O1, it meets OOM, but I can run it without it. It seems so interesting and confusing.  
BTW, a similar issue is reported at [https://github.com/huggingface/transformers/issues/8403](https://github.com/huggingface/transformers/issues/8403)  
Really look forward to your reply  
thanks again

---

<div class="post-metadata">

**Author:** ![anon18297155](https://avatars.discourse-cdn.com/v4/letter/a/b3f665/32.png) [@anon18297155](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fanon18297155)%5C
**Post date:** [November 10, 2020, 6:50am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fseq2seq-distillation-train-distilbart-xsum-error%2F1927%2F4 "2020-11-10T06:50:16Z")

</div>

@sshleifer

---

<div class="post-metadata">

**Author:** ![sshleifer](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/sshleifer/32/3304_2.png) [@sshleifer](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsshleifer)%5C
**Post date:** [November 10, 2020, 2:08pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fseq2seq-distillation-train-distilbart-xsum-error%2F1927%2F5 "2020-11-10T14:08:52Z")

</div>

Yes, as the issue suggests, the only work around at the moment is use torch 1.5 + Apex.  
You should follow that issue for subsequent updates.

---

<div class="post-metadata">

**Author:** ![anon18297155](https://avatars.discourse-cdn.com/v4/letter/a/b3f665/32.png) [@anon18297155](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fanon18297155)%5C
**Post date:** [November 10, 2020, 2:19pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fseq2seq-distillation-train-distilbart-xsum-error%2F1927%2F6 "2020-11-10T14:19:11Z")

</div>

Thanks a lot!