# Error fine-tuning distilled Pegasus with run\_seq2seq.py
**URL:**
**Author:**  [@smaximo](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsmaximo)%5C
**Post date:** [February 16, 2021, 1:40pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ferror-fine-tuning-distilled-pegasus-with-run-seq2seq-py%2F3661%2F1 "2021-02-16T13:40:08Z")
Hello,
This is my first post in the forum.
I have successfully fine-tuned t5-small and distilled bart models using run\_seq2seq.py.
When I try to fine-tune sshleifer/distill-pegasus-xsum-16-8:
```auto
!python examples/seq2seq/run_seq2seq.py \
--model_name_or_path $modelname \
--do_train \
--do_eval \
--task summarization \
--train_file $trainpath \
--validation_file $valpath \
--output_dir $modelsave \
--overwrite_output_dir \
--per_device_train_batch_size=1 \
--per_device_eval_batch_size=1 \
--predict_with_generate \
--text_column ctext \
--save_steps=100000 \
--num_train_epochs=1 \
--summary_column text
```
I get the following error:
```auto
5.27it/s]/opt/conda/conda-bld/pytorch_1603729138878/work/aten/src/ATen/native/cuda/Indexing.cu:658: indexSelectLargeIndex: block: [213,0,0], thread: [96,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/opt/conda/conda-bld/pytorch_1603729138878/work/aten/src/ATen/native/cuda/Indexing.cu:658: indexSelectLargeIndex: block: [213,0,0], thread: [97,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
..................
File "/kaggle/working/transformers/src/transformers/models/pegasus/modeling_pegasus.py", line 340, in forward
if torch.isinf(hidden_states).any() or torch.isnan(hidden_states).any():
RuntimeError: CUDA error: device-side assert triggered
14%|████▉ | 3684/26890 [12:08<1:16:30, 5.05it/s]
```
I have tested with batches of size 1 and 2, and in both cases, the error is triggered at 14% of training steps.
The size of the training dataset is 26890 and the validation dataset 6720.
I tested the code in Google Colab and in Kaggle Kernels.
Has anyone successfully fine-tuned PEGASUS or a distilled version of PEGASUS using run\_seq2seq.py? Which arguments did you use?
Thank you for your valuable time and help
---
**Author:**  [@nielsr](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fnielsr)%5C
**Post date:** [February 17, 2021, 3:55pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ferror-fine-tuning-distilled-pegasus-with-run-seq2seq-py%2F3661%2F2 "2021-02-17T15:55:24Z")
Could you try to run the code on CPU? It usually helps to run the code on CPU when you have a CUDA error, to get a more informative error message.
---
**Author:**  [@smaximo](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsmaximo)%5C
**Post date:** [February 18, 2021, 10:41pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ferror-fine-tuning-distilled-pegasus-with-run-seq2seq-py%2F3661%2F3 "2021-02-18T22:41:22Z")
I followed your suggestion and ran the code on CPU.
I got the following error:
```auto
…./trainer.py", line 1325, in compute_loss
outputs = model(**inputs)
File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
“.../modeling_pegasus.py", line 1260, in forward
return_dict=return_dict,
…..
".../transformers/src/transformers/models/pegasus/modeling_pegasus.py", line 722, in forward
embed_pos = self.embed_positions(input_shape)
File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/torch/autograd/grad_mode.py", line 26, in decorate_context
return func(*args, **kwargs)
"..../transformers/models/pegasus/modeling_pegasus.py", line 138, in forward
return super().forward(positions)
File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/sparse.py", line 126, in forward
self.norm_type, self.scale_grad_by_freq, self.sparse)
File "/usr/local/lib/python3.6/dist-packages/torch/nn/functional.py", line 1852, in embedding
return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)
IndexError: index out of range in self
```
Searching in the forum there was a chance that it was related to max\_position\_embeddings.
The `max_position_embedding` for distill-pegasus-xsum-16-8 is 512.
My dataset contains one element greater than this value. I removed it and I was able to successfully complete the fine-tuning.
Thanks very much for your help!