# Bart-base rouge scores
**URL:**
**Author:**  [@sshleifer](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsshleifer)%5C
**Post date:** [August 11, 2020, 7:00pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fbart-base-rouge-scores%2F683%2F1 "2020-08-11T19:00:38Z")
Has anyone finetuned `bart-base` on xsum or cnn summarization task and willing to report the rouge score they got?
I just got 15.5 for xum which feels low, since bart-large can get to 22 ish.
@astariul @valhalla @VictorSanh ?
---
**Author:**  [@valhalla](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fvalhalla)%5C
**Post date:** [August 12, 2020, 5:44am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fbart-base-rouge-scores%2F683%2F2 "2020-08-12T05:44:41Z")
@sshleifer, could it be due to the `adjust_logits` issue ? Just a guess but as I posted there, after modifying the `adjust_logits_during_generation` BLUE-4 score for my model went from 13.09 to 19.14 for `bart-base`
---
**Author:**  [@valhalla](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fvalhalla)%5C
**Post date:** [August 14, 2020, 2:34pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fbart-base-rouge-scores%2F683%2F3 "2020-08-14T14:34:55Z")
@sshleifer could you also try using `bos` as `decoder_start_token_id` and modifying `adjust_logits_during_generation` to return logits as is instead of forcing `bos` ? If you also get bump in ROUGE score we can confirm the issue. Thanks !
---
**Author:**  [@chrisdoyleIE](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FchrisdoyleIE)%5C
**Post date:** [August 15, 2020, 6:00pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fbart-base-rouge-scores%2F683%2F4 "2020-08-15T18:00:41Z")
Possible suggestion that saves on the re-training could be to check the perplexity values and compare to paper
---
**Author:**  [@sshleifer](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsshleifer)%5C
**Post date:** [August 25, 2020, 3:46pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fbart-base-rouge-scores%2F683%2F5 "2020-08-25T15:46:01Z")
I got 16.6 ROUGE 2 on XSUM, in 3 epochs/ 6hrs
---
**Author:**  [@valhalla](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fvalhalla)%5C
**Post date:** [August 25, 2020, 4:24pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fbart-base-rouge-scores%2F683%2F6 "2020-08-25T16:24:38Z")
bart-base doesn’t seem to be good then, in my other seq2seq experiment t5-small performed similar/better to bart-base
---
**Author:**  [@sshleifer](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsshleifer)%5C
**Post date:** [September 5, 2020, 6:22pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fbart-base-rouge-scores%2F683%2F7 "2020-09-05T18:22:20Z")
Made a google doc to aggregate experiment results. Please add any interesting results!
> **[Seq2Seq Finetuning Leaderboard](https://docs.google.com/document/d/1ZMF67ysGHa_HodC_80muZ777_Qd9PHNXG0VMsDfFMUE/edit?usp=sharing)**
>
> Please state what you did and what you got Pegasus-large on xsum: Released model: 46.87/24.46/39.15 (Rouge1/Rouge2/RougeL) finetuned: {"rouge1": 46.8248, "rouge2": 23.9987, "rougeL": 38.6751, "n\_obs": 11333, "runtime": 4228.170863628387,...
---
**Author:**  [@guoziyuan](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fguoziyuan)%5C
**Post date:** [October 27, 2020, 12:08pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fbart-base-rouge-scores%2F683%2F8 "2020-10-27T12:08:43Z")
How can I change the adjust\_logits\_during\_generation ? thanks
---
**Author:**  [@sshleifer](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsshleifer)%5C
**Post date:** [October 27, 2020, 2:33pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fbart-base-rouge-scores%2F683%2F9 "2020-10-27T14:33:43Z")
By editing the code!
---
**Author:**  [@guoziyuan](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fguoziyuan)%5C
**Post date:** [October 27, 2020, 3:26pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fbart-base-rouge-scores%2F683%2F10 "2020-10-27T15:26:26Z")
Can you provide a example ? I saw the source code of ` adjust_logits_during_generation` and it directly returns the logits.
---
**Author:**  [@sshleifer](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsshleifer)%5C
**Post date:** [October 27, 2020, 4:32pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fbart-base-rouge-scores%2F683%2F11 "2020-10-27T16:32:19Z")
>
in the future `git grep adjust_logits_during_generation`
---
**Author:**  [@guoziyuan](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fguoziyuan)%5C
**Post date:** [October 27, 2020, 4:56pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fbart-base-rouge-scores%2F683%2F12 "2020-10-27T16:56:12Z")
thanks ❤ ❤ ❤