# Track include\_num\_input\_tokens\_seen in Trainer

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftrack-include-num-input-tokens-seen-in-trainer%2F101138%3E%5C
**Category:** 🤗Transformers\
**Created:** [August 6, 2024, 2:14pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftrack-include-num-input-tokens-seen-in-trainer%2F101138 "2024-08-06T14:14:08Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![iarbel](https://avatars.discourse-cdn.com/v4/letter/i/3da27b/32.png) [@iarbel](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fiarbel)%5C
**Post date:** [August 6, 2024, 2:14pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftrack-include-num-input-tokens-seen-in-trainer%2F101138%2F1 "2024-08-06T14:14:08Z")

</div>

It appears that this metric also includes padding tokens. If one would use example packing, then it really tracks the “correct” number of tokens seen by the model.

However, I can think of two cases where this will not be accurate:

1. In cases where packing is not used, training examples are padded to the longest sequence in the batch
2. For SFT training on completions only

A more accurate calculation would be to sum the attention mask.  
Any thoughts?