# Feature Extraction pipeline for a single token

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffeature-extraction-pipeline-for-a-single-token%2F24514%3E%5C
**Category:** Beginners\
**Created:** [October 16, 2022, 11:27am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffeature-extraction-pipeline-for-a-single-token%2F24514 "2022-10-16T11:27:14Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![matheusvolpon](https://avatars.discourse-cdn.com/v4/letter/m/e495f1/32.png) [@matheusvolpon](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fmatheusvolpon)%5C
**Post date:** [October 16, 2022, 11:27am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffeature-extraction-pipeline-for-a-single-token%2F24514%2F1 "2022-10-16T11:27:14Z")

</div>

Hi all.  
I would like to extract the word embeddings for some input tokens (single words, not sentences) from DistilBertForMaskedLM models that I trained with the Hugging Face library. Does the Feature Extraction pipeline compute only contextualized embeddings? Or, when the pipeline receives a single token, it computes a decontextualized embedding? (using the mean of the word’s embeddings from all it contexts).