# Split document into sentences for sentence embedding

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fsplit-document-into-sentences-for-sentence-embedding%2F3553%3E%5C
**Category:** Beginners\
**Created:** [February 9, 2021, 10:25am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fsplit-document-into-sentences-for-sentence-embedding%2F3553 "2021-02-09T10:25:47Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![LostGoatOnHill](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/lostgoatonhill/32/1001_2.png) [@LostGoatOnHill](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FLostGoatOnHill)%5C
**Post date:** [February 9, 2021, 10:25am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fsplit-document-into-sentences-for-sentence-embedding%2F3553%2F1 "2021-02-09T10:25:47Z")

</div>

Hi,

Wondering if there is a Huggingface alternative to the Gensim split\_sentences method to take a document and split into sentences ready for model.encode()?

[https://www.kite.com/python/docs/gensim.summarization.textcleaner.split\_sentences](https://www.kite.com/python/docs/gensim.summarization.textcleaner.split_sentences)

A first timer says many thanks 🙂

---

<div class="post-metadata">

**Author:** ![BramVanroy](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/bramvanroy/32/47360_2.png) [@BramVanroy](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FBramVanroy)%5C
**Post date:** [February 9, 2021, 12:10pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fsplit-document-into-sentences-for-sentence-embedding%2F3553%2F2 "2021-02-09T12:10:36Z")

</div>

So you want to split a text into sentences and then create a sentence embedding for each sentence? Just use a parser like stanza or spacy to tokenize/sentence segment your data. This is typically the first step in many NLP tasks.

---

<div class="post-metadata">

**Author:** ![LostGoatOnHill](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/lostgoatonhill/32/1001_2.png) [@LostGoatOnHill](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FLostGoatOnHill)%5C
**Post date:** [February 9, 2021, 12:26pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fsplit-document-into-sentences-for-sentence-embedding%2F3553%2F3 "2021-02-09T12:26:29Z")

</div>

Indeed, just wondered if Huggingface had their own variant. Thanks
