# Corpus for pre train bert base chinese

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcorpus-for-pre-train-bert-base-chinese%2F106652%3E%5C
**Category:** 🤗Transformers\
**Created:** [September 14, 2024, 2:16am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcorpus-for-pre-train-bert-base-chinese%2F106652 "2024-09-14T02:16:13Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![rocinant2023](https://avatars.discourse-cdn.com/v4/letter/r/97f17d/32.png) [@rocinant2023](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Frocinant2023)%5C
**Post date:** [September 14, 2024, 2:16am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcorpus-for-pre-train-bert-base-chinese%2F106652%2F1 "2024-09-14T02:16:13Z")

</div>

I want to actually distill bert base chinese model from google. But can find the dataset used to do pre train bert base chinese. Chinese corpus is kind of rare on internet. So any information is appreciated 😀

---

<div class="post-metadata">

**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FJohn6666)%5C
**Post date:** [September 14, 2024, 1:33pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcorpus-for-pre-train-bert-base-chinese%2F106652%2F2 "2024-09-14T13:33:19Z")

</div>

True. I followed the link and the Chinese dataset doesn’t come up…

> **[google-bert/bert-base-chinese · Hugging Face](https://huggingface.co/google-bert/bert-base-chinese)**
>
> We’re on a journey to advance and democratize artificial intelligence through open source and open science.

> **[google-bert/bert-base-uncased · Hugging Face](https://huggingface.co/google-bert/bert-base-uncased)**
>
> We’re on a journey to advance and democratize artificial intelligence through open source and open science.

[https://yknzhu.wixsite.com/mbweb](https://yknzhu.wixsite.com/mbweb)

> **[English Wikipedia](https://en.wikipedia.org/wiki/English_Wikipedia)**
>
> The English Wikipedia is the primary\[a\] English-language edition of Wikipedia, an online encyclopedia. It was created by Jimmy Wales and Larry Sanger on 15 January 2001, as Wikipedia's first edition.
> English Wikipedia is hosted alongside other language editions by the Wikimedia Foundation, an American nonprofit organization. Its content is written independently of other editions in various varieties of English, aiming to stay consistent within articles. Its internal newspaper is The Signpost...

I see a lot of Chinese people on HF (maybe), so I’m sure you can find someone who might know and ask them directly by opening a Discussion… or send a mention and they’ll notice.  
I’ve sorted the Chinese datasets by popularity, but I wonder if any of them are above this one.

> **[Hugging Face – The AI community building the future.](https://huggingface.co/datasets?modality=modality:text&language=language:zh&sort=likes)**
>
> We’re on a journey to advance and democratize artificial intelligence through open source and open science.

By the way, how to look for them, for example, if you follow the people who like the following data set (actually, you can), you have a pretty good chance that they are Chinese speakers. Well, not me.

> **[b3x0m/Chinese-H-Novels · Datasets at Hugging Face](https://huggingface.co/datasets/b3x0m/Chinese-H-Novels)**
>
> We’re on a journey to advance and democratize artificial intelligence through open source and open science.