# Random utf-8 errors from dataset

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Frandom-utf-8-errors-from-dataset%2F22134%3E%5C
**Category:** Intermediate\
**Created:** [August 26, 2022, 7:09am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Frandom-utf-8-errors-from-dataset%2F22134 "2022-08-26T07:09:21Z")\
**Posts on this page:** 1\
**Showing post:** 9

<div class="post-metadata">

**Author:** ![jbmaxwell](https://avatars.discourse-cdn.com/v4/letter/j/5daacb/32.png) [@jbmaxwell](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fjbmaxwell)%5C
**Post date:** [December 7, 2023, 9:34pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Frandom-utf-8-errors-from-dataset%2F22134%2F9 "2023-12-07T21:34:22Z")

</div>

I’ve moved on from that project, at this point, so unfortunately I can’t give a stack trace. But I will say that I _think_ it was more a data problem than a `datasets` problem. I still see the error with some of the data I’m using now, but I’ve started including `chardet` in my data pipeline, which seems to fix it (though it’s a bit pokey).

---

_[View the full topic](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Frandom-utf-8-errors-from-dataset%2F22134)._