The Korean UnSmile Dataset, publicly released by Smilegate AI, is a human-annotated multi-label hate speech dataset specifically designed for the Korean language. This dataset defines hate speech as "hostile remarks, ridicule, caricatures, and expressions that reproduce prejudice against specific social (minority) groups," explicitly excluding self-deprecating expressions.

The dataset comprises 18,742 unique sentences, categorized into three primary types: 10,139 hate speech instances, 3,929 abusive/profane language instances, and 4,674 clean sentences. For hate speech, a multi-label annotation scheme is employed across eight distinct categories: "여성/가족" (Female/Family), "남성" (Male), "성소수자" (LGBTQ+), "인종/국적" (Race/Nationality), "연령" (Age), "지역" (Region), "종교" (Religion), and "기타혐오" (Other Hate). The "악플/욕설" (Abusive/Profane) category captures general insults, unpleasant content, or explicit language not directed at specific social groups. A "clean" label signifies sentences free from hate speech, abuse, or offensive content. An additional binary tag, "개인지칭" (Personal Designation), is applied if the utterance targets a specific, identifiable individual.

The annotation methodology for the UnSmile dataset involves a two-stage process. Initially, each comment sentence was classified for its hate categories by three distinct annotators. Subsequently, a final verification and quality assurance step was performed by a panel of five hate speech experts. The developers acknowledge that the perception of hate speech can vary, and thus, the tagging results do not guarantee 100% accuracy.

The dataset is distributed in TSV format, specifically unsmile_train_v1.0.tsv and unsmile_valid_v1.0.tsv, and is readily accessible via the Hugging Face datasets library. The schema for each entry includes the Korean sentence (문장) and binary indicators for each of the ten primary classification labels (e.g., 여성/가족, 남성, ..., 악플/욕설, clean), along with the 개인지칭 label.

A baseline model, smilegate-ai/kor_unsmile, built upon BertForSequenceClassification from the transformers library, is provided. This model employs a TextClassificationPipeline with a sigmoid activation function to handle the multi-label nature of the task, outputting probability scores for each category. Performance evaluation of the baseline model on the validation set utilized standard classification metrics: precision, recall, and F1-score, reported for each category, as well as micro, macro, weighted, and samples averages. The baseline achieved a micro-averaged F1-score of 0.77, a macro-averaged F1-score of 0.76, a weighted-averaged F1-score of 0.77, and a samples-averaged F1-score of 0.75. Notably, the '기타혐오' (Other Hate) category exhibited a lower recall of 0.18 and F1-score of 0.30, indicating a particular challenge in identifying this broad category.

The source code and baseline model are released under the Apache 2.0 License, while the dataset itself is licensed under CC-BY-NC-ND 4.0, restricting commercial use without explicit permission.