MetaHate: A Dataset for Unifying Efforts on Hate Speech Detection
(2401.06526)Abstract
Hate speech represents a pervasive and detrimental form of online discourse, often manifested through an array of slurs, from hateful tweets to defamatory posts. As such speech proliferates, it connects people globally and poses significant social, psychological, and occasionally physical threats to targeted individuals and communities. Current computational linguistic approaches for tackling this phenomenon rely on labelled social media datasets for training. For unifying efforts, our study advances in the critical need for a comprehensive meta-collection, advocating for an extensive dataset to help counteract this problem effectively. We scrutinized over 60 datasets, selectively integrating those pertinent into MetaHate. This paper offers a detailed examination of existing collections, highlighting their strengths and limitations. Our findings contribute to a deeper understanding of the existing datasets, paving the way for training more robust and adaptable models. These enhanced models are essential for effectively combating the dynamic and complex nature of hate speech in the digital realm.
Overview
-
The paper introduces MetaHate, a meta-database that consolidates 36 datasets to aid in online hate speech detection.
-
MetaHate provides over 1.2 million human-authored comments from various social media platforms for more accurate detection models.
-
The work distinguishes between hate speech and offensive content, using the United Nations' definition for consistency.
-
The analysis included modelling experiments with methods like SVM, CNN, and BERT, the latter showing the best performance.
-
MetaHate is a free resource for research with the goal of fostering collaborative efforts to reduce online hate speech.
Introduction to MetaHate
The paper under discussion presents an impressive contribution to the ongoing efforts in combating hate speech online, a problem recognized for causing harm on a global scale. The endeavor, titled "MetaHate," provides a meta-collection of databases that could significantly impact the detection and moderation of hate speech on social media platforms. The collective represents an aggregate of 36 datasets, meticulously curated to create a unified resource aimed at improving the accuracy and adaptability of detection models.
Challenges and Solutions
Although data scientists and researchers have already produced several databases for analyzing online hate speech, their efforts have been fragmented, making it difficult to develop models with broader, more generalizable applications. Furthermore, the absence of a unified definition of hate speech has further compounded the difficulty in creating consistent detection mechanisms. The paper delineates a clear definition that aligns with that of the United Nations, differentiating between hate speech and merely offensive content.
The central obstacle addressed by the paper is the lack of standardized datasets, models, and metrics which hampers the development of effective hate speech detection systems. In response, MetaHate compiles a significant volume of data—more than 1.2 million non-duplicated comments sourced from various social media platforms, standing as a substantial, human-authored dataset. A vital aspect of this collection is its exclusivity to human-generated text, negating synthetic data and alternative sources, thereby enhancing relevance and coherence.
Data Analysis Insights
A comprehensive analysis of the MetaHate collection reveals that around 20% of included posts are classified as hate speech. The paper describes both lexical and psycholinguistic analyses, uncovering commonly used words and terms, as well as emotional connotations present in hate speech versus non-hate speech posts. Additionally, the dataset underwent a series of modelling experiments to provide benchmarks for future research, utilizing standard methods such as Support Vector Machine (SVM), Convolutional Neural Networks (CNN), and BERT. BERT, in particular, demonstrated superior performance outdoing both SVM and CNN in accurately classifying hate speech content.
Future Directions and Ethical Considerations
The paper suggests that the MetaHate dataset enables the advancement of research in the field of hate speech detection by providing access to a more extensive and varied set of data points. It encourages the adoption of sophisticated approaches that can process emotional context and multilingual inputs, offering potential for a more inclusive and omni-faceted perspective. The paper concludes with an acknowledgment of the ethical dimensions associated with such a dataset, particularly noting the potential misuse of offensive language contained within.
The dataset and supporting code are made freely available for research purposes, emphasizing the objective to foster collaborative efforts toward a safer online environment. This positions MetaHate not only as a research tool but as a societal asset in the struggle to diminish the negative impact of hate speech faced by individuals and communities across digital platforms.
Create an account to read this summary for free: