Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
139 tokens/sec
GPT-4o
47 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Analyzing Toxicity in Deep Conversations: A Reddit Case Study (2404.07879v1)

Published 11 Apr 2024 in cs.CL, cs.CY, and cs.SI

Abstract: Online social media has become increasingly popular in recent years due to its ease of access and ability to connect with others. One of social media's main draws is its anonymity, allowing users to share their thoughts and opinions without fear of judgment or retribution. This anonymity has also made social media prone to harmful content, which requires moderation to ensure responsible and productive use. Several methods using artificial intelligence have been employed to detect harmful content. However, conversation and contextual analysis of hate speech are still understudied. Most promising works only analyze a single text at a time rather than the conversation supporting it. In this work, we employ a tree-based approach to understand how users behave concerning toxicity in public conversation settings. To this end, we collect both the posts and the comment sections of the top 100 posts from 8 Reddit communities that allow profanity, totaling over 1 million responses. We find that toxic comments increase the likelihood of subsequent toxic comments being produced in online conversations. Our analysis also shows that immediate context plays a vital role in shaping a response rather than the original post. We also study the effect of consensual profanity and observe overlapping similarities with non-consensual profanity in terms of user behavior and patterns.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (39)
  1. Twitter users’ behavioral response to toxic replies. arXiv preprint arXiv:2210.13420.
  2. Reconstruction of threaded conversations in online discussion forums. In Proceedings of the International AAAI Conference on Web and Social Media, volume 5, pages 26–33.
  3. The role of social networks in information diffusion. In Proceedings of the 21st international conference on World Wide Web, pages 519–528.
  4. RedditBias: A real-world resource for bias evaluation and debiasing of conversational language models. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 1941–1955, Online. Association for Computational Linguistics.
  5. Nuanced metrics for measuring unintended bias with real data for text classification. In Companion proceedings of the 2019 world wide web conference, pages 491–500.
  6. Racial bias in hate speech and abusive language detection datasets. In Proceedings of the Third Workshop on Abusive Language Online, pages 25–35, Florence, Italy. Association for Computational Linguistics.
  7. Automated hate speech detection and the problem of offensive language. In Proceedings of the international AAAI conference on web and social media, volume 11, pages 512–515.
  8. Measuring and mitigating unintended bias in text classification. In Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society, pages 67–73.
  9. Latent hatred: A benchmark for understanding implicit hate speech. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 345–363, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
  10. Large scale crowdsourcing and characterization of twitter abusive behavior. In Twelfth International AAAI Conference on Web and Social Media.
  11. Sarcasm analysis using conversation context. Computational Linguistics, 44(4):755–792.
  12. Cosyn: Detecting implicit hate speech in online conversations using a context synergized hyperbolic network. Preprint, arXiv:2303.03387.
  13. Public wisdom matters! discourse-aware hyperbolic fourier co-attention for social text classification. Advances in Neural Information Processing Systems, 35:9417–9431.
  14. Fueling toxicity? studying deceitful opinion leaders and behavioral changes of their followers. Politics and Governance, 10(4):336–348.
  15. ToxiGen: A large-scale machine-generated dataset for adversarial and implicit hate speech detection. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3309–3326, Dublin, Ireland. Association for Computational Linguistics.
  16. Harold Stanley Heaps. 1978. Information retrieval, computational and theoretical aspects. Academic Press.
  17. John W Jordan. 2020. Profanity from the heart as exceptional civic rhetoric. Quarterly Journal of Speech, 106(2):111–132.
  18. Strangers on your phone: Why people use anonymous communication applications. In Proceedings of the 19th ACM conference on computer-supported cooperative work & social computing, pages 359–370.
  19. Predicting continuity of online conversations on reddit. Telematics and Informatics, 79:101965.
  20. Dynamics of conversations. In Proceedings of the 16th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 553–562.
  21. Alex Leavitt and Joshua A Clark. 2014. Upvoting hurricane sandy: event-based news production processes on a social news site. In Proceedings of the SIGCHI conference on human factors in computing systems, pages 1495–1504.
  22. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692.
  23. A measurement study of hate speech in social media. In Proceedings of the 28th ACM conference on hypertext and social media, pages 85–94.
  24. Abusive language detection in online user content. In Proceedings of the 25th international conference on world wide web, pages 145–153.
  25. Comparing toxicity across social media platforms for covid-19 discourse. ArXiv, abs/2302.14270.
  26. Quick, community-specific learning: How distinctive toxicity norms are maintained in political subreddits. In Proceedings of the International AAAI Conference on Web and Social Media, volume 14, pages 557–568.
  27. HateCheck: Functional tests for hate speech detection models. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 41–58, Online. Association for Computational Linguistics.
  28. The risk of racial bias in hate speech detection. In Proceedings of the 57th annual meeting of the association for computational linguistics, pages 1668–1678.
  29. How social media users perceive different forms of online hate speech: A qualitative multi-method study. New Media & Society, page 14614448221091185.
  30. Namespotting: Username toxicity and actual toxic behavior on reddit. Computers in Human Behavior, 136:107371.
  31. Empirical analysis of multi-task learning for reducing identity bias in toxic comment detection. In Proceedings of the International AAAI Conference on Web and Social Media, volume 14, pages 683–693.
  32. Learning from the worst: Dynamically generated datasets to improve online hate detection. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 1667–1682, Online. Association for Computational Linguistics.
  33. Successful new-entry prediction for multi-party online conversations via latent topics and discourse modeling. In Proceedings of the ACM Web Conference 2022, pages 1663–1672.
  34. Zeerak Waseem. 2016. Are you a racist or am i seeing things? annotator influence on hate speech detection on twitter. In Proceedings of the first workshop on NLP and computational social science, pages 138–142.
  35. Zeerak Waseem and Dirk Hovy. 2016. Hateful symbols or hateful people? predictive features for hate speech detection on Twitter. In Proceedings of the NAACL Student Research Workshop, pages 88–93, San Diego, California. Association for Computational Linguistics.
  36. Ex machina: Personal attacks seen at scale. In Proceedings of the 26th international conference on world wide web, pages 1391–1399.
  37. Demoting racial bias in hate speech detection. In Proceedings of the Eighth International Workshop on Natural Language Processing for Social Media, pages 7–14, Online. Association for Computational Linguistics.
  38. Detection of harassment on web 2.0. Proceedings of the Content Analysis in the WEB, 2:1–7.
  39. Towards developing a measure to assess contagiousness of toxic tweets.
Citations (1)
View on Semantic Scholar

Summary

  • The paper reveals that initial toxic responses often trigger a cascade with a correlation of 0.631, emphasizing the role of early comments in toxicity propagation.
  • It employs a tree-based approach to analyze over one million Reddit replies, mapping user behaviors and the decline of toxicity after early conversation levels.
  • The study finds similar dynamics in both consensual and non-consensual toxic environments, suggesting consistent patterns across varied online contexts.

Insights into Toxicity Dynamics in Online Conversations: A Reddit Analysis

The paper "Analyzing Toxicity in Deep Conversations: A Reddit Case Study" provides an in-depth examination of the proliferation of toxic language in online public discussions, utilizing Reddit as the foundation for its paper. The analysis is administered through a tree-based approach, allowing for the assessment of user behaviors and the dynamics of toxicity within public conversations. By focusing on eight Reddit communities that welcome profanity, the research encompasses over one million responses to understand the propagation and impact of toxicity in these spaces.

Contextual and Temporal Dimensions of Toxicity

A central observation from the paper is the correlation between initial toxic responses and the subsequent perpetuation of toxicity through the thread. Toxic comments often spawn further toxic responses, leading to a potential cascade of negativity. This correlation is quantified by a moderate and statistically significant correlation coefficient of around 0.631, signifying that while not all comments engender toxicity, there is a discernible association favoring its proliferation.

To explore the complexities of conversational context, the paper examines the effect of preceding comments on the toxicity of subsequent responses. Interestingly, it is the immediate predecessor response that significantly impacts the toxicity level of a given comment, highlighting the importance of local context over broader discourse patterns. Toxicity in conversations tends to diminish after the initial few levels, further indicating that such exchanges are usually short-lived in terms of toxic interactions.

User Behavior in Toxic Environments

The investigation into user participation amidst toxic discussions reveals an intriguing dichotomy. There is a bimodal distribution in user responses where both highly toxic and completely non-toxic comments tend to drive more engagement than those with moderate toxicity levels. This suggests that while users may be deterred by moderately toxic exchanges, they are more likely to engage in highly polarized discussions, perhaps due to the inherent contentiousness that drives communication in polarized environments.

Comparative Analysis of Consensual and Non-consensual Toxicity

By analyzing a subreddit dedicated to consensual roasting, the research explores whether toxic behaviors differ in settings where such language is anticipated and embraced. The findings suggest that the dynamics of toxicity in consensual contexts are remarkably similar to those found in non-consensual settings. This parallel indicates a robust pattern of toxic interaction that transcends the expectation of profanity, suggesting that the mechanics of toxic language propagation remain consistent irrespective of user consent to participate in such exchanges.

Implications and Future Directions

The implications of these findings are multifaceted. They highlight the necessity for social media platforms to develop more nuanced models for moderating toxic content, taking into account not just the text itself but the context and conversation structure in which it occurs. Moreover, the results may guide future AI developments in social media moderation, wherein detecting and mitigating toxicity would benefit from understanding its conversational dynamics.

Additionally, further exploration into user engagement patterns in the presence of toxicity might unveil strategies to promote healthier interactions. This would ideally involve developing interventions that not only identify toxicity but also encourage users to disengage from destructive exchanges in favor of constructing positive dialogue.

The foundational characteristics of toxicity in public conversations depicted in this case paper extend the understanding of content dynamics on platforms like Reddit. As these platforms continue to serve as significant arenas for public discourse, insights derived from such research are crucial for fostering productive and respectful online interactions.

File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets