Simulating Social Media Using Large Language Models to Evaluate Alternative News Feed Algorithms (2310.05984v1)

Abstract: Social media is often criticized for amplifying toxic discourse and discouraging constructive conversations. But designing social media platforms to promote better conversations is inherently challenging. This paper asks whether simulating social media through a combination of LLMs (LLM) and Agent-Based Modeling can help researchers study how different news feed algorithms shape the quality of online conversations. We create realistic personas using data from the American National Election Study to populate simulated social media platforms. Next, we prompt the agents to read and share news articles - and like or comment upon each other's messages - within three platforms that use different news feed algorithms. In the first platform, users see the most liked and commented posts from users whom they follow. In the second, they see posts from all users - even those outside their own network. The third platform employs a novel "bridging" algorithm that highlights posts that are liked by people with opposing political views. We find this bridging algorithm promotes more constructive, non-toxic, conversation across political divides than the other two models. Though further research is needed to evaluate these findings, we argue that LLMs hold considerable potential to improve simulation research on social media and many other complex social settings.

• The paper introduces a novel simulation combining LLMs with agent-based models to evaluate alternative news feed algorithms.
• Key findings reveal that bridging algorithms enhance inter-party engagement while minimizing toxicity compared to echo-chamber and popular-content feeds.
• The study demonstrates that transparent, algorithm-driven simulations offer practical insights into improving online discourse and mitigating polarization.

Simulating Social Media Using LLMs to Evaluate Alternative News Feed Algorithms

Introduction

The paper "Simulating Social Media Using LLMs to Evaluate Alternative News Feed Algorithms" (2310.05984) investigates the potential of combining LLMs with Agent-Based Modeling (ABM) to simulate social media environments. The study addresses the persistent issue of harmful discourse on social media, driven by algorithms that prioritize engagement, often fostering echo chambers and heightening polarization. The authors explore whether novel algorithmic designs, such as "bridging" algorithms, can foster more constructive discourse by highlighting consensus across divergent political views.

Methodology

The authors leverage a novel approach by integrating LLMs with ABMs to simulate user behavior in social media environments. The simulation involves creating personas based on data from the American National Election Study (ANES), which encompass detailed demographic and political characteristics reflective of the US electorate. Each persona interacts within synthetic platforms that mimic social media, governed by different news feed algorithms.

In the study, three synthetic platforms are created:

1. Platform 1: Shows posts based on engagement from followed users, mirroring existing echo chambers.
2. Platform 2: Exposes users to popular content from all users, aiming to break echo chambers.
3. Platform 3: Introduces a bridging algorithm that highlights posts with inter-partisan likes.

The simulation is designed to analyze the impact of these algorithms on discourse quality, measured through toxicity and inter-partisan interaction levels.

Figure 1: Illustration of the model developed in this paper, which combines LLMs and Agent-Based Models to simulate the impact of bridging algorithms on social media discourse. Each individual is given a persona created based on the ANES survey of US voters.

Results

The study finds that conventional algorithms tend to entrench users in echo chambers, limiting cross-party interaction and reducing toxic interactions at the cost of homogeneous discourse. Conversely, the platform utilizing the bridging algorithm (Platform 3) facilitates more inter-party conversations while maintaining a lower level of toxicity compared to the other two platforms.

Detailed observations include:

• Platform 1 exhibited minimal inter-partisan engagement and low toxicity due to its insular design.
• Platform 2 demonstrated increased cross-party interaction, accompanied by a surge in toxicity, as high-engagement posts often provoke divisive responses.
• Platform 3, employing the bridging algorithm, managed to increase bipartisan engagement and displayed the lowest toxicity, suggesting the potential of such algorithms in enhancing discourse quality without fostering division.

  Figure 2: Excerpt of the generated timeline from Platform 2.

  

  Figure 3: Excerpt of the generated timeline from Platform 3, using the bridging algorithm.

Implications and Future Directions

The findings suggest that bridging algorithms could be a viable path toward improving online discourse by promoting content that resonates across different political factions. While these results are promising, they highlight the need for further validation with real-world data and human evaluators to assess the fidelity of LLM-generated discourse.

Additionally, the study underscores the importance of transparency in algorithm design and the ethical considerations in using LLMs to simulate human behavior. Open-source LLMs, with known training data and biases, could enhance reproducibility and address ethical concerns.

Future research should explore larger-scale simulations over extended periods and evaluate the interaction of users learning to optimize for algorithm-specific visibility. This research avenue is crucial for developing algorithms that discourage toxicity while fostering constructive engagement on global platforms.

Conclusion

This research illustrates the potential of bridging algorithms in fostering healthier social media discourse, demonstrating the utility of integrating LLMs with ABMs for evaluating the impact of algorithmic design on public conversation. By addressing current limitations and expanding the scope of simulations, this approach could offer actionable insights into mitigating polarization and improving communication in digital public spheres.