How Community Notes Reduce Viral Misinformation | Keith Coleman, Jay Baxter | TED

Introduction to Community Notes

• Community Notes is a feature designed to build a better-informed world by providing access to accurate information, and as it scales to more parts of the internet, more people will have access to this information 10s.
• A Community Note is an example of a note that adds context to a post, such as a post about Iran claiming the USS Lincoln has been damaged and there are casualties, but the image is actually AI-generated, and the note provides specific details about what's wrong in the image 2m6s.
• Community Notes are written by regular users, known as Community Notes contributors, and before they are shown on the platform, they are rated helpful by people from different perspectives, ensuring that the notes are trustworthy and not biased 2m6s.
• The program allows all posts to be eligible for notes, including posts from heads of state, official accounts, ads, and any kind of post, and it has been used to identify AI-generated imagery, deepfake audio of world leaders, and to add context to posts that are correct but otherwise misleading 4m30s.

Origins and Motivation Behind Community Notes

• The origin of Community Notes dates back to 2016, when the founder was a Twitter user following the 2016 election and realized the need for a solution to the problem of misinformation on social media, and after working at Twitter and seeing the issues with traditional fact-checking methods, the idea for Community Notes was developed 8m30s.
• Traditional fact-checking methods had issues with speed, scale, and trust, with fact checks often taking two to four days to complete, and the ability to review only a limited number of posts per day, which led to the development of Community Notes as a more effective solution 10m30s.
• Community Notes was created as a response to people not wanting or trusting tech companies to decide what was accurate, and it involves asking people to trust random strangers on the internet, which may seem counterintuitive in a highly polarized environment 10s.

Traditional Fact-Checking Limitations

• The process behind Community Notes is totally open, transparent, and verifiable, allowing users to download the algorithm code and data to verify that there is no manipulation, and the notes are considered trustworthy by people on both sides of the political spectrum 2m6s.
• The algorithm used to decide which notes to show is based on agreement from people who have disagreed in the past, which is referred to as "surprising agreement" or "bridging," and it tends to produce accurate and neutral notes 2m6s.
• The visualization of Community Notes shows that notes are rated based on their helpfulness and point of view, and only notes that are found helpful by people who typically disagree are shown to everyone, while polarizing notes are not shown 4m30s.

Algorithm and Trustworthiness of Notes

• The community moderation element of Community Notes allows users to lose their privilege to write notes if they write too many low-quality notes, and the algorithm takes advantage of partisanship and polarization to produce accurate and unbiased notes 4m30s.
• The algorithm used by Community Notes is designed to produce accurate and neutral notes by leveraging the fact that people who are predisposed to disagree with a note will fact-check it thoroughly, and the notes that are found helpful tend to use primary sources and neutral language 6m10s.
• If a person, such as a head of state, is noted and tries to get the note removed by contacting the CEO, the response would be that there is no override button, and the person would need to take it up with the community that wrote the note 8m40s.

Community Moderation and Algorithm Design

• The decision to allow community notes to be displayed on posts without the ability to veto them was made to ensure that the notes reflect the people's opinion, rather than the tech company's opinion, and this principle has been stuck to with no way to change the status of a note 10s.
• When a post gets noted, it typically stops going viral, with the number of views flattening out, and this is due to "organic user behavior" where people realize the post is incorrect and like and repost it less, rather than the post being down-ranked by the algorithm 2m6s.
• Research from institutions such as Stanford, MIT, and UW has found that reposts drop by about 50 percent after a note is applied, which is a significant decrease in the scale of social media, and people are also less likely to agree with the core claims in the post after a note is applied 2m6s.

Impact of Community Notes on Viral Misinformation

• The application of a note to a post can have a mixed effect, as post authors are more likely to delete their posts after they get noted, which means that the best notes may be seen infrequently, but seeing a lot of notes can increase skepticism when reading things and serve as inoculation against misinformation 4m30s.
• The fact that people make a choice not to share a post after seeing a correction, regardless of their political spectrum, shows that people are reasonable and want to make good decisions when given information, and this pattern has been seen again and again in the data 6m0s.
• The team working on community notes has seen that there is quite a lot of agreement among people, and they are actually quite reasonable, which is a heartening and optimistic finding in the space of misleading information 8m0s.

Challenges and Risks of Gaming the System

• Community Notes is a system that aims to reduce viral misinformation, and one of the concerns is that it can be gamed or manipulated by malicious actors, such as those using AI-powered agents to create fake agreements and notes, with a potential example being a person forming 5,000 machine agents to manufacture a surprising agreement on a controversial issue 42s.
• To address this concern, Community Notes has several defenses in place, including requiring a verified phone number from a trusted carrier, looking for raters who have rated things similarly in the past, and treating similar behavior as coming from the same person to limit the influence of malicious actors 2m6s.
• Additionally, Community Notes has a self-correcting property, where incorrect notes attract attention and are quickly rated as not helpful, causing them to stop showing, and this property is important in breaking-news situations where information can change rapidly 4m10s.

Improving Speed and Accuracy of Notes

• Community Notes can be incorrect sometimes, but the self-correcting property helps to improve the accuracy of the notes over time, and this process can be seen as similar to Wikipedia or other collaborative knowledge-sharing platforms 5m20s.
• To tackle the speed problem, where niche topics may not have enough attention to bootstrap the initial surprising agreement, Community Notes can appear as often as every 20 minutes on a brand-new post, and can even appear instantly if there's already another note out there that matches on a URL, image, or video 8m30s.
• Furthermore, Community Notes sends push notifications to users who have engaged with a post before a note appears, so they can get the correction as soon as the note comes out, and the system is continually being improved to make it faster and more effective 10m40s.

Open Collaboration and AI Integration

• To increase the speed and effectiveness of Community Notes, an open API for AI contributors was opened up last year, allowing regular people to write their own AI-note writers and submit notes to the system, in the spirit of Community Notes being a totally open and collaborative system 12m10s.
• The Community Notes system is working effectively, with notes being generated quickly and accurately, although they can be incorrect at times due to their AI-generated nature, and a human layer is in place to rate these notes 10s.
• The goal is to develop a collaborative system where humans and AI work together to co-write notes, allowing for faster and more accurate generation of notes, with humans providing feedback and suggestions to improve the AI's performance 42s.

Human-AI Collaboration in Note Creation

• In this collaborative process, AI takes an initial attempt at writing a note, and humans can then provide corrections, suggestions, and feedback, which are used to train the AI and improve its performance, making it less likely to make mistakes and more neutral 1m6s.
• This process is an example of reinforcement learning from community feedback, where the AI model is trained to write notes that are likely to be found helpful by a diverse set of people, and this approach can help to reduce bias and improve the overall quality of the notes 2m6s.
• The system is being developed to address the growing problem of synthetic media and misinformation, with the goal of scaling up corrections and changing the incentives and dynamics of the system to promote more accurate and trustworthy information 4m10s.

Growth and Future Potential of Community Notes

• Despite the challenges posed by the rapid evolution of synthetic media, there are reasons to be optimistic about the potential of Community Notes to evolve and meet the demand for accurate information, with the system already showing promising results, including a doubling of the number of notes shown on the platform in the last four months 6m10s.
• The growth of a scaled service has shown potential for expansion, with a 2X increase in four months, indicating there is headroom for further growth, possibly 10X or 100X 10s.

Policy and Incentive Changes to Combat Misinformation

• To reduce the spread of misinformation, changes have been made to revenue-sharing policies, including suspending users from the program if they post misleading content, such as AI-generated footage of a war or conflict without clear labeling 42s.
• A pilot program has been launched to connect people with different perspectives and elevate voices that promote unity, using Community Notes to highlight ideas and opinions liked by people from various points of view 2m6s.

Promoting Unity and Common Ground

• The pilot program has shown that people are happy to see agreement on certain topics, such as Congress not being allowed to skip TSA lines until TSA is funded, and this agreement can be found across various topics, including immigration, economy, and international conflicts 2m6s.
• The concept of identifying and highlighting agreement on certain topics can incentivize people to speak in a way that promotes unity and can lead to a positive second-order effect, where common ground becomes common knowledge 4m30s.

Expanding the Reach and Open Ecosystem

• The open-source and open-data nature of the program allows other platforms, such as Bluesky and Truth Social, to plug in and use the stream, enabling AI to learn from it and connect communities back together 6m15s.
• The potential application of this engine beyond social media can lead to a future where areas of agreement are pursued, and people can come together to create a better world, with examples including Congress focusing on delivering agreements on topics like immigration and taxes 8m30s.

Introduction and Context of the Discussion

• The guest curators, Audrey Tang and Divya Siddarth, introduce the topic of the discussion, which involves bringing people who are doing incredible work onto the TED stage to share their ideas and create a dialogue about solving problems related to AI, democracy, and other big questions 10s.
• The interview format was chosen to discuss the work of Keith and Jay, who are training an AI to understand the differences between various communities, such as climate justice communities and biblical creation care communities, and how this social translation can impact democracy 1m4s.
• The discussion focuses on Community Notes, a defensive approach that aims to prevent the spread of bad information, but also explores the idea of flipping this approach to focus on positive solutions and information that people agree on 2m6s.
• The idea of flipping the approach involves figuring out the kinds of information and positive solutions that people agree on and making that the primary focus of online interactions, rather than the negative or misleading information 2m6s.
• The discussion concludes with the idea that data can be thought of as soil, and that the understanding between different communities can tend to a garden of AI agents that grow with the communities, loyal to them, and aimed at regenerating deep understanding 2m6s.