Psychology of Content Moderation
Notes on: Different Sides of Fairness: Evaluations of Fairness of Nextdoor’s Content Moderation System (2025)
Before you continue
Warning
- You should have already read the Katsaros et al. (2025) article!!
- If you have not, go do that first!!
- These slides are meant to operate as companion to help you pull out some of the most important content, not a substitute for actually reading the article!!
Background
- Purpose: Investigate how fairness perceptions of Nextdoor’s content moderation system affect user behavior
- Methodology: Self-report survey of both content reporters and creators with removed content
- Data Collection: Combined survey responses with 9 months of logged platform data at:
- 6 months before survey
- 3 months after survey
Katsaros et al. (2025, p. 1)
The Scale of Content Moderation
To illustrate the investment platforms make in content moderation:
“More than 40,000 people globally work on trust and safety issues for TikTok. … This year we expect to invest more than two billion dollars in trust and safety efforts…”
— Shou Zi Chew, CEO of TikTok (2024)
Katsaros et al. (2025, p. 1)
Two Approaches to Platform Governance
Top-Down Governance
- Platforms like Instagram or TikTok
- Complex moderation systems with platform-set rules
- Centralized reporting mechanisms
- Formal review processes and enforcement actions
Community-Driven Governance
- Platforms like Reddit, Discord, and Wikipedia
- Community members (“mods”) screen and manage content
- Locally created rules and norms
- Peer-based conflict diffusion and standard enforcement
Katsaros et al. (2025, p. 2)
The Problem with Current Approaches
Research highlights how moderation systems often create an adversarial dynamic:
“moderation tends to structure platforms and their users as opposing parties”
This creates opportunities for platforms to better engage with moderated users as stakeholders in the design process.
Katsaros et al. (2025, p. 2)
Definitions: Justice Theories
- Procedural Justice
- People care about the quality of a decision-making process, beyond any specific outcome
- Four primary factors: voice, neutrality, dignity/respect, and trust
- Distributive Justice
- Focuses on the fairness of outcomes delivered by decision makers
- Was the decision itself correct and appropriate?
- Restorative Justice
- Focuses on acknowledging and repairing harms caused
- Moves beyond simply punishing violators
Katsaros et al. (2025, p. 3)
Four Elements of Procedural Justice
When evaluating whether a system is fair, people consider:
- Voice: Opportunity to state their case, present evidence, and tell their side
- Neutrality: Decision makers being unbiased, fact-based, and consistent
- Dignity and Respect: Being treated with courtesy as community members in good standing
- Trust: Feeling that authorities have benevolent motivations and are trying to do what is best
Katsaros et al. (2025, p. 3)
Research Questions
- Are content reporters and users who have content removed distinct populations?
- How do perceptions of fairness affect future user behavior?
- Content removal rates
- Content reporting rates
- Platform visitation frequency
Katsaros et al. (2025, p. 3)
Methods
Participants:
Nextdoor users who recently reported content (N = 2,536) or had content removed (N = 1,004)
Logged Platform Data:
6 months prior + 3 months following survey
Survey Measures:
- Procedural Justice: 10 items (Cronbach’s α = .90)
- Distributive Justice: 2 items (Cronbach’s α = .85)
- Overall Fairness: Single item
- Trust in Nextdoor: Single item
Katsaros et al. (2025, p. 4)
About Nextdoor
What makes Nextdoor unique as a research context:
- Real identities → Users sign up with real names and addresses
- Geographic boundaries → Users only interact with people who live physically nearby
- Community-based moderation → Content is often reviewed by volunteer neighbors rather than platform employees
- Voting system → Volunteer reviewers vote to “Remove,” “Maybe Remove,” or “Keep” content
Katsaros et al. (2025, p. 5)
Results
Finding 1: Populations Overlap Significantly
Content reporters and users with removed content were NOT distinct groups:
- 64% of users with removed content also reported content
- 48% of content reporters also had content removed
Implication: The “target” vs. “perpetrator” framing may be reductive — most users experience moderation from multiple perspectives.
Katsaros et al. (2025, p. 6)
Finding 2: Fairness & Content Reporting
- Higher perceived fairness → increased future reporting behavior
- Both procedural and distributive justice elements contribute to overall perceptions of fairness
- Prior reporting behavior strongly predicts future reporting (standardized = 0.585, p < .001)
Hypotheses 1b and 2b: Supported
Katsaros et al. (2025, p. 7)
Finding 3: Fairness & Content Removal
- No significant relationship between fairness perceptions and future content removals (standardized = −0.032, p = .532)
- Prior removal behavior strongly predicts future removals (standardized = 0.370, p < .001)
Hypotheses 1a and 2a: NOT Supported
This differs from findings on other platforms like Twitter and Facebook.
Katsaros et al. (2025, pp. 6–7)
Finding 4: Fairness & Platform Visitation
- Higher perceived fairness → increased platform visitation (standardized = 0.047, p = .002)
- Users who felt fairly treated visited the platform more frequently in subsequent months
- Prior visitation was the strongest predictor (standardized = 0.824, p < .001)
Hypotheses 1c and 2c: Supported
Katsaros et al. (2025, p. 7)
Summary of Findings
| Hypothesis | Relationship | Result |
|---|---|---|
| Fairness → Reporting | Positive | Supported |
| Fairness → Removal | None | Not Supported |
| Fairness → Visitation | Positive | Supported |
| Prior behavior → Future behavior | Positive | Strong |
Katsaros et al. (2025, pp. 6–7)
Why No Effect on Rule-Breaking?
The authors note this differs from prior studies on Twitter and Facebook. Possible explanations:
- Unique moderation structure → On Nextdoor, decisions are made by volunteer neighbors, not the platform itself
- Localized → Users have deeper offline relationships with those they interact with online
- Real-World stakes → Local issues may have more serious real-life impacts than entertainment-focused platforms
Katsaros et al. (2025, p. 12)
Procedural vs. Distributive Justice
An interesting finding about what matters most on Nextdoor:
“In drawing conclusions about fairness of their moderation experience on Nextdoor, people seem to be equally concerned with the outcome fairness as they are with the fairness of the process.”
This contrasts with Twitter, where procedural elements were much more strongly correlated with overall fairness than distributive elements.
Katsaros et al. (2025, p. 12)
Design Implications
1. Rethink moderation as a user journey
- Users interact with moderation in multiple ways over time
- Move beyond transactional, single-interaction approaches
2. Provide broader education about moderation
- Not just individual report/removal explanations
- System-level transparency → What happens to accounts after reports?
3. Treat rule violators as potential safety stewards
- Rather than “bad actors,” engage them as future contributors
- Empower them with information on how to report violations
Katsaros et al. (2025, pp. 10–11)
The Role of LLMs in Future Moderation
The authors suggest Large Language Models could transform moderation:
“LLMs can be leveraged to change the nature of moderation away from a purely transactional system … toward a more dialogic system where a platform user can converse directly with an LLM to get more information about the platform’s rules, who was involved in making any decisions, how rules get applied to others, or even to help a platform user better structure their appeal.”
Katsaros et al. (2025, p. 11)
Limitations
Methodological concerns:
- Observational design → Can only draw correlational conclusions
- Poor model fit → CFI between 0.833 and 0.853
- ~1/3 of removal participants did not recall ever having content removed
- Non-representative response rates → Reporter survey had higher response than removal survey
- Imprecise timing → Survey sent at single period, not immediately after moderation event
Katsaros et al. (2025, pp. 11–12)
Future Directions
What the field needs:
- Experimental studies with causal designs
- Platform experiments with randomized conditions
- Cross-platform comparisons to understand context-specific effects
- Public release of platform experiment results (i.e., Meta’s 2021 experiment on automation transparency)
Open questions:
- How do anonymous platforms change the fairness calculus?
- How do different moderation structures (platform vs. community) affect fairness perceptions?
- Could LLMs or “AI” systems be utilized safely and effectively to build trust with users?
Katsaros et al. (2025, p. 11)
Takeaway
Fairness in content moderation not only influenced compliance with rules, but also overall engagement with the platform.
“An ideal system of conflict management has multiple goals. The first is to lessen the future occurrence of rule breaking. […] Procedural justice is effective in achieving the goal of resolving a conflict in a way that leads both parties to engage more in the platform in the future.”
Katsaros et al. (2025, p. 12)
The Goal of Moderation
According to the authors, effective moderation should:
- Resolve conflicts in ways that encourage continued platform participation
- Maintain engagement from both content reporters and creators
- Build legitimacy through procedural and distributive fairness
“It is also desirable to manage conflicts about online content in ways that do not drive away those who feel victimized by online posts and those who post content that others find objectionable.”
Katsaros et al. (2025, pp. 12–13)
Discussion Questions
How might these findings apply to larger platforms like Facebook or Twitter where moderation is centralized?
Should platforms prioritize procedural fairness (how decisions are made) or distributive fairness (the outcomes themselves)?
What are the risks of treating rule violators as “potential safety stewards”?
How might LLM-based dialogic systems change users’ fairness perceptions?
