The Fallacy of Centralized Chat Moderation

Citizens Must Act in the Interests of Their Own Safety

Apple is responsible.
Facebook is responsible.
Mark Zuckerberg is responsible.
Telegram and Pavel Durov are responsible.

So say governments the world over, as they increasingly push the burden of content moderation onto centralized platforms and even their founders. CEOs are hauled in front of judicial committees or even imprisoned. Companies are threatened with excommunication by app stores or, worse, blocked by national governments.

As tech companies comply, they burden themselves with an impossible task. And they open the door for more stringent controls, which punish the vast majority of users by throttling their privacy. Bad actors, on the other hand, will always find another way to communicate.

As an open-source, decentralized chat client, White Noise faces a lot of tough questions on moderation: Won’t anonymous chat help criminals communicate? How can you guarantee the safety of users? What about children? Who will protect them?

Our position is this: Encrypted decentralized chat is neutral technology. Just as anyone can communicate in the physical world without restriction, so can anonymous users online. In any ecosystem, users themselves will develop methods and practices to discourage and report harmful content. Only users themselves can judge the value or harm of communications.

---

The Flawed Politics of Top-Down Moderation

Below is a concise overview of the most widely used social platforms that include an E2EE (end-to-end encrypted) chat / direct‑messaging feature.

WhatsApp: Moderation relies on user reports (spam, harassment, illegal content). Because the provider cannot read message bodies, it can only act on metadata (e.g., number of reports, abusive account behavior) and may suspend or ban offending accounts.

Signal: Strictly user‑report driven. Signal can block or delete accounts that violate its terms, but it cannot inspect message content. It also uses rate‑limiting and phone‑number verification to curb spam.

Telegram (partially E2EE): Automated AI‑based scanning of non‑secret chats for spam, phishing, extremist content, plus user reports. Secret chats are exempt from server‑side scanning; moderation of those relies entirely on user reports and account bans.

Facebook Messenger (optional encryption): A combination of AI content detection, human review, and user reports. Even for secret conversations, Facebook can act on metadata (e.g., repeated abuse flags) and may disable the account.

Viber: User‑report system; Viber can suspend accounts that repeatedly receive abuse reports. Since content is unreadable, moderation is limited to metadata and account actions.

iMessage (Apple): On‑device machine learning detects spam/phishing and offers “Report Junk”. Apple can act on metadata and user reports; it does not have server‑side access to message content.

Why the moderation approaches differ

Approaches often differ due to differences in functionality or jurisdiction. They also vary due to these factors:

1. Technical limitation — When a platform offers true E2EE, it cannot scan message bodies, so it must rely on metadata analysis (frequency of reports, account‑creation patterns) and user‑initiated reports.

2. Regulatory pressure — Services operating globally (especially in the EU, US, or China) face legal obligations to remove illegal content (e.g., child sexual abuse material, extremist propaganda). Non‑E2EE platforms can comply by scanning content automatically; E2EE platforms must balance privacy with lawful‑access requests, often by providing account‑level takedowns.

3. Product design goals — Consumer‑messaging apps prioritize privacy (Signal, WhatsApp, iMessage), whereas social‑networking or community platforms (Discord, Reddit, Facebook) prioritize community health and therefore invest heavily in AI‑driven moderation pipelines.

Even if the same laws applied to all chat clients, forcing them to moderate certain content before it was viewed by users, calls to moderate further would still likely increase. Offense is subjective, and users will still feel the urge to report and ban content they do not like. Essentially, we want to be the ones in control of our own conversations.

AI content moderation falters because models can’t reliably understand nuance—sarcasm, cultural references, or author intent, leading to false positives and negatives. Their training data often contains biases and is heavily English‑centric, so minority languages and regional contexts receive poorer coverage. Bad actors exploit these gaps with evasive tricks (misspellings, emojis, image‑embedded text), while the systems’ need for real‑time speed forces them to use simpler heuristics that miss subtle abuse. Moreover, the black‑box nature of many models makes decisions opaque, hindering appeals and accountability, and the rapid evolution of societal norms outpaces static training sets, requiring constant human oversight and costly retraining.

“Typically, [moderation] is performed by the corporation owning the network in conjunction with the government where the company exists. These two parties have all the power over who gets to speak and who doesn’t. The rules are often unclear, unequally enforced, and there is little or no transparency or recourse for injustice.” Matt Lorentz, Nos.social.

And even if content moderation can be outsourced to humans, there are huge economic and psychological costs to pay. Court filings from December 2024 showed 144 Facebook moderators diagnosed with PTSD, anxiety, and depression.

Further, binary definitions of what is and is not acceptable online are flawed in two ways. Categorizing language and media is never 100% objective, and with the added layer of AI-generated video, audio, text, and image, deciphering content categories at scale will not be possible, either through automated or analog means.

The second flaw is that when the onus for moderation lies with the platform owners (who don’t want to get sued), tech companies are coerced into verifying ages and identities. Restricting access to one platform does not change the behavior or intentions of malicious accounts.

Despite the calls to “end the veil of anonymity” online made by the Spanish Prime Minister last year, our right to privacy is not an acceptable trade for increased protection against harmful content.

“You have created a world where only the neutered are safe and where only outlaws are free… We expect crime and disagreements and ugliness, and we are prepared to deal with them.” The Declaration of Separation, anonymous author**

No amount of centralized control will keep citizens safe online; top-down moderation will only decrease the freedoms of the majority who act within the bounds of decency.

How Decentralized Moderation Works

You cannot control what you see or hear on the street, but you can control how you react. Think about those who swear or use racist language in front of children, or even perverts who expose themselves to others in public. These acts can be alarming, but one blanket approach does not solve both issues. Not all sentiment can be banned. The swearing may be ignored or discouraged. The racist language may provide an opportunity for a parent to talk to their child. The illegal act of indecent exposure is likely to be reported to the authorities.

However, if all children (or adults, for that matter) had to wear helmets that filtered interactions and only permitted identified, trusted people to interact, we can easily see this as an unacceptable reduction of freedom.

With other decentralized networks (for example, Bitcoin), maintainers and gatekeepers are incentivized by the value that stability and quality afford. And as adoption widens, use cases move from fringe operators to institutions, legitimizing the technology.

Still, moderating language can prove problematic. NIP‑68 (picture‑first feeds) and NIP‑69 (peer‑to‑peer order events) proposed vocabulary for content warnings and reporting on Nostr in 2023. During the review process, several contributors expressed strong opposition to merging them, arguing that they would force a strict vocab‑code scheme on all Nostr users and could fragment the ecosystem. Ultimately, the pull request was not merged, and the proposals remain unaccepted. Subsequent work introduced NIP‑32, which adopts parts of the labeling approach originally intended for NIP‑69.

Decentralized moderation works through users developing their own reporting practices and filters. Primal, for example, launched a content filtering mechanism where users can mute, block, and report content to create their own personalized user experience.

“The final word on content moderation should belong to the user. Public mute lists and open source filtering algorithms are great moderation building blocks on an open network like Nostr. We built the capability to subscribe to other users’ mute lists, as well as opt in and out of filtering algos in a granular fashion.” Miljan Braticevic, Founder of Primal.

---

10 reasons why White Noise users will be motivated to block harmful content.

1. Guard the people you know — When a friend or a contact in your trust circle is targeted, you’ll want to stop the abuse before it harms them.

2. Preserve the credibility of your network — A reputation for safe, civil conversation keeps the people you’ve invited (family, coworkers, activist groups) willing to stay and invite others.

3. Honor mutual responsibility — In a web‑of‑trust model, each participant implicitly promises to look out for the others; reporting is the practical way to keep that promise.

4. Prevent collateral damage to contacts — Spam, scams, or extremist propaganda can affect users you know, exposing them to fraud or legal risk.

5. Maintain usable signal‑to‑noise for your circles — Trusted groups rely on clear, relevant messages; removing noise makes the chat useful for everyone in the network.

6. Avoid being blamed for “hosting” illegal material— Even in a decentralized system, authorities may trace a relay that repeatedly distributes illicit content; reporting helps keep your relay, and the contacts that route through it, safe.

7. Strengthen the web‑of‑trust itself — Every successful report reinforces the idea that the network self‑polices, encouraging more users to join and link their contacts.

8. Contribute to shared tooling— Reports feed open‑source moderation filters that the whole community (including your contacts) benefits from, improving safety for all.

9. Reduce personal exposure to trauma — By flagging harmful posts early, you spare yourself and close contacts from repeatedly encountering disturbing material.

10. Demonstrate stewardship of the community’s values — Acting on abuse signals to your peers that you take the collective standards seriously, reinforcing the social contract that binds the network together.

Each report is a tiny vote on what the community deems “valuable” versus “harmful.” Because the votes come from people who already trust one another (friends, colleagues, allies), they carry extra weight and are instantly visible to the same circles. Over time, these votes coalesce into an informal, yet enforceable, social contract:

Conclusion

The moderation of Big Tech platforms with millions of users is an impossible task. Not only that, it leads only in the direction of more restrictions, tighter control, and less privacy. The only surefire way individuals can remain both in charge and safe is to manage their own experience. All users ultimately act in their own interests and have a vested interest in moderating their own feeds and chats.

Resilient communities build ways to protect themselves and ensure that bad actors are deterred. At White Noise, we encourage this and will continue to work with our community to promote moderation, reporting, and self-management of the content users see. For now, the desire to curate your own experience prompts users to act, but in the future, other methods and value incentives may emerge.

With decentralized infrastructure, we take control of moderation, avoiding the pitfalls of major platforms and the anti-privacy measures that restrict our agency.

How we act and who we associate with is up to us.