Broadcasts.com - "Heres How Anti-Vaxxers Are Spreading Misinformation Despite Your Best Moderation Efforts" (Community Signal)

Business
RELATED

Heres How Anti-Vaxxers Are Spreading Misinformation Despite Your Best Moderation Efforts

Published: Aug. 23, 2021, 9:30 a.m.

What moderation tactics have you used or seen as a mechanism to curtail the spread of misinformation in communities and on social media platforms? Word detection, link blocking, and digital stickers promoting legitimate information sources may immediately come to mind.

But what would happen if you ran your moderation tools against URLs shared in link-in-bio services used in your community? Or what if you learned that folks on your platform were using specific codewords to circumvent word detection? Or posting screenshots of misinformation rather than using plain-text? People are getting creative with how they share all types of information online, misinformation included. Are our moderation strategies keeping up?

In this discussion, Patrick chats with\xa0Joseph Schafer, an undergraduate student of Computer Science and Ethics at the University of Washington and\xa0Rachel Moran, a postdoctoral fellow at the\xa0University of Washington\u2019s Center for an Informed Public. They discuss their research and how anti-vaccine advocates are circumventing content moderation efforts on Facebook, Instagram, Twitter, and large social networks. Some of their findings might surprise you! For example, specific folk theories have emerged that define how some believe social platforms and algorithms work to moderate their content and conversations. And whether these theories are true or not, the strategies forming around them do seem to help people keep questionable content up long enough for researchers to come across it.

So, where do we start? How can we detect misinformation if people are using codewords like pizza or Moana to get around our tools and teams? There may not be precise solutions here just yet, but Rachel and Joseph both offer ideas to help us down the right path, which starts with deciding that the engagement that brews around misinformation is not safe for the long-term health of your community.

Among our topics:

Why Linktree needs community guidelines and how link-in-bio sites have become a vector for misinformation
The folk theories that are informing how we perceive and operate around social media algorithms
Adapting your moderation strategies to better find misinformation

Our Podcast is Made Possible By\u2026

If you enjoy our show, please know that it\u2019s only possible with the generous support of our sponsor:\xa0Vanilla, a one-stop shop for online community.

Big Quotes

Using lexical variation to circumvent moderation filters (2:45):\xa0\u201cThey found this big group of people who were using \u2018dancing\u2019 or other kinds of verbs to mean getting the vaccine. Complete replacement of the word [vaccine]. You wouldn\u2019t know that that meant vaccination unless you were a member of that community and had the institutional knowledge that comes with being a member. We see [lexical variation] on a spectrum.\u201d \u2013@rachelemoran

Emojis, code words, and symbols can form the insider language of a community (3:08):\xa0\u201cWe see \u2018v@ccine\u2019 where the A is an @ sign or people using the vaccine emoji rather than using the word at all. They believe that if they put that instead of spelling out vaccine, \u2026 they\u2019ll avoid being caught up in the algorithmic moderation that happens on platforms.\u201d \u2013@rachelemoran

Misinformation finds a hiding place in link-in-bios (5:05):\xa0\u201cThere\u2019s a variety of ways that you can \u2026 get around [link blocks]. One might be, for example, using a screenshot of an article or something that is vaccine misinformation, rather than putting in the text of the misinformation directly. \u2026 There\u2019s also various websites like URL shorteners or URL compilers, or even just a Word document \u2026 that is filled with links to sites that maybe these major platforms are moderating and blocking.\u201d \u2013@joey__schafer

Using vaccination promotion tools to promote anti-vaccine content (10:56):\xa0\u201c[On Instagram stories, you can use] that little sticker that says, \u2018Let\u2019s get vaccinated.\u2019 Then Instagram collates those of your friends that have [used that] sticker \u2026 and it goes at the top of your [stories section]. \u2026 [We\u2019re seeing people] put a sticker over the top of that sticker or they are like, \u2018Let\u2019s not get vaccinated.'\u201d \u2013@rachelemoran

The engagement surrounding misinformation isn\u2019t good for the long-term health of your community or your business (32:06):\xa0\u201cPart of the problem with misinformation is that it\u2019s really engaging. When you\u2019re making money off of engagement, there\u2019s only so far you\u2019re going to go to take down misinformation without going too far into your bottom line. \u2026 I feel like there is a tide-turning moment happening where the bigger platforms are realizing that misinformation is a vulnerability that degrades the product that can have economic disadvantages.\u201d \u2013@rachelemoran

About Joseph Schafer and Rachel Moran

Joseph Schafer\xa0is a fourth-year undergraduate student at the University of Washington, studying Computer Science and Ethics. He has also worked as a research assistant for the university\u2019s\xa0Center for an Informed Public\xa0since January of 2020, studying various forms of online misinformation and disinformation. Joseph hopes to pursue graduate school in information science, in order to understand how misinformation takes advantage of recently developed socio-technical systems, like social media, to influence our society.

Rachel Moran\xa0is a postdoctoral fellow at the University of Washington\u2019s Center for an Informed Public. Moran received her doctoral degree from the Annenberg School for Communication and Journalism at the University of Southern California. Her research explores the role of trust in digital information environments and is particularly concerned with how trust is implicated in the spread of mis- and dis-information. Her research has been published in academic journals and been covered by the\xa0New York Times,\xa0Vox,\xa0Vice, and others. She was also an affiliate fellow at George Washington University\u2019s Institute for Data, Democracy, and Politics and UNC Chapel Hills\u2019 Center for Information, Technology, and Public Life.