Verification Badges May Actually Encourage Users to Post Fake News, Study Finds

One potential cause for the explosion of hoaxes, political disinformation, and fabricated clickbait on social media in recent times is the goddamn blue checkmarks, in keeping with new peer-reviewed analysis accepted for publication within the Journal of Management Information Systems.

According to the examine, which is titled “Cure or Poison? Identity Verification and the Posting of Fake News on Social Media,” verifying the identities of social media customers seems to typically decrease their proclivity in the direction of sensationalist or made-up crap. But when a verified consumer is awarded a public-facing badge indicating their standing, researchers discovered that any enchancment within the high quality of data {that a} consumer spreads was “significantly weakened.”

The paper was co-authored by researchers from the City University of New York, Temple University, and the University of Houston. They first sought to find whether or not id verification, typically, impacts a consumer’s propensity to submit doubtful content material. In the midst of their analysis, they found that when a consumer is given a particular badge to point they’ve been verified (one thing like Twitter’s blue checkmark), the outcomes have been totally different.

“It’s a huge problem, it’s one of the biggest problems that we’re dealing with right now,” Min-Seok Pang, a Temple University affiliate professor of administration data methods and the Milton F. Stauffer Research Fellow of its Fox School of Business, stated in a press release. “Fake news is becoming a ‘life-and-death’ matter and eroding trust and respect with each other, which is a backbone of any civilized society.”

The researchers relied on an information set from Weibo, a Chinese social media web site that they wrote was preferrred for this analytical function as a result of it explicitly designates totally different sorts of rule-breaking content material and “actively investigates the posts of fake news reported by other users, along with the evidence provided by the accusers and the author, and discloses the results, which gives us an official way to label fake news.” Additionally, in October 2011, Weibo mandated that each one customers confirm their real-life identities, in contrast to different social media websites that solely confirm sure sorts of customers (or within the case of Twitter, solely a relative handful of celebrities, politicians, and media personalities).

G/O Media could get a fee

All colours on sale at present
Gizmodo describes these premium headphones as “annoyingly incredible.” This is the lowest we’ve seen the Apple AirPods Max yet.

The authors argued that users posting to social media weigh several categories of benefit—psychological gain like attention from peers, political gain as in the case of propaganda, or financial gain such as driving web traffic—as well as the potential negative consequences. In the U.S., that might mean facing a defamation lawsuit, whereas in China, where the internet is heavily monitored by state censors, authorities can charge people who disseminate what is determined to be disinformation with a crime that carries a seven-year prison sentence.

The authors relied on a sample size of about 5,000 Weibo users who had been found by the platform to have posted what it considers fake news, restricted to users who had registered before the October 2011 change and to exclude celebrities and opinion leaders like politicians. The team then reduced the sample to roughly 1,335 users on whom they were able to collect more extensive post histories. To control for the potential impact of commercial or political bias by Weibo moderators—after all, Weibo is constantly monitored by Chinese authorities for what they consider to be misinformation, but might otherwise be viewed as political dissidence—the researchers randomly selected 200 posts flagged as fake to verify the determination was fair. They also wrote that they “excluded all fake news relevant to the government or politics from our sample” to manage for the “sensitivity of fake political news.”

They found that outside of a few instances in which the misinformation in question concerned unverifiable personal information, the posts did in fact contain “fake news” as defined by Weibo: things like false claims, fabricated details, discrepancies from the source material, exaggerations, or information that was outdated, incomplete, or taken out of context. The researchers also conducted a subsequent survey of Weibo and Twitter users in the U.S. to verify some of their prior assumptions, such as the users’ views on different verification methods, or whether they thought verification badges added credibility to posts. The results largely supported their assumptions and hypotheses. For example, 66% of users expressed willingness to post suspicious news if they had a verification badge that would help them get more engagement, as opposed to 43% who would without the badge.

So, while this data set might come from a social and political context that is very different from that of sites based in the U.S., the authors argue it’s a decent baseline to make some observations on user behavior that are “not unique to Chinese users.”

While it may “intuitively” make sense that identity verification without a badge would have little impact on a user’s willingness to post false content, the researchers wrote, it actually has a “negative and [statistically] significant” impact on the amount of false information they shared, because that user knows their online identity can be tied to their offline one. But when a visible verification badge is thrown into the mix, the researchers wrote, it wiped out any sign of those gains:

Our study found that, while identity verification without a verification badge reduces users’ propensity to post fake news, identity verification with a verification badge does not. These mixed effects suggest that the presence of a verification badge for verified accounts significantly weakens the impact of identity verification. More interestingly, we found that when users volunteer to pass identity verification with a verification badge, they post more fake news after verification.

The researchers offered several explanations. For example, users may believe that the boost in perceived credibility to other users brought by the verification badge lowers the risk they will suffer reputational harm or platform consequences from posting fake content. The authors wrote:

One potential explanation is that when users realize that their verified status will be disclosed by a badge after verification, the increased benefit of posting fake news (given by the badge as a signal of credibility) may incentivize some users to actively verify their identities and take more risks to post fake news.

The paper refers to this as a “perverse incentive to post fake news”—the verification system effectively giving a blank check to users intent on gaming the system for personal or financial gain.

Anyone who hasn’t fallen by means of an web rabbit gap in observable actuality ought to acknowledge the relevance of those findings—see the proliferation of verified Twitter bullshit artists stateside. The German Marshall Fund of the United States, for instance, released a study in February 2021 finding that the producers and manipulators of falsified content had respectively tripled and doubled shares of their content on verified Twitter accounts since 2018. Far-right site Gateway Pundit had received more shares from verified Twitter users than the Washington Post. That doesn’t, necessarily, mean that everyone sharing a link was endorsing its content, but that’s a whole other conversation.

The study’s findings also align with previous research generally indicating that the spread of hoaxes and falsified content on social media sites often isn’t an organic phenomenon, but the result of a relatively small but extremely determined group of individuals trying to push a specific viewpoint (such as coronavirus vaccine disinformation, or voter fraud conspiracy theories). Earlier this year, the Oxford Internet Institute released a report that found at least 81 countries were using “social media to spread computational propaganda and disinformation about politics,” with evidence political or government actors in at least 48 of those countries had enlisted private companies to act as “cyber troops.” Perpetrators included both sides of the Libyan civil war, a MAGA group called Turning Point USA dedicated to convincing older Donald Trump supporters that GOP policies are popular with college students, and the failed presidential campaign of billionaire Michael Bloomberg. Tactics ranged from managing armies of bots and professional creation of manipulated media to data-driven targeted advertising of misinformation and harassment/doxxing of activists and journalists.

“When you’re verified, your posts carry more weight, and it’s more damaging when you share fake news,” Pang, the Temple University professor, said in the news release. “While we did not investigate this specifically, it seems like some individuals are using the verification process to game the system. [Social media platforms] have to enforce it more strongly. They have to be open to the possibility that a user has gamed the system and they have to prevent workarounds. We think that is one of the strongest takeaways from this research.”