Programming note: Platformer is off Monday for Labor Day.

Zoe Schiffer and I reported this story for The Verge.

It had been a long pandemic for Twitter’s research team. Tasked with solving some of the platform’s toughest problems around harassment, extremism, and disinformation, staffers absconded to Napa Valley in November 2021 for a company retreat. Despite a tumultuous change in leadership — Jack Dorsey had recently stepped down, appointing former chief technology officer Parag Agrawal to take his place — the group felt unified, even hopeful. After months of fighting bad actors online, employees took a moment to unwind. “We finally felt like we had a cohesive team,” one researcher says.

But at the goodbye brunch on the last day, people’s phones started pinging with alarming news: their boss, Dantley Davis, Twitter’s vice president of design, had been fired. Nobody knew it was coming. “It was like a movie,” says one attendee, who asked to remain anonymous because they are not authorized to speak publicly about the company. “People started crying. I was just sitting there eating a croissant being like, ‘What’s up with the mood?’”

The news foreshadowed a downward spiral for the research organization. Although the group was used to reorganizations, a shakeup in the middle of an outing meant to bond the team together felt deeply symbolic.

The turmoil came to a head in April, when Elon Musk signed a deal to buy Twitter. Interviews with current and former employees, along with 70 pages of internal documents, suggest the chaos surrounding Musk’s acquisition pushed some teams to the breaking point, prompting numerous health researchers to quit, with some saying their colleagues were told to to deprioritize projects to fight extremism” in favor of focusing on bots and spam. The Musk deal might not even go through, but the effects on Twitter’s health efforts are already clear.

The health team, once tasked with fostering civil conversations on the famously uncivil platform, went from 15 full-time staffers down to two.

In 2019, Jack Dorsey asked a fundamental question about the platform he had helped create: “Can we actually measure the health of the conversation?”

Onstage at a TED conference in Vancouver, the beanie-clad CEO talked earnestly about investing in automated systems to proactively detect bad behavior and “take the burden off the victim completely.”

That summer, the company began staffing up a team of health researchers to carry out Dorsey’s mission. His talk convinced people who’d been working in academia, or for larger tech companies like Meta, to join Twitter, inspired by the prospect of working toward positive social change. 

When the process worked as intended, health researchers helped Twitter think through potential abuses of new products. In 2020, Twitter was working on a tool called “unmention” that allows users to limit who can reply to their tweets. Researchers conducted a “red team” exercise, bringing together employees across the company to explore how the tool could be misused. Unmention could allow “powerful people [to] suppress dissent, discussion, and correction” and enable “harassers seeking contact with their targets [to] coerce targets to respond in person,” the red team wrote in an internal report. 

But the process wasn't always so smooth. In 2021, former Twitter product chief Kayvon Beykpour announced the company’s number one priority was launching Spaces. (“It was a full on assault to kill Clubhouse,” one employee says.) The team assigned to the project worked overtime trying to get the feature out the door and didn’t schedule a red team exercise until August 10th — three months after launch. In July, the exercise was canceled. Spaces went live without a comprehensive assessment of the key risks, and white nationalists and terrorists flooded the platform, as The Washington Post reported. 

When Twitter eventually held a red team exercise for Spaces in January 2022, the report concluded: “We did not prioritize identifying and mitigating against health and safety risks before launching Spaces. This Red Team occurred too late. Despite critical investments in the first year and a half of building Spaces, we have been largely reactive to the real-world harms inflicted by malicious actors in Spaces. We have over relied on the general public to identify problems. We have launched products and features without adequate exploration of potential health implications.”

Earlier this year, Twitter walked back plans to monetize adult content after a red team found that the platform had failed to adequately address child sexual exploitation material. It was a problem researchers had been warning about for years. Employees said that Twitter executives have been aware of the problem but noted the company has not allocated the resources necessary to fix it. 

By late 2021, Twitter’s health researchers had spent years playing whack-a-mole with bad actors on the platform and decided to deploy a more sophisticated approach to dealing with harmful content. Externally, the company was regularly criticized for allowing dangerous groups to run amok. But internally, it sometimes felt as though certain groups, like conspiracy theorists, were kicked off the platform too soon — before researchers could study their dynamics.

“The old approach was almost comically ineffective, and very reactive — a manual process of playing catch,” says a former employee, who asked to remain anonymous because they are not authorized to speak publicly about the company. “Simply defining and catching ‘bad guys’ is a losing game.”

Instead, researchers hoped to identify people who were about to engage with harmful tweets, and nudge them toward healthier content using pop-up messages and interstitials. “The pilot will allow Twitter to identify and leverage behavioral — rather than content — signals and reach users at risk from harm with redirection to supportive content and services,” read an internal project brief, viewed by The Verge.

Twitter researchers partnered with Moonshot, a company that specializes in studying violent extremists, and kicked off a project called Redirect, modeled after work that Google and Facebook had done to curb the spread of harmful communities. At Google, this work had resulted in a sophisticated campaign to target people searching for extremist content with ads and YouTube videos aimed at debunking extremist messaging. Twitter planned to do the same. 

The goal was to move the company from simply reacting to bad accounts and posts to proactively guiding users toward better behavior.

Read the rest of this story at The Verge.

Governing

• Twitter began testing an edit button, raising fears that it could provide a new vector for misinformation. It bears watching, but generally I think the fears are overblown and Twitter owed this to the user base, for whom this was has been the No. 1 feature request for many years. (Rachel Lerman and Will Oremus / Washington Post)

• The United Kingdom’s Competition and Markets Authority said it will require an in-depth review of Microsoft’s planned $69 billion takeover of Activision Blizzard unless Microsoft can allay its fears by Sept. 8. Good luck with that! (Katherine Gemmell / Bloomberg)

• The UK’s Information Commissioner’s Office said it would mount a new effort to require porn sites and other adult-only services to verify user ages to prevent children from accessing them. (Olivia Solon / Bloomberg)

• Google and YouTube outlined their plans to handle misinformation during the US midterm elections, saying they would add information panels and voting information to search. (Lauren Forristal / TechCrunch)

• Misinformation ranks third behind climate change among the major concerns of more than 24,000 people surveyed in 19 countries, according to new research from Pew. (Stuart A. Thompson / New York Times)

• Google, Apple, and other tech giants have begun to reduce their reliance on China for manufacturing amid ongoing geopolitical tensions. (Daisuke Wakabayashi and Tripp Mickle / New York Times)

• Google defended its “Jedi Blue” advertising deal with Meta in court, arguing there was nothing illegal about giving Facebook advantages in its ad market. (Leah Nylen / Bloomberg)

• House Republicans demanded that Mark Zuckerberg hand over any warnings the FBI may have issued to Meta related to Hunter Biden’s laptop in the weeks before the 2020 election. (Billy House / Bloomberg)

• “The European Commission said that at least 15 different component parts should be made available for at least five years from the date of a smartphone’s introduction to the market and that batteries should survive at least 500 full charges without deteriorating to below 83 per cent of their capacity.” (Alice Hancock and Tim Bradshaw / Financial Times)

• “California Gov. Gavin Newsom is set to sign a recently passed bill that would require digital asset exchanges and other crypto companies to obtain a license to operate in the state.” (Sam Reynolds / CoinDesk)

• Music publishers unexpectedly reached an agreement with streaming services over the rate they will be paid from 2023 to 2027, avoiding a protracted legal battle. (Jem Aswad / Variety)

• A look at Fog Reveal, a surveillance tool that police have been using without first getting a warrant to track people’s movements back in time over several months. (Garance Burke / Jason Derean / AP)

• Google announced $20 million to support computer science education in the United States. (Monica Chin / The Verge)

Industry

• ByteDance told employees it currently has no plans to go public, amid a failure to meet business goals and a pullback in non-core-spending. (Coco Feng / South China Morning Post)

• OnlyFans revenue rose 160 percent to $932 million year over year, and creators earned nearly $4 billion collectively. It’s the biggest success in the creator economy since YouTube, which pays out about $10 billion a year. (Kaya Yurieff / The Information)

• Netflix’s ad-supported tier cold go live Nov. 1 in the United States, Canada, the United Kingdom, Germany, and France. (Todd Spangler / Variety)

• A borderline mean-spirited review of Mark Zuckerberg’s appearance on “The Joe Rogan Experience” that nonetheless connects some important dots between the slow development of the virtual reality market and Zuckerberg’s insistence on being the face of it. (Max Chafkin / Bloomberg)

• Facebook is shutting down its Nextdoor clone next month after tests in the United States and Canada. It seems like if anyone would be able to make local work in this way, it would be Facebook, no? (Sarah Perez / TechCrunch)

• Here’s a nice requiem for Zenly, a beautiful social mapping app that Snap never quite figured out to do with despite its surprising popularity. (Paul Sawers and Romain Dillet / TechCrunch)

• Reddit bought ad tech company Spiketrap for an undisclosed sum. (Sarah Perez / TechCrunch)

• An AI-generated artwork won first place at the Colorado State Fair’s fine art competition, outraging other artists. (Matthew Gault / Vice)

Those good tweets

s*d @06fordexplorer

your honor it was not a HIPAA violation i was simply BeingReal

August 31st 2022

13,127 Retweets163,079 Likes

xanax in a cinnabon (ali) @xanabon

i love when my cat pushes open the bathroom door the same way a cowboy would bust open the doors to a saloon

August 31st 2022

14,137 Retweets147,993 Likes

shane @stinkylittlehog

bartender: the usual my man? me: yessir bartender: [kisses me on the forehead]

August 30th 2022

21,439 Retweets189,521 Likes

pain @vibinpain

I’m not too good at pushups or situps but fuck ups ? let me tell ya

August 30th 2022

22,727 Retweets110,255 Likes

jim rose circus @jimrosecircus1

They should announce a sequel to Groundhog Day and then just re-release the original.

August 30th 2022

10,413 Retweets117,846 Likes

Talk to me

Send me tips, comments, questions, and edited tweets: casey@platformer.news.