I. The warning before Blue 

On Thursday evening, after a full day of chaos on the timeline, Elon Musk’s Twitter halted new enrollment into its $8-a-month Blue subscription offering. Offering anyone the chance to slap a “verified” badge on their account had led to widespread impersonation of government officials, corporations, and celebrities. The resulting mayhem, which led to memorable hoaxes from accounts misrepresenting themselves as Eli Lilly, Tesla, Lockheed Martin and others, had triggered an advertiser pullout and a general sense that the platform had descended into chaos.

As the significance of all this began to register with Musk, he tweeted that satirical accounts must now include “parody” in both their name and bio. But if any of the fallout had come as a shock to Musk and his team, they can’t say they weren’t warned. 

Days before the Nov. 9 launch, the company’s trust and safety team had prepared a seven-page list of recommendations intended to help Musk avoid the most obvious and damaging consequences of his plans for Blue. The document, which was obtained by Platformer, predicts with eerie accuracy some of the events that follow.

“Motivated scammers/bad actors could be willing to pay … to leverage increased amplification to achieve their ends where their upside exceeds the cost,” reads the document’s first recommendation, which the team labeled “P0” to denote a concern in the highest risk category. 

“Impersonation of world leaders, advertisers, brand partners, election officials, and other high profile individuals” represented another P0 risk, the team found. “Legacy verification provides a critical signal in enforcing impersonation rules, the loss of which is likely to lead to an increase in impersonation of high-profile accounts on Twitter.”  

On November 1, when the document was circulated internally, Musk was considering a $99-a-year annual subscription for Blue; only later, after an exchange online with writer Stephen King, did he lower the cost. The move wound up increasing the risk for scams, as the desire to make fun of brands and government officials became an impulse buy at $8.

The team also noted removing the verified badge and its related privileges from high-profile users unless they paid, coupled with the heightened impersonation risk, would potentially drive them away from Twitter for good. “Removing privileges and exemptions from legacy verified accounts could cause confusion and loss of trust among high profile users,” they wrote. “We use the health-related protections … to manage against the risk of false-positive actions on high-profile users, under the assumption that the accounts have been heavily vetted. If that signal is deprecated, we run the risk of false positives or the loss of privileges such as higher rate limits resulting in escalation and user flight.” 

The team identified several other risks for which Twitter has yet to identify any solutions. For starters, the company lacks any automated way to remove verified badges from user accounts. “Given that we will have a large amount of legacy verified users on the platform (400K Twitter customers), and that we anticipate we’ll need to debadge a large number of legacy verified accounts  if they decide not to pay for Blue, this will require high operational lift without investment.”

(And this was before the company laid off 80 percent of its contractors, but we’ll get to that.)

The company’s trust and safety team did win support for some solutions, including retaining verification for some high-profile accounts using the “official badge.” 

For the most part, though, the document offers a wish list for features that would make the product safer and easier to use, most of which have not been approved.

It was presented to Esther Crawford, a director of product management at the company who in recent weeks has risen to become one of Musk’s top lieutenants. Musk was briefed as well, sources said, as was his attorney Alex Spiro. And while Crawford appeared sympathetic to many of the concerns in the document, sources said, she declined to implement any suggestions that would delay the launch of Blue. (Crawford did not respond to a request for comment.)

Despite the warnings, the launch proceeded as planned. A few hours later, with the predictions of the trust and safety team largely realized, Musk belatedly stopped the rollout.

II. The contractor wipeout

On Saturday afternoon, a week after an initial round of layoffs had cut Twitter in half, we were the first to report that a second massive wave of cuts had hit the company. This time, the cuts were aimed at Twitter’s contract workers. And on a percentage basis, these losses were even more severe: by the next day, we reported, about 4,400 of 5,500 or so contractors — 80 percent of the team — had lost their jobs.

Functions affected included content moderation, recruiting, ad sales, marketing, and real estate, among others. At the moment, it’s unclear how the loss of what may have been thousands of moderators will affect the service. But it seems clear that Twitter now has dramatically fewer people available to police the site for harmful material.

Unlike Twitter’s full-time employees, who at least got the courtesy of an email informing them that layoffs were coming a night before, contractors received nothing. Neither did their managers, who discovered one by one over the weekend that people they had been counting on to perform critical tasks had suddenly disappeared from the company’s systems. 

“One of my contractors just got deactivated without notice in the middle of making critical changes to our child safety workflows," one manager noted in the company’s Slack channels. This is particularly worrisome because Twitter has for years struggled to adequately police child sexual exploitation material on the platform, as we previously reported.

Over the course of the day, similar messages trickled in on Blind, an app for coworkers to anonymously discuss their workplaces, and on external Slacks that employees have established to have more candid discussions.

Several workers said they had learned about their employment status after seeing our tweets, attempting to log in to Gmail and Slack, and finding that they no longer had access.

“Found out through your tweet and just happened to check,” one person who worked on content moderation told us. “What a lovely Saturday night.”

Said another, who worked in recruiting: “If I didn’t see your tweet just now, I wouldn’t have even known.” 

Some employees told us that they had been bracing for cuts ever since the layoffs earlier this month. But the abrupt nature of the cuts will likely send many former contractors scrambling: as we were first to report, vendors told them via email their medical benefits would end today, their final day of employment.

Meanwhile, the company’s Slack channels over the weekend told a story of already-low morale finding a new basement.

“I’m wondering when people will realize the value of Twitter was the people that worked here,” one employee said, according to screenshots obtained by Platformer. 

Replied another: “In 2 weeks Twitter has gone from being the most welcoming and healthy workplace I’ve ever known to the most openly hostile and degrading I’ve ever known.”

Employees continue to show a great deal of solidarity among one another. But not to the coterie of volunteer venture capitalists and on-loan engineers from Tesla and the Boring Company that have been carrying out Musk’s orders: those they refer to universally, including on Slack, as “the goons.” 

III. Code freeze

On Monday morning, at around 1:45 AM, Twitter engineers were called into an emergency meeting. A new order had just come down from Musk: freeze all production changes on Twitter systems, effective immediately. 

This was more than just a run-of-the-mill code freeze, during which engineers can commit code but not deploy it. Those are fairly common, and Twitter has been under one for most of the time since Musk took over. Such freezes are generally intended to reduce the chances that a bug disrupts Twitter’s systems. 

This time, however, engineers were told they couldn’t even write any code — “until further notice,” according to an internal email obtained by Platformer. Exceptions will be granted if there is an “urgent change that is needed to resolve an issue with a production service, including any changes reflecting hard promised deadlines for clients,” the email said, and employees get “approval from VP level and Elon explicitly stating that the change needs to be made.” 

On Slack, even engineers who attended the late-night meeting were confused. “Is there a ticket I can reference?” asked an engineer who was being tasked with implementing the freeze. “I don’t see any context.” “We don’t have much context as of now,” a colleague responded. “But this is coming from Elon’s team.”

In the meantime, we’re told engineers are writing code locally, on their laptops, and waiting for the freeze to end.

Meanwhile over the last day, Musk has made several public statements about the quality of Twitter’s code and service that have drawn rebukes from current engineers.

“I’d like to apologize for Twitter being super slow in many countries. App is doing >1000 poorly batched RPCs just to render a home timeline!” Musk tweeted on Sunday morning, referring to remote procedure calls. Musk also complained about the number of microservices Twitter employs, which are generally understood to prevent the entire site from breaking every time one part of it goes down. 

Engineer Eric Frohnhoefer pushed back on Musk’s criticism, and offered a detailed thread about why Twitter loads more slowly in some places than others. Musk fired him by the end of the day, Bloomberg reported, along with a second engineer who commented on the affair: “As the former tech lead for timelines infrastructure at Twitter, I can confidently say that this man has no idea wtf he’s talking about.”

Another current engineer explained it to us this way:

The fact that he's focusing on performance being worse in certain countries kind of shows that he doesn't know what he's talking about.

If it was an issue with the microservices talking to each other inside of a data center, everyone in the world would have the same crappy experience.

Instead, the experience is not great in India, for example. That's because the payload gets delivered from further away (laws of physics come into effect) and that back-and-forth data transfer between the phone and the data center starts compounding.

Not to mention that places like India have a higher concentration of low power phones that tend to perform worse in general — as opposed to all of our overpowered iPhones and such.

So why the code freeze? No one knows for sure, but some are speculating that Musk has grown paranoid that some disgruntled engineers may intend to sabotage the site on their way out.

IV. Fallout

On Friday, after the disaster of the Blue rollout, Eli Lilly paused all its ad campaigns on Twitter. The move potentially cost Twitter millions of dollars in revenue, according to the Washington Post. (A “verified” fake account impersonating Eli Lilly had said insulin would now be free, and it took Twitter six hours to remove the tweet.) 

The pharmaceutical giant is one of many large companies pulling ad dollars from Twitter in recent days. Companies including Volkswagen and Pfizer have paused their campaigns, and large advertising firms like IPG’s Mediabrands and Omnicom Media Group are advising clients to do the same. 

The news has left Twitter’s ad teams — particularly those responsible for managing ad agency relationships — in a lurch, according to internal screenshots and conversations with current employees. 

“I know that many of your markets and clients are seeing large declines in Q4 and in particular L7D,” wrote Twitter’s global business lead in Slack. “Please add any commentary, questions, issues in this thread and I’ll endeavor to raise as many as possible TY!”

One employee responded that T-Mobile had requested to “pause the campaigns due to brand safety concerns.” (Three days later, former T-Mobile CEO John Legere asked Elon Musk to let him run Twitter, to which Musk responded simply “no”). 

Another Twitter employee said General Motors had also asked to pause campaigns. “The initial reason they gave is elections, but it looks like an open-ended pause, because the team requested to meet next week to help them make a case to global on why they shouldn’t.” Later, this same employee added: “Pause on [GM] til end of year confirmed and implemented. The reason now is brand safety.”

GroupM, the largest media-buying agency in the world, with $60 billion in annual media spend, told its clients that Twitter was a high-risk media buy, according to Digiday and an email obtained by Platformer. Twitter’s agency partnerships lead explained the situation in Slack: “Given the recent senior departures in key operational areas (specifically Security, Trust & Safety, Compliance), GroupM have updated Twitter’s brand safety guidance to high risk. While they understand that our policies remain in place, they feel that Twitter’s ability to scale and manage infractions at speed is uncertain at this time.”

The employee went on: 

“GroupM will consider lowering the risk grade when the below requirements have been met, which we are working through with leadership:

• Return to baseline levels of NSFW / toxic conversation on the platform

• Re-population of IT Security, Privacy, Trust & Safety senior staff

• Establishment of internal checks & balances

• Full transparency on future development plans of community guidelines / content moderation / anything affecting user security or brand safety

• Demonstrated commitment of effective content moderation, enforcing current Twitter Rules (e.g. account impersonation, violative content removal timing, intolerance of hate speech and misinformation)”

As with the list of product recommendations prepared by trust and safety for the Blue rollout, advertisers appear to have a better understanding of what Twitter needs than Musk does. Massive cuts to the content moderation team, a paralyzing code freeze, and open hostility between the “goons” and the pre-Musk Tweeps have created a company that continues to court a larger crisis.

Mid-afternoon on Monday, after Musk announced he would begin disconnecting up to 80 percent of unspecified microservices, two-factor authentication temporarily stopped working via SMS. Other users reported noticing partial site outages and difficulty downloading their archives.

There are people who know how to fix all those things, but they either no longer work for the company or have been told not to ship any new code. And the question haunting engineers at the end of the day was not whether any new cracks in the service would emerge, but how many, and when.

Governing

• Twitter executives could face significant fines from the FTC, despite Elon Musk’s lawyer trying to reassure employees they wouldn’t be held personally liable if the company disregarded a federal order to consider and protect users' privacy when launching new features. (Diane Bartz and Paresh Dave / Reuters)

• Twitter is reportedly no longer fulfilling key obligations required for it to claim Ireland as its so-called main establishment under the European Union’s General Data Protection Regulation. (Natasha Lomas / TechCrunch)

• The FBI came close to deploying Pegasus, a spyware tool made by Israeli firm NSO, for its own criminal investigations, according to newly released documents. (Mark Mazzetti and Ronen Bergman / New York Times)

• Google agreed to pay a $391.5 million settlement to 40 states over allegations that it tracked users’ locations without their knowledge. (Emma Roth / The Verge)

• North Korea has stolen approximately $1 billion from decentralized crypto exchanges since the start of 2022. The money is helping to fund the government’s illicit ballistic missile and nuclear programs. (Christian Davies and Scott Chipolina / Financial Times)

• Xbox published its first transparency report, saying it took action on more than 4.3 million inauthentic accounts between January and June. (Cecilia D'Anastasio / Bloomberg)

FTX

While we obsess about Twitter, everyone else in tech is fixated on the truly remarkable collapse of the FTX crypto exchange. Some highlights:

• On Friday, FTX announced that CEO Sam Bankman-Fried had stepped down and the company was filing for bankruptcy. (MacKenzie Sigalos / CNBC)

• Then FTX was apparently hacked for $600 million. (Cheyenne Ligon, Sam Reynolds, Sam Kessler, Nikhilesh De, Reilly Decker / CoinDesk)

• FTX said it was moving the remaining customer funds to cold wallets to mitigate the damage. (Shaurya Malwa / CoinDesk)

• But on Saturday, the Securities Commission of the Bahamas says it didn’t direct or authorize FTX to let Bahamas-based customers to withdraw funds — and the withdrawals could be voided in bankruptcy. (Nikhilesh De / CoinDesk)

• FTX’s balance sheet was bad: roughly two-thirds of the money it owed customers was backed by its own tokens that it had made up, per Matt Levine. (Bloomberg)

• Also, Sam Bankman-Fried previously transferred $10 billion of customer funds from FTX to Alameda Research and much of that money has disappeared??? (MacKenzie Sigalos / CNBC)

• And finally: Senior FTX employees who worked in the Bahamas have left for Hong Kong. (Bradley Saacks / Semafor)

Also

• Sam Bankman-Fried was a prominent spokesperson for crypto in Washington. His fall could push legislators to regulate the industry. (Stefania Palma and Scott Chipolina / Financial Times)

• Miami-Dade County and the Miami Heat ended their arena naming rights deal with FTX. We say leave it FTX Arena as a warning about the hubris of man!! (Brian Windhorst / ESPN)

• Crypto exchanges Binance and Huobi blocked deposits of FTT after about $400 million worth of the tokens were released out of schedule, with no official explanation. (Shaurya Malwa / CoinDesk)

• Binance CEO Changpeng Zhao, known as CZ, is the undisputed winner of the FTX collapse. Employees and business partners describe him as a “divisive entrepreneur on a relentless drive to grow at all costs.” (Joshua Oliver and Adam Samson / Financial Times)

• CZ said Binance is setting up a recovery fund to help rebuild the crypto industry after “the cascading negative effects of FTX.” (Sam Reynolds and Jack Schickler / CoinDesk)

Industry

• TikTok is on track to make nearly $10 billion in ad revenue this year, eclipsing rivals like Twitter and Snap. (Kalley Huang, Isabella Simonetti and Tiffany Hsu / New York Times)

• TikTok launched TikTok Shop in the US to compete with Amazon. (Louise Matsakis / Semafor)

• Elon Musk said “exceptional” performers at Twitter will be rewarded with stock options, mimicking his policy at SpaceX. (Alex Heath / The Verge)

• Mark Zuckerberg said Meta would provide immigration support to laid off employees, but those who’ve reached out said they’ve heard “radio silence.” (Pranav Dixit / BuzzFeed)

• Roughly half the Meta layoffs hit people in technology roles, while 54 percent affected people in business positions. (Paresh Dave / Reuters)

• The Meta Quest Pro is better at showcasing VR’s weaknesses than its new strengths, says this review. (Adi Robertson / The Verge)

• Amazon plans to lay off roughly 10,000 people this week from its Alexa division as well as retail and HR. (Karen Weise / New York Times)

• Amazon’s new MK30 drone will go into service in 2024 — a core part of the company’s Prime Air delivery service project. (Umar Shakir / The Verge)

• Netflix is getting into live streaming and Chris Rock will be the first comedian to perform live at an event in 2023. (Peter White / Deadline)

• The founders of AI startup Laserlike, which Apple acquired in 2018, have left to rejoin Google — including one who led Apple’s web search technology division. (Jon Victor / The Information)

• Apple retail employees say the company’s focus on quality customer service has declined, and with more pressure to upsell, they’re unionizing. (Josh Eidelson / Bloomberg)

• YouTube is introducing a new “Live Q&A” feature to make it easier for creators to interact with viewers during livestreams. (Aisha Malik / TechCrunch)

• Mastodon has an “antiviral” design where purposeful friction is meant to slow users down. (Clive Thompson / UX Collective)

• DreamUp, an AI text-to-image generator powered by Stable Diffusion, tried to appease artists by letting them opt out of AI image training, but it set opt-in as the default, which angered many people. (Benj Edwards / Ars Technica)

• Customers pulled funds from Crypto.com after the company said it had mishandled a roughly $400 million transaction, sending it to the wrong type of account on another exchange. (Caitlin Ostroff and Elaine Yu / Wall Street Journal)

  ---

Those good tweets

Tom Zohar @TomZohar

Twitter’s demise is all fun and games until you realize it might happen before the Barbie movie comes out

1:17 AM ∙ Nov 14, 2022

---

12,791Likes1,433Retweets

Brooks Otterlake @i_zzzzzz

Twitter is currently in its "1970s New York" phase... going bankrupt, rule of law collapsing, capital pulling out, dweebs fleeing to lamer places. This is why it's better than ever

9:02 AM ∙ Nov 12, 2022

---

13,541Likes1,543Retweets

Rim 🌸✨🎀 @rim_aj

Elon Musk is a great example of a gamer who thinks he can do a better job than the devs.

5:16 PM ∙ Nov 12, 2022

---

67,660Likes6,410Retweets

Gillian Branstetter @GBBranstetter

"Grandma what was life like during the pandemic"

1:46 AM ∙ Nov 13, 2022

---

33,618Likes3,545Retweets

Talk to us

Send us tips, comments, questions, and your favorite microservices: casey@platformer.news and zoe@platformer.news.