Not Boring by Packy McCormick - Attention is All You Need
Welcome to the 1,205 newly Not Boring people who have joined us over the past week! If you haven’t subscribed, join 194,036 smart, curious folks by subscribing here: Today’s Not Boring is brought to you by… Secureframe Prospects are asking about security earlier in the buying cycle. Companies need a way to get compliant that is faster and takes less employee involvement. With Secureframe you can get compliant in weeks—not months—so you can focus on growing your business. Secureframe makes it fast and easy to get SOC 2, ISO 27001, HIPAA, PCI & GDPR compliant. With guided workflows and one-click integrations, the entire process is automated and backed by a support team of industry experts. Get compliant fast with Secureframe. Click here to set up a demo. Mention “Not Boring” during your demo to get 20% off your first year of Secureframe. Hi friends 👋, Happy Monday! I had a lot of fun writing this one. While I’ve written about AI, I haven’t written much about OpenAI because there’s already so much attention on the company that it’s been hard to find much new to say. GPT-4 is amazing. There will be implications for white collar workers. Here are 75 things you didn’t know you could do with ChatGPT 🧵 👇 Are foundation models defensible? Are GPT Wrappers defensible? Then, on Thursday, OpenAI introduced ChatGPT plugins, and the more I thought about it, the more excited I got. Through a series of seeming accidents, great product, and user pull, OpenAI has attracted attention, turned that attention into the must-use API, and with the release of plugins, is turning that attention into a platform unlike any platform that exists, one that ingests the capabilities of the apps built on it. If OpenAI chooses to, it can build an Apex Aggregator and define the new best business model on the internet. We need to lay some groundwork to explain why. Let’s get to it. Attention is All You NeedThe paper that kicked off the AI Revolution had a catchy title, as these papers go: Attention is All You Need. Written by a team at Google Brain in 2017, the paper introduced the now-famous Transformer architecture that powers large language models such as OpenAI’s GPT-4. As Chroma co-founder Anton Troynikov explained it to me in our first episode of Anton Teaches Packy AI, Transformers can do a lot just by paying attention to the right parts of the input. They don't need other types of neural network layers, like the ones used for convolutions or recurrent connections, to perform well. Attention is all they need. That same year, in a follow-up to his canonical Aggregation Theory, Ben Thompson wrote Defining Aggregators, to catalog businesses that capture value by controlling demand for abundant resources. Aggregators have three defining characteristics:
Think Google and Facebook, the Super-Aggregators. Billions of users come to their sites, where they’re served content created by other people for free, and it gets cheaper to acquire users over time as demand attracts supply which attracts more demand which attracts more supply, and so on. Better experience leads to more users which leads to more suppliers and advertisers which leads to more users, and so on. The takeaway from that post was essentially the same as the Transformer paper: to win big on the internet, attention is all you need. OpenAI began its life in December 2015 as a non-profit. With over $1 billion committed by Elon Musk, Peter Thiel, Sam Altman, Jessica Livingston, Reid Hoffman, Ilya Sutskever, and others, it had no plans to make money. It didn’t need to support itself. One prominent VC who looked at investing early on said he loved the idea and the team, but that with no revenue and no plans to make any, he couldn’t justify investing his LPs’ money in the startup. As recently as last summer, when DALL•E 2 brought OpenAI into the limelight, a team member told me that while their tech was excellent, they weren’t quite sure how they were going to monetize it. Then, in December, OpenAI released ChatGPT. The team wasn’t expecting much – it was a “research preview” – but practically overnight, it caught fire. Its December release was one of those “I remember where I was when” moments for those of us who love this stuff (I was in a hotel lobby in LA and tweeted this after my first experience). ChatGPT exploded. Two months in, it reached 100 million users. Nearly half of the YC W23 class is building apps using OpenAI’s APIs. GPT-4 became the most eagerly anticipated drop since the last Beatles album, and it delivered. Then last week, OpenAI released ChatGPT plugins. They downplayed their foresight on this one too, explaining the launch by saying: “Users have been asking for plugins since we launched ChatGPT.” Since the launch on Thursday, a bunch of people have compared plugins to Apple’s App Store, which paid developers $60 billion in 2022 (and likely took $15 billion for itself). I think it has the potential to be much bigger than that, and I really don’t want to be yet another person who overhypes OpenAI, so I’ll spend this entire piece walking you through why I think so. In under a year, OpenAI has gone from under-the-radar and unsure-how-to-monetize to one of the most potentially dominant strategic positions and business models in the history of tech. It happened almost by accident. While the technology deserves the praise it receives, its translation into a juggernaut business was far from certain until tens of millions of people rushed to use its new consumer-friendly interface. Since then, the dominos have fallen rapidly. When it comes to turning mind-blowing technology into an equally mind-blowing business opportunity, it turns out, you guessed it: attention is all you need. As it stands today, if OpenAI chooses to, it can build the Apex Aggregator by building an Action Engine. The Action Engine subsumes search and any number of products that let users do things, and it does them for users with nothing more than a simple prompt. If Aggregators controlled demand and commoditized supply, the Apex Aggregator can control demand on multiple fronts, turn attention into actions, disaggregate any supplier that feeds it, and even aggregate the aggregators. It could make Apple, Google, and Facebook’s models seem soft and cuddly… until they fight back. I’ll explain what’s gotten me so excited about OpenAI’s accidental business model, but first, we need to do a quick refresher on the current tech business model kings. The Super-AggregatorsIf you’re reading this, I assume you know a lot about how Google and Facebook’s business models – the greatest cash-generating machines known to humanity – work, so I’ll keep this section very brief. Facebook (and its family of apps) use billions of users to attract billions of users, collect a lot of data on those users, and let advertisers target those users based on that data. If you’re a company that wants to get in front of a lot of a certain type of buyer because you think that type of buyer might like your product, Facebook is the place to go. Then, you just serve them a colorful ad, a certain number of them will click on it, and a certain number of those will buy. Because Facebook has the most social attention – over 3 billion users – it attracts more people, and because it has the most people and the most data on them, it attracts the most brand advertising dollars. Both demand and supply cost it practically nothing to acquire, and ads cost it practically nothing to serve. Google’s model is even more direct. Billions of people go to Google explicitly to tell the site exactly what they’re looking for – from information to hotels in certain cities to accounting software – and Google auctions off well-placed links to advertisers who offer exactly what those people are searching for. Those people have expressed intent, and Google rakes in cash from advertisers intent on capturing it. Because Google captures the most search attention – 93% of searches worldwide go through Google – it attracts the most advertising dollars of any business on earth. Both demand and supply cost it practically nothing to acquire, and ads cost it practically nothing to serve. In 2022, Facebook/Meta did $116.6 billion in revenue and Google/Alphabet did $282.8 billion in revenue, the vast majority of which, in both cases, came from serving ads. It’s hard to think of better internet business models. But I think OpenAI just did by accident. Attention with IntelligenceOpenAI’s potential business model is hard to comp directly because it’s so many things at once: an aggregator, an API, and a platform. Imagine that a bunch of apps used Google Search as their core feature, Google was the main place to find those apps, and those apps worked better when you used them in concert with each other through Google, and you’re getting close to what OpenAI might be able to do. OpenAI hits its partners from all angles:
By attracting so much user attention with a genuinely excellent product, OpenAI has also attracted developers to build with its Intelligence API and on its plugin platform. In the short-term, this is great for developers – they get more capabilities and more users – but it’s a sort of Faustian bargain. Developers building on the Intelligence API are relying on OpenAI for their core functionality, and plugin developers are handing their core functionality to OpenAI. In both cases, OpenAI gains more attention, reach, and power. But the bullets above are based on what’s been announced to date. I’ll save speculation for later. For now, let’s go a little deeper into each leg of the stool. Attention Grabber Attention is all you need. And ChatGPT is an attention grabber. Time it took to reach 100 million users worldwide:
Telephone: 75 years
Mobile phone: 16 years
World Wide Web: 7 years
iTunes: 6.5 years
Twitter: 5 years
Facebook: 4.5 years
WhatsApp: 3.5 years
Instagram: 2.5 years
Apple App Store: 2 years
ChatGPT: 2 months Just two months after its launch, it reached 100 million users, the fastest ever to reach that mark by an order of magnitude. While OpenAI hasn’t updated that number, and doesn’t release daily active user numbers, it’s not hard to imagine that in a year, hundreds of millions of people will start their internetting with ChatGPT each and every day. In Defining Aggregators, Thompson wrote, “The key characteristic of Aggregators is that they own the user relationship.” If people go to your site to find anything on the internet, you can tell them where to go. If people open your app to ride, you can tell them whose car to get in. Owning the customer relationship lets aggregators commoditize their suppliers and capture more value from each transaction. ChatGPT (and GPT more broadly) takes this a step further and completely disintermediates its “suppliers.” Google can show you any website its algorithm wants, and it can summarize the website in a prominent box up top, but its job is still to send people to websites. ChatGPT, on the other hand, reads most of the available content on the internet and spits out complete answers to any prompt right in the chat. It’s not just a starting point, but a destination for an increasing percentage of the things a user might want to do online. When ChatGPT came out, there was a fair amount of harrumphing from AI gray hairs that it wasn’t really a technical advance, just a shiny interface on an old model. That missed the point on the product side – interfaces are crucial – but it especially missed the point on the business model side. ChatGPT’s interface was an attention grabber, and once you have attention, you can direct it in any way you choose. Intelligence API ChatGPT didn’t just capture consumer attention, it captured developers’ imaginations. While GPT-3 was already available via API, and while some developers were already building with it, the buzz around ChatGPT pushed seemingly every developer to think about how to incorporate GPT into their products. Something like half of the current YC batch is building on OpenAI’s APIs. These companies are derisively called “GPT wrappers,” and many have wondered about their viability as venture-scale businesses. What worries me about these businesses is how much they’re handing over to an API. In APIs All the Way Down, I pointed out that there seems to be a sweet spot for API-first companies:
Plugging in an API makes a ton of sense when APIs deliver cloud hosting or payments or text messages or any number of things that every company needs to do but that don’t provide a competitive advantage. But what happens when APIs deliver intelligence? OpenAI’s APIs are essentially Intelligence APIs. Write a few lines of code, and your product can do a bunch of things that a pretty smart human could, like answer customer service chats or discover drug formulations or tutor a student. Of course, most of the products that use OpenAI’s APIs do other things besides just delivering ChatGPT in a different interface, and they certainly involve some fine-tuning to better address the customer’s specific needs, but for most of these products, it really feels like they’re breaking the Cardinal Rule of APIs: don’t outsource the thing that makes your beer taste better. But what’s a company to do? Spend hundreds of millions to train their own models in the hopes of producing something different enough to matter? Not make their products intelligent? This is the Chinese Finger Trap that’s captured so many companies before. As I wrote in Shopify and the Hard Thing About Easy Things:
In just a couple of months, we’re already at a point at which there’s no real competitive advantage to using OpenAI’s Intelligence APIs, but there’s a disadvantage for not using them. From OpenAI’s perspective, however, this is an incredible situation. Developers have to pay to use OpenAI’s APIs, and each company that writes those few lines of code to incorporate intelligence into their products makes it even more table stakes for the next one. Plus, it gives OpenAI more attention: now, there are armies of marketers pushing the value of OpenAI’s products on its behalf. Until a week ago, the situation was tenable for GPT wrappers. Some people would be happy to just use ChatGPT and get an ~80% experience on that specific thing – get a good recommendation for a travel itinerary but go execute it themselves – while others would be happy to pay $10/month for an app that gave more tailored recommendations and booked the trip. Then OpenAI announced ChatGPT plugins. To read about ChatGPT plugins, how OpenAI could create the Apex Aggregator, and the choices it’s going to have to make…Thanks to Dan for editing, Anton for teaching me about Attention is All You Need, and Ben Thompson for the insights that keep on giving (and congrats on 10 years!)! That’s all for today. If you found today’s post useful, share it with a friend or two. If you didn’t, thanks for making it all the way down here anyway :) We’ll be back in your inbox on Friday with another Weekly Dose of Optimism. Thanks for reading, Packy Not Boring by Packy McCormick is free today. But if you enjoyed this post, you can tell Not Boring by Packy McCormick that their writing is valuable by pledging a future subscription. You won't be charged unless they enable payments. |
Older messages
Weekly Dose of Optimism #35
Friday, March 24, 2023
Deutsch, Plugins, AI Age, RSV Vaccine, Cowen on the Economy, Succession
The Happiness Hypothesis
Monday, March 20, 2023
Welcome to the 3463 newly Not Boring people who have joined us over the past two weeks! If you haven't subscribed, join 192831 smart, curious folks by subscribing here: Today's Not Boring is
Weekly Dose of Optimism #34
Friday, March 17, 2023
GPT-4, TikTok Ban, Vesuvius Challenge, Varda, Zipline
Weekly Dose of Optimism #33
Friday, March 10, 2023
Superconductor Batteries, US/EU Mineral Alliance, ERC-4337 PaLM-E, Buying Happiness
Internet Computers
Monday, March 6, 2023
The Third Browser War and the Fight to Bring the OS to the Cloud
You Might Also Like
Get a nuclear viral mechanism
Monday, November 25, 2024
Every year we bring the highest quality software to RocketHub for an insane BFCM event. This year is no different! BFCM starts now so check the page below for one new lifetime deeaaal drop each day.
$250K or this...
Monday, November 25, 2024
This could help you get ahead of the crowd... View in browser ClickBank Logo Want to know the one thing separating you from the affiliates making over $250K a year? A lot less than you might think. In
The Arguments For Not Raising at a Unicorn Valuation
Sunday, November 24, 2024
And the top SaaStr news of the week To view this email as a web page, click here The Arguments For Not Raising at a Unicorn Valuation The Arguments For Not Raising at a Unicorn Valuation So for a while
👟When small wins add up
Sunday, November 24, 2024
There's no room for ego in marketing View in browser hey-Jul-17-2024-03-58-50-7396-PM Art and marketing are made for each other, and this week's master proves that. He's done collabs with
Marketing Weekly #207
Sunday, November 24, 2024
3-Step Approach To Get Your SaaS Mentioned on Top Tier Publications • How to Respond to a Savage Review • How TikTok Made Cottage Cheese Cool Again • Why “Why” is the Most Important Word in Marketing •
🙄 Weekend Update: Forbes Fights Back, Flops
Sunday, November 24, 2024
The Weekend Update... ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏
The Profile: The Architect of the Future & the Least Known Operative in America
Sunday, November 24, 2024
This edition of The Profile features Elon Musk, Susie Wiles, Amber Venz Box, Jaylen Brown, and others. ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏
Manufacturers prep for new tariff regime
Sunday, November 24, 2024
Plus: AI and agriculture, European VC valuations on the rise & more Read online | Don't want to receive these emails? Manage your subscription. Log in The Weekend Pitch November 24, 2024 The
Sunday Thinking
Sunday, November 24, 2024
"When the world feels like too much, focus on what you can control: your actions, your mindset, and your heart."
Brain Food: Work Hard In Silence
Sunday, November 24, 2024
FS | BRAIN FOOD November 24, 2024 | #603 | read on fs.blog | Free Version Welcome to Brain Food, a weekly newsletter full of timeless ideas and insights you can use in life and work. Tiny Thoughts *