Tomasz Tunguz - How Much Does it Cost to Use an LLM?
Tomasz TunguzVenture Capitalist If you were forwarded this newsletter, and you'd like to receive it in the future, subscribe here. How Much Does it Cost to Use an LLM?
How much does AI cost? It depends on a a few different dimensions, but for a typical use case where a user enters a query & expects about a 200 word response, the cost varies from $0.03 to $3.60. Different models cost different amounts. Also, the size of the context window is an important factor. The context window is a fancy way of saying the quantity of background information the user can provide to the AI. When asking an AI to write a two-paragraph essay in the style of Hemingway, having a few sentences of his writing won’t help much. But perhaps uploading one-third of the Old Man & the Sea imbues Ernest’s style within the prose - no adverbs, simple direct sentences. Longer tracts cost more. A 500 word essay totals $8.40 with the fanciest version of GPT-4 & $0.07 with Llama2, Facebook’s open-source model. There is a 120x difference in cost between the top of the line AI & the free version. Which is better? It depends on the use case & the amount of optimization one is willing to commit to the model. These prices aren’t fixed. Earlier this week, OpenAI announced a 3x reduction in costs - a trend that will continue as engineers optimize the chips, the algorithms, & the systems for AI. Relative to other infrastructure costs, these fees can be significant & understanding the gross-margin impact of this new layer of software will be essential, & will likely push startups from the more expensive larger models to more capital-efficient AI over time. |
Older messages
The Software Industry's Productivity Boom: Analyzing Revenue per Employee Trends
Monday, November 13, 2023
Tomasz Tunguz Venture Capitalist If you were forwarded this newsletter, and you'd like to receive it in the future, subscribe here. The Software Industry's Productivity Boom: Analyzing
Purchasing Patterns in Software : Positivity Punctuating Q3
Friday, November 10, 2023
Tomasz Tunguz Venture Capitalist If you were forwarded this newsletter, and you'd like to receive it in the future, subscribe here. Purchasing Patterns in Software : Positivity Punctuating Q3
Top 10 Trends for Data in 2024
Thursday, November 9, 2023
Tomasz Tunguz Venture Capitalist If you were forwarded this newsletter, and you'd like to receive it in the future, subscribe here. Top 10 Trends for Data in 2024 At the IMPACT Summit yesterday,
Only The Dogged Thrive : More Good News in Startupland
Tuesday, November 7, 2023
Tomasz Tunguz Venture Capitalist If you were forwarded this newsletter, and you'd like to receive it in the future, subscribe here. Only The Dogged Thrive : More Good News in Startupland On the
What's Measured is Managed : Earnings Surprise in Public Software in Q3 2023
Monday, November 6, 2023
Tomasz Tunguz Venture Capitalist If you were forwarded this newsletter, and you'd like to receive it in the future, subscribe here. What's Measured is Managed : Earnings Surprise in Public
You Might Also Like
[VIDEO] Meet the woman who built a $100m+ soda empire
Wednesday, September 18, 2024
Watch her incredible story design-2-header-newsletter Hi there, We're excited to share an incredible new episode of our podcast featuring Allison Ellsworth, the visionary founder of Poppy! 🌟 Meet
🔥 Swipe Right on User Growth!
Wednesday, September 18, 2024
🗓️ You Are Invited! AMA with Sandhya Ganesh, User Growth Product Lead at Tinder
Love To Build? Maybe Don’t Raise Capital.
Wednesday, September 18, 2024
Looking Back On Seedscout's Fundraising Journey ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏
the simple technique that saved my biz
Wednesday, September 18, 2024
Read time: 1 min. 07 sec. Be honest: How distracted do you feel right now? Slack pings, 𝕏 doom scrolling, iPhone buzzing... We live in a world designed to steal our focus. And this sh*t is getting
Ramen Tools and Timesible
Wednesday, September 18, 2024
Comprehensive time and budget tracking tool for teams BetaList BetaList Daily Ramen Tools See what tools other indie makers are using Timesible Comprehensive time and budget tracking tool for teams Too
🌿 What’s Draining Your Productivity? You Might Be Surprised…
Wednesday, September 18, 2024
Staying healthy while working online can be a challenge
There's a new community of SaaS founders growing
Wednesday, September 18, 2024
MicroConf Hey Rob!👋 In just the first few days of its launch, The SaaS Launchpad 🚀 has seen an incredible response from the SaaS community! Over 110 SaaS founders and aspiring entrepreneurs have joined
the future of energy
Wednesday, September 18, 2024
uncover where key players are making their moves, the emerging markets to bet on, and the startups that should be on your radar as we approach 2025 Hi there, Our expert is breaking down emerging energy
Founder Weekly - Issue 654
Wednesday, September 18, 2024
View this email in your browser Founder Weekly Welcome to issue 654 of Founder Weekly. Let's get straight to the links this week. General Non-Obvious Signs of Early Startup Traction — And How to
🏆 He built a $100m ecomm brand
Wednesday, September 18, 2024
And he's teaching you to do the same Hey Friend , You've probably had that dream—the one where you wake up, finally in control of your time, working on your own terms, and building something