Tomasz Tunguz - Whispers of the Future
Tomasz TunguzVenture Capitalist If you were forwarded this newsletter, and you'd like to receive it in the future, subscribe here. Whispers of the Future
I’ve been fascinated by dictation software for many years. Speaking has struck me as the most natural interface. It also happens to be at least 3 times faster than typing. When OpenAI released their dictation model Whisper, I was keen to try it. So I installed it on my MacBook Pro. Over the last few weeks, I’ve been using it. As expected, it’s sensational. In fact, this entire blog post is dictated using Whisper. The major advantage of using one of these large models is that it recognizes words in context in ways older systems struggle. For example, saying the words ’three-hundred million’ might produce 3000000000 or 3 hundred million or three hundred million. I know which one I want to see you when I speak it, but it’s hard for the computer to know. Words like SaaS might be SAS the software company, SaaS like Software as a Service, or sass, meaning someone who is cheeky or rude. The context matters to disambiguate. The better a computer can predict which one is the right one, the more natural the interaction becomes, keeping the user in the flow. But these models require huge amounts of hardware to run. My MacBook Pro has 64GB of RAM & uses one of the most powerful Mac GPUs, the M1 Max. Even using the smallest model, the computer can struggle to manage memory & Whisper crashes frequently. I wondered how slower Mac hardware is compared to Nvidia. While benchmarks are often fraught with nuances, the consensus amongst testers is Nvidia is about 3x faster to run these models. Apple optimize their chips for power consumption while Nvidia opts for raw performance. In addition, many of the core machine learning libraries have not yet rewritten natively for Apple. Setting the hardware concerns aside, LLMs will transform dictation software. The big question is how to deploy them. These models require significant horsepower to run which implies a few options :
My bet is on the fourth. This hybrid architecture will allow some usage when not connected to the Internet, but take advantage of a cloud when available. In addition, it reduces the serving cost for the provider. The user’s computer works, which is free to the SaaS dictation software company, improving margins. With inference costs likely the vast majority of cloud expenses as a vendor, this hybrid architecture will be a key competitive advantage. |
Older messages
The Rise of Data Lakes in Software Architecture
Thursday, September 7, 2023
Tomasz Tunguz Venture Capitalist If you were forwarded this newsletter, and you'd like to receive it in the future, subscribe here. The Rise of Data Lakes in Software Architecture Historically,
The Evolution of Open Source Business Models
Tuesday, September 5, 2023
Tomasz Tunguz Venture Capitalist If you were forwarded this newsletter, and you'd like to receive it in the future, subscribe here. The Evolution of Open Source Business Models Open source
Context.ai - Unlocking Insight into LLM-Based Applications
Wednesday, August 30, 2023
Tomasz Tunguz Venture Capitalist If you were forwarded this newsletter, and you'd like to receive it in the future, subscribe here. Context.ai - Unlocking Insight into LLM-Based Applications
A New Marketing Behemoth Klaviyo : How 7 Key Benchmarks Stack Up in the S-1
Monday, August 28, 2023
Tomasz Tunguz Venture Capitalist If you were forwarded this newsletter, and you'd like to receive it in the future, subscribe here. A New Marketing Behemoth Klaviyo : How 7 Key Benchmarks Stack
A New Marketing Behemoth Klayviyo : How 7 Key Benchmarks Stack Up in the S-1
Monday, August 28, 2023
Tomasz Tunguz Venture Capitalist If you were forwarded this newsletter, and you'd like to receive it in the future, subscribe here. A New Marketing Behemoth Klayviyo : How 7 Key Benchmarks Stack
You Might Also Like
🗞 What's New: YouTube just launched Hype to help small creators get discovered
Thursday, September 19, 2024
Also: Running newsletter ads!
SaaSHub Weekly - Sep 19
Thursday, September 19, 2024
SaaSHub Weekly - Sep 19 Featured and useful products Schedul Threads logo Schedul Threads Boost your Threads following, reach monetization status fast and automate your Threads content publishing for
95 new Shopify apps for you 🌟
Thursday, September 19, 2024
New Shopify apps hand-picked for you 🙌 Week 37 Sep 9, 2024 - Sep 16, 2024 New Shopify apps hand-picked for you 🙌 What's New at Shopify? 🌱 New granular staff permissions for Gift cards Improvement ⸱
[SaaS Club] How firing customers built a 7-figure SaaS
Thursday, September 19, 2024
The SaaS Club Newsletter Hey Reader Here's a quick round up of what's been going on at SaaS Club: In this week's newsletter: 🎧 Discover how firing customers fueled SaaS growth 📈 Turn
The MrBeast Guide
Thursday, September 19, 2024
Today's newsletter is brought to you by LCA. We help companies build their future. From new products to websites, apps, and AI solutions. We work with businesses doing $10M-$10B in revenue. We
Writing Software for Robots
Thursday, September 19, 2024
Tomasz Tunguz Venture Capitalist If you were forwarded this newsletter, and you'd like to receive it in the future, subscribe here. Writing Software for Robots In a few years, most feature flags
KoppieOS, ScrapX, Notch, Magic Inspector, and Entyx
Thursday, September 19, 2024
KoppieOS, where Ai meets Personality. BetaList BetaList Daily Magic Inspector Automate browser testing in natural language KoppieOS KoppieOS, where Ai meets Personality. ScrapX Monitor your competitors
“I’m waiting for the right time”
Thursday, September 19, 2024
Read time: 1 min. 03 sec. "Hey Pat, can I talk to you for a second?" My buddy pulled me aside at a wedding last weekend. I already knew what he wanted to ask... "I really want to do what
Mastering Portfolio Construction
Thursday, September 19, 2024
How to build a strategy that maximizes returns. ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏
join me: Big Tech in Fintech
Thursday, September 19, 2024
thought you would be interested Hi there, Laura here, Principal Analyst at CB Insights. Thought you would be interested in this new briefing on big tech's activity in financial services. In under