Tomasz Tunguz - AI Design Patterns
Tomasz TunguzVenture Capitalist If you were forwarded this newsletter, and you'd like to receive it in the future, subscribe here. AI Design Patterns
As we’ve been researching the AI landscape & how to build applications, a few design patterns are emerging for AI products. These design patterns are simple mental models. They help us understand how builders are engineering AI applications today & which components may be important in the future. The first design pattern is the AI query router. A user inputs a query, that query is sent to a router, which is a classifier that categorizes the input. A recognized query routes to small language model, which tends to be more accurate, more responsive, & less expensive to operate. If the query is not recognized, a large language model handles it. LLMs much more expensive to operate, but successfully returns answers to a larger variety of queries. In this way, an AI product can balance cost, performance, & user experience.
The evaluation is a topic of much debate today because we lack a gold standard of model greatness. The challenge with evaluating these models is the inputs can vary enormously. Two users are unlikely to ask the same question in the same way. The outputs can also be quite variable, a result of the non-determinism & chaotic nature of these algorithms. Adversarial models will be used to test & evaluated AI. Adversarial models can suggest billions of tests to stress the model. They can be trained to have strengths different to the target model. Just as great teammates & competitors improve our performance, adversarial models play will play that role for AI. The core security around LLMs has two components. A user component, here it’s called a proxy, & a firewall, which wraps the model. The proxy intercepts a user query both on the way out & on the way in. The proxy eliminates personally identifiable information (PII) & intellectual property (IP), logs the queries, & optimizes costs. The firewall protects the model & the infrastructure it uses. We have a minimal understanding of how humans can manipulate models to reveal their underlying training data, their underlying function, & the orchestration for malicious acts today. But we know these powerful models are vulnerable. Other security layers will exist within the stack, but in terms of the query path, these are the most important. The last of our current design patterns in the AI developer design path. The developer’s machine is secured with endpoint detection & response, or EDR, to ensure that the data being used to train models & the underlying models are not poisoned. The developer’s code is sent to a CICD system. The CICD system checks the model & the data are correct using signatures (Sig Verification). Today, most softwares’ signatures are verified. But not AI models. Also, the large language model will be subjected to a testing harness (a series of tests) to ensure that it performs as expected. Real user queries from live traffic will inform the harness. Once those tests pass, the model is pushed to production. These are our four current mental models for how large language models will be built, secured, & deployed. These are sketches of each leg of an elephant we are trying to draw in a dark room. If you have ideas about other design patterns or improvements to the current ones, please contact us. We’d love to improve these to help others. |
Older messages
Standard Issue AI
Friday, February 2, 2024
Tomasz Tunguz Venture Capitalist If you were forwarded this newsletter, and you'd like to receive it in the future, subscribe here. Standard Issue AI “For some companies, [AI is] going to be
Experience as a Status Symbol
Wednesday, January 31, 2024
Tomasz Tunguz Venture Capitalist If you were forwarded this newsletter, and you'd like to receive it in the future, subscribe here. Experience as a Status Symbol Last year, I argued every
The Fastest Growing Software Sectors in 2024
Monday, January 29, 2024
Tomasz Tunguz Venture Capitalist If you were forwarded this newsletter, and you'd like to receive it in the future, subscribe here. The Fastest Growing Software Sectors in 2024 The fastest
AI Drove the Largest New Bookings of Any New Product
Friday, January 26, 2024
Tomasz Tunguz Venture Capitalist If you were forwarded this newsletter, and you'd like to receive it in the future, subscribe here. AI Drove the Largest New Bookings of Any New Product
Dissecting Delegation: Diving Deep on The Missing B-School Class
Wednesday, January 24, 2024
Tomasz Tunguz Venture Capitalist If you were forwarded this newsletter, and you'd like to receive it in the future, subscribe here. Dissecting Delegation: Diving Deep on The Missing B-School
You Might Also Like
the problem with $1M ideas
Friday, July 26, 2024
Read this in 60 seconds I'll cut to the chase. Today we're launching something really, really exciting. Our new course: How To Find A $1M Idea It's a compilation I've learned on how to
Issue #122: Building $1K-$10K MRR Micro SaaS Products around AI SaaS Starter kits, Interactive training videos, AI…
Friday, July 26, 2024
Hi fellow founders! ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏
Learn how to use Bluesky — ahead of our launch 🎉
Friday, July 26, 2024
Plus tips, news & Buffer updates for your social media journey Image Hey there 👋🏾 The subject line probably gave you a hint — and to be honest, I've been tooting this horn for weeks now —
10words: Top picks from this week
Friday, July 26, 2024
Today's projects: Packetriot • Mubert • Tactyqal • Uvodo • Hey.bio • VendorfulAI • AI Tools • Transferhunt • PicStock • Ampifire • Faith Chatbot • Strike.money • GrantsFinders 10words Discover new
⏰ LESS THAN 72 HOURS TO GO
Friday, July 26, 2024
Biggest sale ever for Start & Scale Hi , For the next 72 hours, you can grab our Biggest Sale Ever For Start & Scale which is our single best offer for this course! Ending-Soon Yes, for the
Doing Things that Don’t Scale …Unintentionally — The Bootstrapped Founder 337
Friday, July 26, 2024
What happens when something that was good enough isn't good enough anymore?
What's next for Revolut?
Friday, July 26, 2024
Plus: GB Energy's role in climate tech; 2024's serial entrepreneurs View in browser Sponsor Card - Flagship-23 Good morning there, Who said the summer is slow? Today we unpack the news that
Get SOC 2 certified as an indie hacker
Friday, July 26, 2024
All the details about the process and the cost of getting SOC 2 ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏
SaaSHub Weekly - Jul 25
Thursday, July 25, 2024
SaaSHub Weekly - Jul 25 Featured and useful products FieldPulse logo FieldPulse All-in-one field service management solution #Work Management #Project Management #Field Service Management Uptime Kuma
🍎 30 Tactics to Boost Health as a Time-Crunched Solopreneur
Thursday, July 25, 2024
Balancing the hustle of entrepreneurship with a healthy lifestyle can be tough.