Tomasz Tunguz - AI Design Patterns
Tomasz TunguzVenture Capitalist If you were forwarded this newsletter, and you'd like to receive it in the future, subscribe here. AI Design Patterns
As we’ve been researching the AI landscape & how to build applications, a few design patterns are emerging for AI products. These design patterns are simple mental models. They help us understand how builders are engineering AI applications today & which components may be important in the future. The first design pattern is the AI query router. A user inputs a query, that query is sent to a router, which is a classifier that categorizes the input. A recognized query routes to small language model, which tends to be more accurate, more responsive, & less expensive to operate. If the query is not recognized, a large language model handles it. LLMs much more expensive to operate, but successfully returns answers to a larger variety of queries. In this way, an AI product can balance cost, performance, & user experience.
The evaluation is a topic of much debate today because we lack a gold standard of model greatness. The challenge with evaluating these models is the inputs can vary enormously. Two users are unlikely to ask the same question in the same way. The outputs can also be quite variable, a result of the non-determinism & chaotic nature of these algorithms. Adversarial models will be used to test & evaluated AI. Adversarial models can suggest billions of tests to stress the model. They can be trained to have strengths different to the target model. Just as great teammates & competitors improve our performance, adversarial models play will play that role for AI. The core security around LLMs has two components. A user component, here it’s called a proxy, & a firewall, which wraps the model. The proxy intercepts a user query both on the way out & on the way in. The proxy eliminates personally identifiable information (PII) & intellectual property (IP), logs the queries, & optimizes costs. The firewall protects the model & the infrastructure it uses. We have a minimal understanding of how humans can manipulate models to reveal their underlying training data, their underlying function, & the orchestration for malicious acts today. But we know these powerful models are vulnerable. Other security layers will exist within the stack, but in terms of the query path, these are the most important. The last of our current design patterns in the AI developer design path. The developer’s machine is secured with endpoint detection & response, or EDR, to ensure that the data being used to train models & the underlying models are not poisoned. The developer’s code is sent to a CICD system. The CICD system checks the model & the data are correct using signatures (Sig Verification). Today, most softwares’ signatures are verified. But not AI models. Also, the large language model will be subjected to a testing harness (a series of tests) to ensure that it performs as expected. Real user queries from live traffic will inform the harness. Once those tests pass, the model is pushed to production. These are our four current mental models for how large language models will be built, secured, & deployed. These are sketches of each leg of an elephant we are trying to draw in a dark room. If you have ideas about other design patterns or improvements to the current ones, please contact us. We’d love to improve these to help others. |
Older messages
Standard Issue AI
Friday, February 2, 2024
Tomasz Tunguz Venture Capitalist If you were forwarded this newsletter, and you'd like to receive it in the future, subscribe here. Standard Issue AI “For some companies, [AI is] going to be
Experience as a Status Symbol
Wednesday, January 31, 2024
Tomasz Tunguz Venture Capitalist If you were forwarded this newsletter, and you'd like to receive it in the future, subscribe here. Experience as a Status Symbol Last year, I argued every
The Fastest Growing Software Sectors in 2024
Monday, January 29, 2024
Tomasz Tunguz Venture Capitalist If you were forwarded this newsletter, and you'd like to receive it in the future, subscribe here. The Fastest Growing Software Sectors in 2024 The fastest
AI Drove the Largest New Bookings of Any New Product
Friday, January 26, 2024
Tomasz Tunguz Venture Capitalist If you were forwarded this newsletter, and you'd like to receive it in the future, subscribe here. AI Drove the Largest New Bookings of Any New Product
Dissecting Delegation: Diving Deep on The Missing B-School Class
Wednesday, January 24, 2024
Tomasz Tunguz Venture Capitalist If you were forwarded this newsletter, and you'd like to receive it in the future, subscribe here. Dissecting Delegation: Diving Deep on The Missing B-School
You Might Also Like
Broken engine?
Saturday, September 7, 2024
Hey friends, I'm back from my two-week trip to the US. I've been meaning to write you an update about my experiences in the podcast industry, hanging out with my co-founder in Chicago, and
Hunch
Saturday, September 7, 2024
A note app for product people to help with the discovery process BetaList BetaList Daily Hunch A note app for product people to help with the discovery process Too much email in your life? Switch to
💄 Building a million-dollar beauty brand
Saturday, September 7, 2024
Alicia Scott's inspiring story Hey Friend , Does it ever feel like your entrepreneurial dreams are just out of reach? Maybe you feel like the problem you're solving is too niche. Or the market
3+ Years of daily posting on X: Top Lessons
Saturday, September 7, 2024
+ DNA School Invite
What’s 🔥 in Enterprise IT/VC #410
Saturday, September 7, 2024
Speed mode + also, is the great unbundling of SaaS powered by agentic workflows? ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏
🗞 What's New: Launch tips from Product Hunt's Head of Growth
Saturday, September 7, 2024
Also: Increasing your luck as a founder!
🚀 BlackSky To Reverse Split
Saturday, September 7, 2024
Plus Lynk Global delists from the Nasdaq, $ASTS mixed shelf offering, $PL and $SATX earnings, and more! The latest space investing news and updates. View this email in your browser The Space Scoop Week
my crazy idea…
Friday, September 6, 2024
Read time: 1 min. 18 sec. Today I want to tell you about something I've been working on. And to be honest… it might be the best thing I've ever built. Warning: Do not read this unless you'
Issue #125: Building $1K-$10K MRR Micro SaaS Products around Client Onboarding, AI Agent for online shopping, AI-b…
Friday, September 6, 2024
Build Profitable SaaS products!! ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏
How to write a blog post (according to the Buffer Content team)
Friday, September 6, 2024
Plus tips, news & Buffer updates for your social media journey ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏