S. Somasegar on the Present and Future of Generative AI
Was this email forwarded to you? Sign up here S. Somasegar on the Present and Future of Generative AIFormer Sr.Vice President of Microsoft's Developer Division and now a managing partner at Madrona Ventures, S. Somasegar has been one of the most active VCs evangelizing the potential of generative AI.👤 Quick bio
Spent nearly 27 years at Microsoft working mostly on platforms and my last role at Microsoft was running the Developer Division. To the extent a developers needs anything to be able to build applications and services on top of the Microsoft platform, my team was responsible for that. Products like Visual Studio, Visual Studio Code, .NET, Visual Studio Team Services and programming languages like VB, C#, TypeScript, F#, etc. were some of what my team delivered. While at Microsoft, I was also responsible for a lot of the Global Development Centers that we had around the world including R&D centers in India, China and Israel. A little over 7 years ago, I left Microsoft and came to Madrona Ventures, the leading Venture Capital firm in Seattle. I am a Managing Director at Madrona Ventures and am mostly focused on investments in enterprise and B2B focused start-ups including next generation cloud infrastructure, ML/AI, Intelligent and Generative Applications and SaaS. Some of my investments while at Madrona include Snowflake, UiPath, Pulumi, StatSig, SeekOut, Go1, Clari, Coda, CommonRoom and Temporal. 🛠 ML Work
Madrona believes that all applications will be intelligent. Intelligent Applications leverage ML models that use both historical and real-time data to build continuous learning systems and deliver insights that can help users make better decisions. We believe that Generative AI is a subsegment of the broader intelligent applications theme and we believe there will be both generative native apps (e.g., apps that natively leverage FMs) and generative enhanced apps (e.g., apps that currently exists and will embed generative features). Today, most Generative AI is used in high affordability use cases (where the generated content doesn’t have to be 100% correct) and across modalities such as text, image, or code generation; however, we believe new multi-modal applications of Generative AI will emerge across new modalities and as models get more accurate, they will be deployed in lower affordability use cases. We also are big believers in Generative AI being immensely helpful in enterprise use cases as well as consumer use cases.
Developer tools, such as GitHub Copilot, can turn natural language prompts into coding suggestions. This makes software engineers more productive and allows them to focus on more important aspects of coding. We're also seeing other generative code companies that assist with not only code creation, but also unit testing, flagging suspicious behavior in the code base, and fixing other code issues. These type of developer tools will become re-imagined through Foundation Models.
There is an interesting battle between the close-source API-based and open source distribution channels for foundation models. In some ways, it resembles the IOS-vs-Android race. How are you seeing the friction and balance between these two distribution models?
One interesting new capability we are seeing is the idea of “agents”. Agents are powerful units that can be used to determine which actions to take and in what order. Agents can interpret natural language (capable of reasoning an input), connect to external knowledge bases (capable of connecting to your databases), and then leverage APIs to take action (sends an email or imports/loads a file).
The pace of activity continues to be extraordinary as we are seeing order of magnitude shifts every 6 months. While techniques like fine-tuning and few shot learning are improving performance of today's models, it is unclear how necessary they will be in tomorrow's models. We believe that all the excitement around generative models in academia and by practitioners will continue to yield new techniques for getting the most out of language models. As mentioned earlier, one trend we are particularly excited about is a trend of building "agents" on top of language models that are capable of working with external data sources and performing actions. We are still in the early days of leveraging LLMs to perform actions, but when they can we believe there will be an exciting new wave of applications built. For example a next generation travel application would use an LLM to understand the query "book me a flight to Seattle", have an up to date database of the most recent flight information, understand the parameters around date of travel, class of travel, cost basis and have the capability to book an appropriate flight. 💥 Miscellaneous – a set of rapid-fire questions
Well, Microsoft is a trillion dollar company on its way to become a generative AI company. Does that count ;) ? But yes, I believe OpenAI has the potential to be a trillion dollar Generative-AI company.
Computational resources. Training generative models can be expensive and there is currently a lack of GPUs. Legal/ethical/compliance concerns are another important set of things to continue working on as they could be another big roadblock for mainstream adoption.
I think the top 10 technology companies are all making huge strides in Generative AI – some, like Microsoft, ahead of the pack. I do think that companies like Google and Amazon are working hard on generative AI but unclear as they haven’t been too vocal yet about what they are doing. I think the rest of this year is going to be very interesting in that we will hear more from the top technology companies about what they are all doing and that will shed a little more clarity on who might be falling behind.
The principle behind Turing Test is still relevant. The Turing test at its core is a measurement for how good an autonomous agent is relative to a human at performing a task. We are seeing LLMs being applied to a variety of tasks and based on the evaluation mechanism that developers use, it is good to benchmark human performance compared to model performance. For example, how does an AI generated summary compare to a human generated summary? What is exciting is that we are actually beginning to see AI perform just as well, if not better than humans at tasks like summarization and question answering! However, there are many tasks that humans still outperform AI like programming. For these tasks, we like copilot type experiences where AI is assistive. Perhaps an alternative measurement will be to what degree an AI is assistive vs automates an entire task. You’re on the free list for TheSequence Scope and TheSequence Chat. For the full experience, become a paying subscriber to TheSequence Edge. Trusted by thousands of subscribers from the leading AI labs and universities. |
Key phrases
Older messages
Sparrow Might be the Foundation of DeepMind's ChatGPT Competitor
Thursday, March 16, 2023
Sparrow uses a combination of large language models and reinforcement learning to enable a safer conversational experience.
Edge 273: Horizontal Federated Learning
Tuesday, March 14, 2023
Horizontal federated learning, Google's research on personalized federated learning Syft framework.
What a Week for Generative AI
Sunday, March 12, 2023
Sundays, The Sequence Scope brings a summary of the most important research papers, technology releases and VC funding deals in the artificial intelligence space.
Edge 272: Inside Toolformer, Meta AI New Transformer Learned to Use Tools to Produce Better Answers
Thursday, March 9, 2023
The model mastered the use of tools such as calculators, calendars, or Wikipedia search queries across many downstream tasks.
A Taxonomy to Understand Federated Learning
Tuesday, March 7, 2023
Classifying different types of federated learning methods, Meta AI research about highly scalable and asynchronous federated learning pipelines and Microsoft's FLUTE framework.
You Might Also Like
Charted | Economic Growth Forecasts for G7 and BRICS Countries in 2024 📊
Thursday, April 25, 2024
The IMF has released its economic growth forecasts for 2024. How do the G7 and BRICS countries compare in expected real GDP growth? View Online | Subscribe Presented by: Access European benchmarks with
Build5Nines Newsletter - April 25, 2024
Thursday, April 25, 2024
View this email in your browser Build5Nines Build5Nines Newsletter Thank you for subscribing! I look forward to sharing with you the latest cloud news, technical help, and other thoughts around DevOps
Discover the World's Easiest Parallel File System
Thursday, April 25, 2024
Join us in exploring the future of data management with Bjorn Kolbeck, a Google engineer turned CEO and Co-founder of Quobyte, the creators of the world's easiest parallel file system. ͏ ͏ ͏ ͏ ͏ ͏
Issue 314 - New Model 3 Performance is here
Thursday, April 25, 2024
View this email in your browser If you are just now finding out about Tesletter, you can subscribe here! If you already know Tesletter and want to support us, check out our Patreon page Issue 314 - New
Programmer Weekly - Issue 202
Thursday, April 25, 2024
View this email in your browser Programmer Weekly Welcome to issue 202 of Programmer Weekly. Let's get straight to the links this week. Quote of the Week "Computer science inverts the normal.
Python Weekly - Issue 647
Thursday, April 25, 2024
View this email in your browser Python Weekly Welcome to issue 647 of Python Weekly. Let's get straight to the links this week. From Our Sponsor Get Your Weekly Dose of Programming A weekly
Web Tools #562 - Voilà Review, CSS Tools, Media, React Native
Thursday, April 25, 2024
WEB VERSION Issue #562 • April 25, 2024 The following is a paid product review for Voilà, an AI assistant for the browser that enables you to improve your writing, coding, brainstorming, and research
Everyone wants to build the AI dev tool of the future
Thursday, April 25, 2024
A new startup called Augment has raised north of $250 million to build AI-powered dev tools. View this email online in your browser By Alex Wilhelm Thursday, April 25, 2024 Welcome to TechCrunch AM!
7 reasons to use Copilot over ChatGPT
Thursday, April 25, 2024
Coros Vertex 2S; Top 5 news apps; New Yeedi M12 Pro+ -- ZDNET ZDNET Tech Today - US April 25, 2024 placeholder 7 reasons I use Copilot instead of ChatGPT I reach for Copilot every day, and here's
Why they signed up for my Private AI Mentorship
Thursday, April 25, 2024
There are 3 reasons: use cases, accountability, and time.