Big vs. Small, Open Source vs. API Based, the Philosophical Frictions of Foundation Models
Was this email forwarded to you? Sign up here Big vs. Small, Open Source vs. API Based, the Philosophical Frictions of Foundation ModelsSundays, The Sequence Scope brings a summary of the most important research papers, technology releases and VC funding deals in the artificial intelligence space.Next Week in The Sequence
📝 Editorial: Big vs. Small, Open Source vs. API Based, the Philosophical Frictions of Foundation ModelsInnovation is the foundation of the models space, which is accelerating at a frantic pace, and we are seeing new models popping up everywhere. While the space is still in a very nascent state, we are already seeing conflicting forces that will be highly influential in the evolution of the market. Currently, two major frictions are influencing different philosophical divisions in the foundation model space:
Regarding foundation models, the rule that larger is better has proven true for the last few years. Larger models simply exhibited cognitive capabilities that were not possible with smaller architectures. However, in recent months, we have seen the emergence of models like LLaMA and variations with RLHF that have been able to come close to matching the performance of larger alternatives. The second friction is the generative AI version of the iOS vs. Android debate. Models like GPT-4, LaMDA, and Claude are distributed via commercial APIs, while models like Dolly 2 and Stable Diffusion are distributed via open-source models. The rationale of this debate goes beyond the commercial model and encompasses aspects such as fairness and safety concerns. The most surprising dynamic of the two frictions at the center of the evolution of foundation models is that they are not defining four camps, but rather two. In a not surprising coincidence, the vendors favoring super large models also rely on API-based distribution, while the open-source models are also relatively smaller. On one camp, we have OpenAI, Anthropic, Microsoft, or Google, while in the other, we can currently see Databricks, Stability AI, and maybe Meta. Are these two market frictions the same? I personally don’t think so. In the near future, we are likely to see open-source distributions of super large models or smaller models only available via APIs. But also, let’s remember that generative AI is different from any other market. 📣 Re-tooling around LLMs?Share your POV! Take this one-minute survey and be entered for a chance to win a swag pack or O’Reilly books! Take the flash poll.* 🔎 ML ResearchHuggingGPTMicrosoft Research published a paper detailing HuggingGPT, a framework that uses language models to connect various foundation models for diverse AI tasks. HuggingGPT uses ChatGPT to determine the tasks to execute in ML models for specific tasks —> Read more. Text-Guided Video GenerationGoogle Research published a paper proposing UniPi, a model that can learn a diverse, universal policy in text t- video models. UniPi can be seen as a universal interface for inferring actions in videos based on text descriptions —> Read more. Animated DrawingsMeta AI Research published a paper and open source a dataset to streamline the animation of amateur drawings. The dataset includes 180,000 animated pictures and a demo that could encourage researchers to innovate in this area —> Read more. 🤖 Cool AI Tech ReleasesCache LLM Queries with GPTCacheGPTCache, an MIT-licensed open-source semantic cache, is now available to reduce your ChatGPT bill and improve the performance of your LLM app —> Read more. Amazon BedrockAmazon finally unveiled its play in the generative AI space with the release of Bedrock, a platform that enables interaction with several foundation models —> Read more. Auto-GPTA super interesting open source experiment that attempts to make GPT4 more autonomous —> Read more. StackLLaMAHugging Face open sourced StackLLaMA, a model based on Meta AI’s LLaMA and fined tuned on Stack Exchange questions —> Read more. Dolly 2.0Databricks open sourced the second version of its Dolly model, a 12B instruction following model which is available for commercial use —> Read more. Stable Diffusion SDXL Stability AI released Stable Diffusion XL Beta as part of its DreamStudio, a new model with enhanced capabilities including the ability to generate legible text in images —> Read more. DeepSpeed ChatMicrosoft Research open sourced DeepSpeed Chat, a framework for large scale RLHF training of LLMs —> Read more. 🛠 Real World MLResponsible AI at LinkedInLinkedIn discusses some of the best practices for ensuring responsible AI usage in their applications —> Read more. Generative AI for Compliance at GitHubThe GutHub engineering team presents some ideas about generative AI can be used for software development compliance tasks —> Read more. ML Computing Costs at LyftLyft’s engineering team outlines some of the best practices used to save compute costs in their ML and big data workloads —> Read more. 📡AI Radar
You’re on the free list for TheSequence Scope and TheSequence Chat. For the full experience, become a paying subscriber to TheSequence Edge. Trusted by thousands of subscribers from the leading AI labs and universities. |
Key phrases
Older messages
📝 Guest Post: How to Enhance the Usefulness of Large Language Models*
Wednesday, April 19, 2023
In this guest post, Filip Haltmayer, a Software Engineer at Zilliz, explains how LangChain and Milvus can enhance the usefulness of Large Language Models (LLMs) by allowing for the storage and
Edge 283: Federated Learning and Differential Privacy
Wednesday, April 19, 2023
Applying deferential privacy to federated learning(FL) scenarios, Meta AI's research and the best open source frameworks in this area.
Edge 281: Cross-Device Federated Learning
Tuesday, April 11, 2023
Cross device federated learning(FL), Google's work on FL with differential privacy and the FedLab framework
📝 Guest Post: Caching LLM Queries for Improved Performance and Cost Savings*
Monday, April 10, 2023
If you're looking for a way to improve the performance of your large language model (LLM) application while reducing costs, consider utilizing a semantic cache to store LLM responses. By caching
The LLama Effect: How an Accidental Leak Sparked a Series of Impressive Open Source Alternatives to ChatGPT
Sunday, April 9, 2023
Sundays, The Sequence Scope brings a summary of the most important research papers, technology releases and VC funding deals in the artificial intelligence space.
You Might Also Like
SBF gets 25 years
Thursday, March 28, 2024
Sam Bankman-Fried is sentenced View this email online in your browser By Christine Hall Thursday, March 28, 2024 Welcome back to TechCrunch PM! The editorial team spent a chunk of the day discussing
💎 Issue 410 - Being laid off in 2023-2024 as an early-career developer
Thursday, March 28, 2024
This week's Awesome Ruby Newsletter Read this email on the Web The Awesome Ruby Newsletter Issue » 410 Release Date Mar 28, 2024 Your weekly report of the most popular Ruby news, articles and
💻 Issue 403 - Microsoft defends .NET 9 features competing with open source ecosystem
Thursday, March 28, 2024
This week's Awesome .NET Weekly Read this email on the Web The Awesome .NET Weekly Issue » 403 Release Date Mar 28, 2024 Your weekly report of the most popular .NET news, articles and projects
💻 Issue 410 - Node.js TSC Confirms: No Intention to Remove npm from Distribution
Thursday, March 28, 2024
This week's Awesome Node.js Weekly Read this email on the Web The Awesome Node.js Weekly Issue » 410 Release Date Mar 28, 2024 Your weekly report of the most popular Node.js news, articles and
💻 Issue 410 - JSDoc as an alternative TypeScript syntax
Thursday, March 28, 2024
This week's Awesome JavaScript Weekly Read this email on the Web The Awesome JavaScript Weekly Issue » 410 Release Date Mar 28, 2024 Your weekly report of the most popular JavaScript news, articles
📱 Issue 404 - Dependency Injection for Modern Swift Applications Part II
Thursday, March 28, 2024
This week's Awesome iOS Weekly Read this email on the Web The Awesome iOS Weekly Issue » 404 Release Date Mar 28, 2024 Your weekly report of the most popular iOS news, articles and projects Popular
💻 Issue 328 - My new open-source repository to schedule all your content!
Thursday, March 28, 2024
This week's Awesome React Weekly Read this email on the Web The Awesome React Weekly Issue » 328 Release Date Mar 28, 2024 Your weekly report of the most popular React news, articles and projects
📱 Issue 407 - Apple just announced WWDC24. The keynote for WWDC24 will be held on Monday, June 10th.
Thursday, March 28, 2024
This week's Awesome Swift Weekly Read this email on the Web The Awesome Swift Weekly Issue » 407 Release Date Mar 28, 2024 Your weekly report of the most popular Swift news, articles and projects
💻 Issue 405 - 2024 Edition Update
Thursday, March 28, 2024
This week's Awesome Rust Weekly Read this email on the Web The Awesome Rust Weekly Issue » 405 Release Date Mar 28, 2024 Your weekly report of the most popular Rust news, articles and projects
🤖 What to Expect From Google I/O 2024 — How to Stop Apps From Leaking Your Data
Thursday, March 28, 2024
Also: The Best Camera Straps of 2024, and More! How-To Geek Logo March 28, 2024 📩 Get expert reviews, the hottest deals, how-to's, breaking news, and more delivered directly to your inbox by