3 vs. 3: The Open vs. Closed Battle for Big AI
Was this email forwarded to you? Sign up here Next Week in The Sequence:
You can subscribe to The Sequence below:📝 Editorial: 3 vs. 3: The Open vs. Closed Battle for Big AIWhen the open vs. closed weight model debate started a couple of years ago, many thought it was going to be a battle between OpenAI, Anthropic, and Google on one side, and hundreds of open-source models on the other. Reality turned out to be quite different. The open-source space for massively large foundation models has been reduced to three key players: Meta, Mistral, and xAI. This shouldn’t come as a surprise if we consider that training a multi-hundred-parameter model surpasses $100 million in training costs. Open sourcing that kind of investment is something only a few companies can afford. So GPT-x, Claude, and Gemini versus Llama, Mistral, and Grok. How will this shape up? When the first versions of these open-weight models came out, they were a couple of iterations behind the quality of the large commercial models. That’s no longer the case, and this week was a good reminder of how competitive the big open-source models can be.
Big AI is a game for big budgets, and there might still be room for a few more competitors in this race (maybe Illya’s new company). However, the space is not going to change drastically. It’s OpenAI, Google, and Anthropic vs. Meta, Llama, and xAI. One thing is for certain: open-source big AI is going to be competitive. 🔎 ML ResearchAlphaProof and AlphaGeometry 2Google DeepMind published details about AlphaProof and AlphaGeometry 2, two systems that combines to achieve silver medalist status in this year’s International Mathematical Olympiad (IMO). AlphaProof is a reinforcement learning model for math reasoning while AlphaGeometry uses a neurosymbolic architecture that combines LLMs and symbolic models —> Read more. The Llama 3 Herd of ModelsMeta AI published paper detailing the architecture and processes for building the Llama 3 family of models. The paper also introduces a compositional approach to integrates image, video and speech recognition capabilities into Llama 3 —> Read more. OpenDevinResearchers from elite AI universities such as UC Berkeley, Yale, Carnegie Mellon and others published a paper introducing OpenDevin, a framework for developing AI agents that interact with environments similar to human programmers. OpenDevin agents are able to collaborate with human programmers in different tasks such as bug fixing, feature building, testing and many others —> Read more. Model CollapseResearchers from Oxford, Cambridge, Imperial Collegue of London and other institutions published a paper in Nature outlining a curious phenomenon in LLMs coined as model collapse. The thesis of model collapse states that LLMs will start showing irreversible degenerative behavior when trained in data created by other AI models —> Read more. Visual Haystacks BenchmarkBerkeley AI Research(BAIR) published a paper introducing the Visual Haystacks Benchmark(VHS) for multi-image reasoning. VHS evalautes retrieval and reasoning capabilities across large collections of uncorrelated images —> Read more. Pruning and Distillation in LLMsNVIDIA Research published a paper proposing a set of effective compression best practices to build compact LLMs. The techiques combine the best strategies for depth, width, attention and MLP pruning with knowledge distillation-based retraining —> Read more. SlowFast-LLaVAApple Research published a paper detailing SlowFast-LLaVA(SF-LLaVA), a video language model optimized for capturing the spatial semantics and temporal context in videos. SF-LLaVA uses a two-stream input design to aggregate features from different video frames in ways that facilitate knowledge extraction —> Read more. 🤖 AI Tech ReleasesLlama 3.1Meta open sourced Llama 3.1 including its 405B parameter model as well as complementary tools and applications —> Read more. Mistral LargeMistral unveiled Mistral Large, a 123B parameter model that rivals Llama 3.1 —> Read more. SearchGPTOpenAI unveiled a preview of a new AI-first search engine —> Read more. NVIDIA AI FoundryNVIDIA announced the availability of its AI Foundry to enable the creation of custom models for enterprises —> Read more. Phi-3 Serverless Fine-TuningMicrosoft unveiled new AI features in the Azure platform including a serverless infrastructure to fine-tune Phi-3 models —> Read more. Stable Video 4DStability AI announced the release of Stable Video 4D, its latest video generation model —> Read more. 🛠 Real World AIOrchestration at NetflixNetflix open sourced Maestro, its engine for orchestration of data and ML pipelines —> Read more. Product Categorization at WalmartWalmart Global Tech discussed some of their work behind Ghotok, their predictive generative AI engine used for product categorization —> Read more. 📡AI Radar
You’re on the free list for TheSequence Scope and TheSequence Chat. For the full experience, become a paying subscriber to TheSequence Edge. Trusted by thousands of subscribers from the leading AI labs and universities. |
Older messages
Edge 416: Inside Apple's 4M-21 Model that Could be the Foundation of its On-Device Multimodal Experience
Thursday, July 25, 2024
The model was trained simultaneously across 21 different modalities. ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏
Edge 415: Agents that Remember Actions with Procedural Memory
Tuesday, July 23, 2024
One of the most unique forms of memory in autonomous agents. ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏
📝 Guest Post: Local Agentic RAG with LangGraph and Llama 3*
Monday, July 22, 2024
In this guest post, Stephen Batifol from Zilliz discusses how to build agents capable of tool-calling using LangGraph with Llama 3 and Milvus. Let's dive in. LLM agents use planning, memory, and
One Week, 7 Major Foundation Model Releases
Sunday, July 21, 2024
Apple, HuggingFace, OpenAI, Mistral, Groq all released innovative models in the same week. ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏
📽 [Virtual Talk] Supercharge Production AI with Features as Code
Friday, July 19, 2024
Data is essential for AI/ML systems but often becomes a development bottleneck. Data scientists and engineers face challenges in building and maintaining feature pipelines, ensuring data consistency
You Might Also Like
💻 Issue 428 - C# different way to do a proof of concept
Thursday, September 19, 2024
This week's Awesome .NET Weekly Read this email on the Web The Awesome .NET Weekly Issue » 428 Release Date Sep 19, 2024 Your weekly report of the most popular .NET news, articles and projects
💎 Issue 435 - Ruby-SAML pwned by XML signature wrapping attacks
Thursday, September 19, 2024
This week's Awesome Ruby Newsletter Read this email on the Web The Awesome Ruby Newsletter Issue » 435 Release Date Sep 19, 2024 Your weekly report of the most popular Ruby news, articles and
💻 Issue 435 - Oracle, it's time to free JavaScript
Thursday, September 19, 2024
This week's Awesome JavaScript Weekly Read this email on the Web The Awesome JavaScript Weekly Issue » 435 Release Date Sep 19, 2024 Your weekly report of the most popular JavaScript news, articles
📱 Issue 429 - iOS 18 breaks IMAPS self-signed certs
Thursday, September 19, 2024
This week's Awesome iOS Weekly Read this email on the Web The Awesome iOS Weekly Issue » 429 Release Date Sep 19, 2024 Your weekly report of the most popular iOS news, articles and projects Popular
💻 Issue 353 - Why React Won the Front-End Race
Thursday, September 19, 2024
This week's Awesome React Weekly Read this email on the Web The Awesome React Weekly Issue » 353 Release Date Sep 19, 2024 Your weekly report of the most popular React news, articles and projects
💻 Issue 435 - DevSecOps Project: "Secure Full-Stack Node.js Web Application Deployment with Jenkins, Docker, Kubernetes, and HashiCorp Vault"
Thursday, September 19, 2024
This week's Awesome Node.js Weekly Read this email on the Web The Awesome Node.js Weekly Issue » 435 Release Date Sep 19, 2024 Your weekly report of the most popular Node.js news, articles and
📱 Issue 432 - Swift 6
Thursday, September 19, 2024
This week's Awesome Swift Weekly Read this email on the Web The Awesome Swift Weekly Issue » 432 Release Date Sep 19, 2024 Your weekly report of the most popular Swift news, articles and projects
💻 Issue 430 - Days since last Minecraft server written in Rust was released
Thursday, September 19, 2024
This week's Awesome Rust Weekly Read this email on the Web The Awesome Rust Weekly Issue » 430 Release Date Sep 19, 2024 Your weekly report of the most popular Rust news, articles and projects
Ranked | The Largest Producers of Wind Power, by Country ⚡
Thursday, September 19, 2024
Global wind power capacity hit fresh records in 2023 thanks to strategic government investment and lower technology costs. View Online | Subscribe | Download Our App Presented by: NEW REPORT: Brought
🧠 ChatGPT Passed the Turing Test — 5 Tips to Make Your Laptop Last Longer
Thursday, September 19, 2024
Also: How to Sideload Apps on Android TV, and More! How-To Geek Logo September 19, 2024 Did You Know Babies seem to have such large eyes because humans are born with eyes approximately 75 percent of