One Week, 7 Major Foundation Model Releases
Was this email forwarded to you? Sign up here One Week, 7 Major Foundation Model ReleasesApple, HuggingFace, OpenAI, Mistral, Groq all released innovative models in the same week.Next Week in The Sequence:
You can subscribe to The Sequence below:📝 Editorial: What a Week for Foundation ModelsBuilding high-quality, large-scale foundation models is hard. Just a year ago, it seemed that the foundation model space was going to be highly fragmented, with new models coming to market literally every week. After the high computational and capital realities became obvious, the space seems to have consolidated into a dozen or so relevant models per modality, with a few more in the language space. At the moment, two trends seem to be emerging to catalyze the next generation of foundation models:
Last week was exceptional in terms of model releases in these areas. Just to list a few:
As you can see, the releases emphasize the domain specialization and small model trends. Even by the crazy standards of the generative AI market, last week was a remarkable week in terms of model releases. 📽 [Virtual Talk] Supercharge Production AI with Features as CodeOn July 24, at 9 AM | 12PM ET, join us to discuss how declarative frameworks are transforming production AI. Sergio Ferragut, Principal Developer Advocate at Tecton, will show how to enhance collaboration, automate feature materialization, and support diverse data types. Discover how to improve feature reusability, eliminate training-serving skew, and simplify complex feature development. He will also cover how these frameworks automate production-ready pipelines, speeding up AI projects and making AI-powered applications more intelligent. Key topics include:
🔎 ML ResearchWinning the AI Math OlympiadThe teams from Numina and HuggingFace published a detailed blog post about NuminaMath 7B TIR, the model that achieved the first prize in the AI Math Olypimpiad. NuminaMath 7B TIR is based on a combination of an LLM reasoning agent and code generation and the architecture is totally fascinating —> Read more. Proven-Verifier Games in LLMsOpenAI published a paper unveiling a prover-verifier game to improve the legibility of LLM outputs. The core idea is to train large models in producing outputs that can be verified by weaker models —> Read more. LLMs for SpreadsheetsMicrosoft Research published a paper detailing SPREADSHEETLLM, an encoding method for manipulating spreadsheets with LLMs. SPREADSHEETLLM includes a multi-step encoding framework that include capabilities such as tructural-anchor-based compression, inverse index translation, and data-format-aware aggregation —> Read more. Gen AI for DatabasesResearchers from MIT, CMU and other AI labs published a paper detailing GenSQL, a generative AI system for databases. GenSQL extends SQL with several probabilistic primitives that automate tasks such as predictions, anomaly detection, guess missing values, fix errors, or synthetic data generation —> Read more. Qwen2Alibaba published a research paper diving into Qwen2, a series of languave and multimodal models ranging from 500M to 72B parameters. The Qwen2 family includes different architectures including dense and MoE models and shows strong performance across different benchmarks —> Read more. Long Video UnderstandingResearchers from King Abdullah University of Science and Technology and Harvard University published a paper introducing Godlfish, a method for long form video understanding. Goldfish takes an instruction as input and then gathers the top-k more important video clips relative to that instruction and uses those to generate a response —> Read more. 🤖 Cool AI Tech ReleasesGPT-4o MiniOpen AI released a smaller and most cost efficient version of GPT-4o —> Read more. Apple DCLMApple open sourced a new series of small models that seem to outperform some of the best open source alternatives in the market —> Read more. Llama-3-Groq-Tool-UseGroq open sourced Llama-3-Groq-Tool-Use, a series of models optimized for function calling —> Read more. MathstralMistral released Mathstral, a model specialized in math and scientific discovery. Codestral MambaMistral also released Codestral Mamba, an SSM based model for code generation. Mistral NeMoNVIDIA and Mistral collaborated in the release of Mistral NeMo, a 12B parameter LLM optimized for enterprise scenarios —> Read more. SmolLMHuggingFace open sourced SmolLM, a series of small, high performance LLMs —> Read more. Cohere ToolkitCohere open sourced new additions such as HTML UI generation or authentication to its Toolkit framework —> Read more. 🛠 Real World AIMoving ML Fast at MetaMeta engineering shares some of the best practices for iterating fast in ML engineering —> Read more. Text to Image at PinterestPinterest discusses some details about Canvas, its text-to-image model —> Read more. 📡AI Radar
You’re on the free list for TheSequence Scope and TheSequence Chat. For the full experience, become a paying subscriber to TheSequence Edge. Trusted by thousands of subscribers from the leading AI labs and universities. |
Older messages
📽 [Virtual Talk] Supercharge Production AI with Features as Code
Friday, July 19, 2024
Data is essential for AI/ML systems but often becomes a development bottleneck. Data scientists and engineers face challenges in building and maintaining feature pipelines, ensuring data consistency
Edge 414: Inside Meta AI's HUSKY: A New Agent Optimized for Multi-Step Reasoning
Thursday, July 18, 2024
New research from Meta AI, Allen AI, and the University of Washington tackles one of the most important problems in LLM reasoning. ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏
Edge 413: Autonomous Agents and Semantic Memory
Tuesday, July 16, 2024
Can agents capture memory that encodes actual knowledge? ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏
📽 [Virtual Talk] Building a Resilient, Real-Time Fraud System at Block
Monday, July 15, 2024
Data is crucial for AI/ML systems but often becomes a bottleneck in development. Data scientists and engineers grapple with the complexity of building and maintaining feature pipelines, ensuring
The Most Important Algorithm for Transformers
Sunday, July 14, 2024
FlashAttention has a new version. Plus some important research milestones and major funding activity in AI. ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏
You Might Also Like
Charted | How U.S. Household Incomes Have Changed (1967-2023) 💰
Friday, December 27, 2024
When looking at inflation adjusted data, US households have definitely gotten a whole lot richer since 1967. View Online | Subscribe | Download Our App FEATURED STORY How US Household Incomes Have
Can Pirates Save Democracy?
Friday, December 27, 2024
Top Tech Content sent at Noon! Boost Your Article on HackerNoon for $159.99! Read this email in your browser How are you, @newsletterest1? 🪐 What's happening in tech today, December 27, 2024? The
The 2025 Predictions You Can't Afford to Miss 🔮
Friday, December 27, 2024
Get a head start on what's to come in the New Year. Join VC+ to gain access to our 2025 Global Forecast Series and other exclusive insights! View email in browser HOW LEADERS STAY AHEAD IN 2025 The
DeveloPassion's Newsletter #182 - 2024 Retrospective
Friday, December 27, 2024
A newsletter discussing Knowledge Management, Knowledge Work, Zen Productivity, Personal Organization, and more! Sébastien Dubois DeveloPassion's Newsletter DeveloPassion's Newsletter #182 -
End 2024 on a High Note: The Top Writing Tips and Templates You Need
Friday, December 27, 2024
What's good, @newsletterest1! As we welcome 2025, let's take a moment to celebrate the incredible stories that fueled our hacker minds in 2024! We've compiled a roundup of the most-used
Private AI data + AI in Hollywood
Friday, December 27, 2024
my 2024 favorites ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏
🐧 The best Linux distro of 2024
Friday, December 27, 2024
Extension cord don'ts; AI's biggest challenge; Wired network hack -- ZDNET ZDNET Tech Today - US December 27, 2024 The default elementary OS 8 desktop. The best Linux distribution of 2024 is
Issue #573: Ray browser, focus shift, and Nimrods
Friday, December 27, 2024
View this email in your browser Issue #573 - December 27th 2024 Weekly newsletter about Web Game Development. If you have anything you want to share with our community please let me know by replying to
Palo Alto Releases Patch for PAN-OS DoS Flaw — Update Immediately
Friday, December 27, 2024
THN Daily Updates Newsletter cover Backups: The Key to Cybersecurity How Much Cybersecurity is Enough? Recovery + Resistance = Resilience Download Now Sponsored LATEST NEWS Dec 27, 2024 Cloud Atlas
SWLW #631: You can’t measure productivity, Ask uncommonly clear questions, and more.
Friday, December 27, 2024
Weekly articles & videos about people, culture and leadership: everything you need to design the org that makes the product. A weekly newsletter by Oren Ellenbogen with the best content I found