The Sequence Chat: The End of Data. Or Maybe Not
Was this email forwarded to you? Sign up here The Sequence Chat: The End of Data. Or Maybe NotOne of the most passionate arguments in generative AI.
Are we running out of data? This is a contentious debate in the world of generative AI, with passionate supporters and detractors on both sides. Most large foundation models have been virtually trained on the entirety of the internet, using datasets like Fine-Web that encapsulate much of the publicly available data. As a result, the question of AI hitting a 'data wall' has become increasingly relevant. After all, the famous scaling laws are fundamentally dependent on the availability of vast amounts of data. Without these scaling laws, one could argue that the entire foundation model ecosystem risks plateauing. This essay explores the thesis of the "end of data" for AI models, examining both sides of the argument and delving into potential solutions such as extracting higher quality data and generating synthetic datasets. Let’s start with some points that validate the data wall argument: The Data Scarcity Argument...Subscribe to TheSequence to unlock the rest.Become a paying subscriber of TheSequence to get access to this post and other subscriber-only content. A subscription gets you:
|
Older messages
Edge 449: Getting Into Adversarial Distillation
Tuesday, November 19, 2024
A way to distill models using inspiration from GANs. ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏
The Toughest Math Benchmark Ever Built
Sunday, November 17, 2024
Frontier Math approach math reasoning in LLMs from a different perspective. ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏
📽 Webinar: How Convirza Scaled SLMs for Real-Time Call Analytics – Without Breaking the Bank
Friday, November 15, 2024
Companies that rely on analyzing high volumes of data face a core dilemma: how to deliver real-time insights without burning through budget or engineering resources. ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏
The Sequence Chat: Small Specialists vs. Large Generalist Models and What if NVIDIA Becomes Sun Microsystems
Friday, November 15, 2024
A controversial debate and a crazy thesis. ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏
Edge 448: Meta AI's Technique For Building LLMs that "Think Before they Speak"
Friday, November 15, 2024
Thought Preference Optimization can set the baseline for building reasoning LLMs. ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏
You Might Also Like
Coca-Cola's AI ad ruined Christmas 🎄
Wednesday, November 20, 2024
plus, Apple's AI wall tablet ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏
How to turn Android into a desktop OS
Wednesday, November 20, 2024
The Star Trek farewell we needed; AI transformation; Free AI training -- ZDNET ZDNET Tech Today - US November 20, 2024 ailaptopbluegettyimages-1311072879 How Google turns Android into a desktop OS in 5
Tomorrow: How to find and fix security gaps in Okta, M365 & GSuite
Wednesday, November 20, 2024
Learn how to implement effective security posture management for your IdP infrastructure.
Your habits and routines make you
Wednesday, November 20, 2024
Introduction We humans are creatures of habits. We tend to keep doing the same things, following the same patterns. In a way, our habits and routines are part of us, but they also define who we are.
Warning: Apple Issues Urgent Patches for Actively Exploited Zero-Day Vulnerabilities
Wednesday, November 20, 2024
THN Daily Updates Newsletter cover Python Feature Engineering Cookbook, Third Edition ($35.99 Value) FREE for a Limited Time A complete guide to crafting powerful features for your machine learning
Post from Syncfusion Blogs on 11/20/2024
Wednesday, November 20, 2024
New blogs from Syncfusion Prepare for Syncfusion's Biggest Sale of the Year – Black Friday 2024! By Jollen Moyani Discover our most anticipated sale of the year! Don't miss Syncfusion Essential
⚙️ New computing paradigm
Wednesday, November 20, 2024
Plus: Algorithmic bias and mass surveillance in Denmark
The Floater Manifesto 🛟
Wednesday, November 20, 2024
Some people use one operating system. I use many. Here's a version for your browser. Hunting for the end of the long tail • November 19, 2024 The Floater Manifesto Our technology should be good
Boost Elastic Skills & Break Limits in 2025!
Wednesday, November 20, 2024
Transforming Elastic Customer Support with GenAI ㅤㅤㅤㅤㅤㅤㅤㅤㅤㅤㅤㅤㅤㅤㅤㅤㅤㅤㅤㅤㅤㅤㅤㅤㅤㅤ ㅤㅤㅤㅤㅤㅤㅤㅤㅤㅤㅤㅤㅤㅤㅤㅤㅤㅤㅤㅤㅤㅤㅤㅤㅤㅤ ㅤㅤㅤㅤㅤㅤㅤㅤㅤㅤㅤㅤㅤㅤㅤㅤㅤㅤㅤㅤㅤㅤㅤㅤㅤㅤ elastic | Search. Observe. Protect Get your annual pass Edu-Email-