Edge 390: Diving Into Databricks' DBRX: One of the Most Impressive Open Source LLMs Released Recently
Was this email forwarded to you? Sign up here Edge 390: Diving Into Databricks' DBRX: One of the Most Impressive Open Source LLMs Released RecentlyThe model uses an MoE architecture which exhibits remarkable perfromance on a relatively small budget.
The open-source generative AI landscape is experiencing tremendous momentum. Innovation comes not only from startups like HuggingFace, Mistral, or AI21 but also from large AI labs such as Meta. Databricks has been one of the tech incumbents exploring different angles in open source generative AI, mainly after the acquisition of MosaicML. A few days ago, Databricks open sourced DBRX, a massive general-purpose LLM that show incredible performance across different benchmarks.
DBRX builds on the mixture-of-experts(MoE) approach used by Mixtral which seems to be more and more the standard to follow in transformer based architecutures. Databricks released both the baseline model DBRX Base as well as the intstruction fine-tuned one DBRX Instruct. From the initial reports, it seems that Databricks’ edge was the quality of the dataset and training process although there are few details in those. Architecture...Subscribe to TheSequence to read the rest.Become a paying subscriber of TheSequence to get access to this post and other subscriber-only content. A subscription gets you:
|
Older messages
Edge 389: Understanding Large Action Models
Tuesday, April 23, 2024
One of the most important concepts in autonomous agents. ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏
Some Cool Details About Llama 3
Sunday, April 21, 2024
Solid performance, new tokenizer, fairly optimal training and other details about Meta AI's new model. ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏
Edge 388: Google DeepMind's SIMA can Follow Language Instructions in 3D Games Just Like Humans
Thursday, April 18, 2024
The AI agent represents a major improvement relative to expensive reinforcement learning methods. ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏
Edge 387: Tool Learning in Autonomous Agents
Tuesday, April 16, 2024
Agents that master tools and APIs, UC Berkeley's Gorilla and Microsoft's TaskWeaver ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏
Neuro-Symbolic Models are Making a Comeback
Sunday, April 14, 2024
A new startup called Symbolica comes out of stealth with a very different value proposition. ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏
You Might Also Like
📈 Why Is My Ping So High While Gaming? — How to Keep Your Android From Overheating
Saturday, May 4, 2024
Also: Using ChatGPT to Craft a Resume, and More! How-To Geek Logo May 4, 2024 📩 Get expert reviews, the hottest deals, how-to's, breaking news, and more delivered directly to your inbox by
JSK Daily for May 4, 2024
Saturday, May 4, 2024
JSK Daily for May 4, 2024 View this email in your browser A community curated daily e-mail of JavaScript news The Power of React's Virtual DOM: A Comprehensive Explanation Modern JavaScript
Daily Coding Problem: Problem #1431 [Medium]
Saturday, May 4, 2024
Daily Coding Problem Good morning! Here's your coding interview problem for today. This problem was asked by MongoDB. Given a list of elements, find the majority element, which appears more than
Ranked | The World's Top Media Franchises by All-Time Revenue 📊
Saturday, May 4, 2024
From Pokémon to Star Wars, some media franchises are globally recognizable. How do media franchises compare in terms of all-time revenue? View Online | Subscribe Presented by Voronoi: The App Where
Noonification: Read Code Like a Hacker With the SAST
Saturday, May 4, 2024
Top Tech Content sent at Noon! Get Algolia: AI Search that understands How are you, @newsletterest1? 🪐 What's happening in tech today, May 4, 2024? The HackerNoon Newsletter brings the HackerNoon
Weekend Reading — May the fourth
Saturday, May 4, 2024
This week we setup our new Minecraft server, play Spacewar, avoid burnout, wonder about Facebook AI spam, lose our passkeys, and claim stairs on the way back home. 😎 Labnotes (by Assaf Arkin) Weekend
Google lays off workers
Saturday, May 4, 2024
Plus: Tesla cans its Supercharger team and UnitedHealthcare reveals security lapses View this email online in your browser By Kyle Wiggers Saturday, May 4, 2024 Image Credits: Tomohiro Ohsumi / Getty
When It Rains, It Pours ☔
Saturday, May 4, 2024
Why the umbrella's design can't be beat. Here's a version for your browser. Hunting for the end of the long tail • May 04, 2024 Hey there, Ernie here with a refreshed piece about umbrellas
🐍 New Python tutorials on Real Python
Saturday, May 4, 2024
Hey there, There's always something going on over at realpython.com as far as Python tutorials go. Here's what you may have missed this past week: Python's unittest: Writing Unit Tests for
Microsoft Outlook Flaw Exploited by Russia's APT28 to Hack Czech, German Entities
Saturday, May 4, 2024
THN Daily Updates Newsletter cover Webinar -- Data Security is Different at the Petabyte Scale Discover the secrets to securing fast-moving, massive data sets with insights from industry titans