TheSequence - Edge 441: SSMs Beyond Language
Was this email forwarded to you? Sign up here
In this issue:
💡 ML Concept of the Day: SSMs Beyond LanguageThroughout this series, we have explored the fundamentals latest research in state space models(SSMs) as one of the main alternatives to transformer architectures. SSMs provide a more efficient scaling mechanism than transformers which makes it ideal for models with large context windows. Given the state of the market, the core focus on SSMs have been in LLMs but, surprisingly, some of the core applications of SSMs are surfacing in other modalities. Take audio, for instance, SSMs have emerged as one 3of the most efficient techniques in this modality given its efficiency processing continuous irregular continuous data. Models like AudioMamba , RawMamba and some of the work done by Cartesia are great examples of SSMs applied to audio... Subscribe to TheSequence to unlock the rest.Become a paying subscriber of TheSequence to get access to this post and other subscriber-only content. A subscription gets you:
|
Older messages
The Sequence Chat: Why Transformers are the Best Thing that Ever Happened to NVIDIA
Monday, October 21, 2024
A discussion about some controvertial and original ideas in AI. ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏
NVIDIA Releases Nemotron 70B
Sunday, October 20, 2024
The new model has been making the headlines due to its impressive performance. ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏
AI Dropped the Mic at the Nobel Party
Sunday, October 20, 2024
Two Nobel Prizes were awarded to AI scientists ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏
Edge 439: SSMs with Attention, Understanding Zamba
Sunday, October 20, 2024
Combining the best of SSMs and transformers in a single architecture. ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏
Edge 440: Interested in AI Evaluation? Meet Microsoft's EUREKA
Sunday, October 20, 2024
The framework provides an evaluation pipeline as well as a collection of benchmarks for evaluating language and vision capabilities. ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏
You Might Also Like
JS0/JSSugar; converting CommonJS to ESM; top-level await in Node.js; Node.js v23.0.0; Bun v1.1.31;
Tuesday, October 22, 2024
We have 8 links for you - Stay up-to-date on JavaScript and tools Discussing JS0/JSSugar docs.google.com We linked to the JS0/JSSugar slides in last week's issue. We now have reactions to those
Gartner's 2025 tech trends: Adapt - and fast
Tuesday, October 22, 2024
iPhone 16 Camera Control secrets; Lyft Election Day discount; Best TVs -- ZDNET ZDNET Tech Today - US October 22, 2024 future-tunne-gettyimages-108356213 Gartner's 2025 tech trends show how your
⚙️ Microsoft, Google & AI Agents
Tuesday, October 22, 2024
Plus: Europeans would let an AI vote for them
Post from Syncfusion Blogs on 10/22/2024
Tuesday, October 22, 2024
New blogs from Syncfusion Automate Flowchart Creation from External Data with Blazor Diagram By Suganthi Karuppannan Learn how to automate flowchart creation from external data using the Syncfusion
Urgent: VMware Releases vCenter Server Update to Fix Critical RCE Vulnerability
Tuesday, October 22, 2024
THN Daily Updates Newsletter cover See Yourself in Cyber: Security Careers Beyond Hacking ($17.00 Value) FREE for a Limited Time A one-of-a-kind discussion of how to integrate cybersecurity into every
New Blogs on ThomasMaurer.ch for 10/22/2024
Tuesday, October 22, 2024
View this email in your browser Thomas Maurer Cloud & Datacenter Update This is the update for blog posts on ThomasMaurer.ch. Azure Hybrid Cloud Pre-Day at Microsoft Ignite 2024 By Thomas Maurer on
BetterDev #270 - Should We Chat, Too? Security Analysis of WeChat’s MMTLS Encryption Protocol
Tuesday, October 22, 2024
Better Dev #270 Oct 21, 2024 Hi all, Welcome to another issue of BetterDev! This week I come across Colmi, a smart ring where you can write your own software to interact with it. It's also have a
JSK Daily for Oct 21, 2024
Monday, October 21, 2024
JSK Daily for Oct 21, 2024 View this email in your browser A community curated daily e-mail of JavaScript news Getting Started with Piecesjs: Building Native Web Components with a Lightweight Framework
📑 Microsoft Word Helps Me Overcome Writer's Block — VR Mods That'll Make You Want a Headset
Monday, October 21, 2024
Also: How to Check Your iPhone's Battery Health, and More! How-To Geek Logo October 21, 2024 Did You Know The brand name "Crayola" was created by Alice (Stead) Binney, the wife of the