Edge 420: Inside FlashAttention-3, The Algorithm Pushing the New Wave of Transformers
Was this email forwarded to you? Sign up here Edge 420: Inside FlashAttention-3, The Algorithm Pushing the New Wave of TransformersThe new algorithm takes full advantage of the capabilities of H100 GPUs.
There are few algorithms that have had as much impact on the recent generation of transformer architectures as FlashAttention. Originally developed by researchers from Princeton University, including the renowned Tri Dao, FlashAttention and its successor FlashAttention-2 were able to improve the performance of attention mechanisms in GPUs by minimizing read-writes. Almost immediately after the original publication, FlashAttention was rapidly adopted within the new generation of transformers. There were not many complaints about FlashAttention, but one of the few was that it was unable to take full advantage of new hardware architectures. For instance, FlashAttention-2 is only able to achieve 35% utilization of max FLOPs in H100 GPUs. But now we have a new version... Subscribe to TheSequence to unlock the rest.Become a paying subscriber of TheSequence to get access to this post and other subscriber-only content. A subscription gets you:
|
Older messages
Edge 419: Everything You Need to Know About Autonomous Agents in 19 Posts
Tuesday, August 6, 2024
A summary of our long series about automous agents. ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏
Gemma 2: A Release That Matters
Sunday, August 4, 2024
A new model, a guardrails framework and an interpretability tool. ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏
Gemma 2: A Release That Matters
Sunday, August 4, 2024
A new model, a guardrails framework and an interpretability tool. ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏
📽 [Webinar] Beat GPT-4 with a Small Model and 10 Rows of Data*
Friday, August 2, 2024
Small language models (SLMs) are increasingly rivaling the performance of large foundation models like GPT-4. ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏
Edge 418: Meet The New DSPy: The Hot Framework to Build LLM Apps You Should Know About
Thursday, August 1, 2024
Created by Stanford researchers, the framework provides an interesting alternative to LangChain or LlamaIndex. ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏
You Might Also Like
Ruijie Networks' Cloud Platform Flaws Could Expose 50,000 Devices to Remote Attacks
Thursday, December 26, 2024
THN Daily Updates Newsletter cover Improve IT Efficiency with a Standardized OS: Nine considerations for building a standardized operating environment Optimize your IT with a standardized operating
Edge 460: Anthropic's New Protocol to Link AI Assistants to Data Sources
Thursday, December 26, 2024
Model Context Protocols is one of the recent AI contributions of the AI lab. ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏
December 26th 2024
Thursday, December 26, 2024
Curated news all about PHP. Here's the latest edition Is this email not displaying correctly? View it in your browser. PHP Weekly 26th December 2024 Hi everyone, It's boxing day in some parts
Re: This took me 10 minutes and protects my privacy
Thursday, December 26, 2024
Christmas may be over, but you still have one more chance to take advantage of Incogni's amazing holiday promotion! Protect your personal data from hackers and scammers today with Incogni's 58%
Daily Coding Problem: Problem #1648 [Medium]
Wednesday, December 25, 2024
Daily Coding Problem Good morning! Here's your coding interview problem for today. This problem was asked by Quora. Given an absolute pathname that may have . or .. as part of it, return the
🎮 The Best Games to Go With Your New Console — Streaming Services Could Learn From YouTube
Wednesday, December 25, 2024
Also: Don't Throw Christmas Gift Boxes on the Curb, and More! How-To Geek Logo December 25, 2024 Did You Know Years before The Nightmare Before Christmas, Tim Burton was sprinkling references to
Charted | Global Economic Confidence in 2025, by Country 🌎
Wednesday, December 25, 2024
While emerging markets in Asia have the strongest confidence in the global economy looking ahead, European countries are most pessimistic. View Online | Subscribe | Download Our App FEATURED STORY
Top Tech Deals 🎅 Sony Headphones, iPhone Cases, 4K Projector, and More!
Wednesday, December 25, 2024
The season of giving is upon us. How-To Geek Logo December 25, 2024 Top Tech Deals: Sony Headphones, iPhone Cases, 4K Projector, and More! The season of giving is upon us. Happy Holidays! If you're
Why the Race to AGI is Humanitys Defining Moment
Wednesday, December 25, 2024
Top Tech Content sent at Noon! Boost Your Article on HackerNoon for $159.99! Read this email in your browser How are you, @newsletterest1? 🪐 What's happening in tech today, December 25, 2024? The
Iran's Charming Kitten Deploys BellaCPP: A New C++ Variant of BellaCiao Malware
Wednesday, December 25, 2024
THN Daily Updates Newsletter cover The Data Science Handbook, 2nd Edition ($60.00 Value) FREE for a Limited Time Practical, accessible guide to becoming a data scientist, updated to include the latest