TheSequence - You Need to Know About Groq
Was this email forwarded to you? Sign up here Next Week in The Sequence:
You can subscribe to The Sequence below:📝 Editorial: Groq’s Massive MilestoneMaking inference fast is one of the north stars for the next generation of AI infrastructure providers. The default assumption is that the AI inference market will be dominated by NVIDIA, and while that might turn out to be correct, there is innovation happening across all levels of the AI infrastructure stack. One of the most intriguing newcomers in this space is Groq, a startup that has emerged to become synonymous with fast AI inference. Groq is the maker of the Language Processing Unit (LPU), a chip optimized for fast AI inference. The Groq LPU is a single-core processor designed for LLMs, interconnected with a fast switchless routing fabric using 288 QSFP28 optical cables. A rack is built from 9 GroqNode 1 servers (with 1 server acting as a redundant resource), featuring a fully connected internal RealScale network delivering accelerated compute performance of up to 48 PetaOPs (INT8) or 12 PFLOPs (FP16). Groq Cloud is an LPU-based cloud offering that already includes many of the top generative AI models and counts over 350,000 developers. Groq is fast. In Groq Cloud, models can perform at over 500 tokens per second, with a 10x comparable improvement over GPT-4. The rise of Groq might seem sudden given the frantic pace of the AI market. However, the startup has been working on LPUs since 2016. Their perseverance has been rewarded. Last week, Groq announced a $640 million funding round led by Blackrock, along with Neuberger Berman, Type One Ventures, Cisco, KDDI, and Samsung Catalyst Fund. AI hardware is a tough market, but Groq now has the resources to innovate and compete. AI inference is still NVIDIA’s world, but Groq is a fascinating new player in it. 🔎 ML ResearchRobots for Table TennisGoogle DeepMind published a paper introducing the techniques building the first robot agent to achieved human competitive level in table tennis. The paper details different techniques such as hierarchical policy learning, sim-to-real and many others that are combined in a very clever way —> Read more. CodexGraphAliaba Research published a paper introducing CodexGraph, a system that integrates LLMs with a graph database based on code repositories. The graph model allows LLMs to navigate more sophisticated code structures adn tackle more complex tasks —> Read more. GENEVAMicrosoft Research published a paper introducing GENEVA, a tool that can generate rich narrative graphs based on a high level description and a set of constraints. GENEVA, explores different narractive paths through a visual, graph interface —> Read more. RAG FoundryResearchers from Intel published a paper detailing RAG Foundry, a framework for streamlining RAG use cases. RAG Foundry enables capabilities such as data creation, inference, evaluation in a single workflow —> Read more. LLM Scaling Without Increasing ParametersGoogle DeepMind published a paper discussing the importance of scaling test time computation in order to scale LLMs. The paper explores increasing test-time computation by searching against dense, process-based verifier reward models and updating the model distribution based on prompts at test time —> Read more. Self-Taught EvaluatorsMeta AI published a paper proposing an LLM-as-a-judge technique to improve evaluators withouht using human synthetic data. The method trains an LLM to produce reasoning traces and final judgments and repeats that process to obtain improved prediction —> Read more. 🤖 AI Tech ReleasesOpenAI API Structured OutputsOpenAI unveiled a new feature that enables structured JSON outputs as part of its API —> Read more. 🛠 Real World AIPrompt PoetCharacter.ai open sourced Prompt Poet, their framework for prompt design —> Read more. Smart Notifications at PinterestPinterest discusses the machine learning techniques used in their notification systems —> Read more. 📡AI Radar
You’re on the free list for TheSequence Scope and TheSequence Chat. For the full experience, become a paying subscriber to TheSequence Edge. Trusted by thousands of subscribers from the leading AI labs and universities. |
Older messages
📝 Guest Post: RAG Evaluation Using Ragas*
Friday, August 9, 2024
In this guest post, the teams from Zilliz and Ragas discuss key RAG evaluation metrics, their calculation, and implementation using the Milvus vector database and the Ragas package. ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏
Edge 420: Inside FlashAttention-3, The Algorithm Pushing the New Wave of Transformers
Thursday, August 8, 2024
The new algorithm takes full advantage of the capabilities of H100 GPUs. ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏
Edge 419: Everything You Need to Know About Autonomous Agents in 19 Posts
Tuesday, August 6, 2024
A summary of our long series about automous agents. ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏
Gemma 2: A Release That Matters
Sunday, August 4, 2024
A new model, a guardrails framework and an interpretability tool. ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏
Gemma 2: A Release That Matters
Sunday, August 4, 2024
A new model, a guardrails framework and an interpretability tool. ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏
You Might Also Like
💻 Issue 428 - C# different way to do a proof of concept
Thursday, September 19, 2024
This week's Awesome .NET Weekly Read this email on the Web The Awesome .NET Weekly Issue » 428 Release Date Sep 19, 2024 Your weekly report of the most popular .NET news, articles and projects
💎 Issue 435 - Ruby-SAML pwned by XML signature wrapping attacks
Thursday, September 19, 2024
This week's Awesome Ruby Newsletter Read this email on the Web The Awesome Ruby Newsletter Issue » 435 Release Date Sep 19, 2024 Your weekly report of the most popular Ruby news, articles and
💻 Issue 435 - Oracle, it's time to free JavaScript
Thursday, September 19, 2024
This week's Awesome JavaScript Weekly Read this email on the Web The Awesome JavaScript Weekly Issue » 435 Release Date Sep 19, 2024 Your weekly report of the most popular JavaScript news, articles
📱 Issue 429 - iOS 18 breaks IMAPS self-signed certs
Thursday, September 19, 2024
This week's Awesome iOS Weekly Read this email on the Web The Awesome iOS Weekly Issue » 429 Release Date Sep 19, 2024 Your weekly report of the most popular iOS news, articles and projects Popular
💻 Issue 353 - Why React Won the Front-End Race
Thursday, September 19, 2024
This week's Awesome React Weekly Read this email on the Web The Awesome React Weekly Issue » 353 Release Date Sep 19, 2024 Your weekly report of the most popular React news, articles and projects
💻 Issue 435 - DevSecOps Project: "Secure Full-Stack Node.js Web Application Deployment with Jenkins, Docker, Kubernetes, and HashiCorp Vault"
Thursday, September 19, 2024
This week's Awesome Node.js Weekly Read this email on the Web The Awesome Node.js Weekly Issue » 435 Release Date Sep 19, 2024 Your weekly report of the most popular Node.js news, articles and
📱 Issue 432 - Swift 6
Thursday, September 19, 2024
This week's Awesome Swift Weekly Read this email on the Web The Awesome Swift Weekly Issue » 432 Release Date Sep 19, 2024 Your weekly report of the most popular Swift news, articles and projects
💻 Issue 430 - Days since last Minecraft server written in Rust was released
Thursday, September 19, 2024
This week's Awesome Rust Weekly Read this email on the Web The Awesome Rust Weekly Issue » 430 Release Date Sep 19, 2024 Your weekly report of the most popular Rust news, articles and projects
Ranked | The Largest Producers of Wind Power, by Country ⚡
Thursday, September 19, 2024
Global wind power capacity hit fresh records in 2023 thanks to strategic government investment and lower technology costs. View Online | Subscribe | Download Our App Presented by: NEW REPORT: Brought
🧠 ChatGPT Passed the Turing Test — 5 Tips to Make Your Laptop Last Longer
Thursday, September 19, 2024
Also: How to Sideload Apps on Android TV, and More! How-To Geek Logo September 19, 2024 Did You Know Babies seem to have such large eyes because humans are born with eyes approximately 75 percent of