The Next RLHF Effect: Three Breakhroughts that can Unlock the Next Wave of Innovation in Foundation Models
Was this email forwarded to you? Sign up here The Next RLHF Effect: Three Breakhroughts that can Unlock the Next Wave of Innovation in Foundation ModelsSundays, The Sequence Scope brings a summary of the most important research papers, technology releases and VC funding deals in the artificial intelligence space.Next Week in The Sequence:
📝 Editorial: Three Breakhroughts that can Unlock the Next Wave of Innovation in Foundation ModelsReinforcement learning with human feedback (RLHF) could arguably be credited as the technique that unlocked the generative AI frenzy we are currently experiencing. While RLHF is not as relevant as transformer architectures, it facilitated the transition from powerful yet uninteresting models like GPT-3 to world-changing phenomena like ChatGPT. In essence, RLHF enabled language models (LLMs) to overcome a significant hurdle by offering two key capabilities:
These two factors serve as the foundation of ChatGPT and the new generation of LLMs we have witnessed in recent months. If RLHF was the major breakthrough for ChatGPT, we must now question what the next research milestones will be that unlock a new wave of innovation in LLMs. Currently, there are three key areas in LLM research that have the potential to have a "RLHF effect" in the next generation of models:
If ChatGPT felt magical with knowledge limited to 2021, just imagine the impact of LLMs that can reason through tasks, access real-time data, and expand their knowledge in real time. The aforementioned areas represent some of the most active research domains in the LLM space, and these techniques are gradually being incorporated into new models. It won't be long before we witness another magical moment akin to the "RLHF effect." 🔎 ML ResearchNeuralangeloNVIDIA unveiled Neuralangelo, a generative AI model that can turn 2d video clips into 3D objects. The model can interpret complex aspects of videos such as textures or complex materials into the 3D representation —> Read more. GorillaResearchers from UC Berkeley and Microsoft Research published a paper detailing Gorilla, an LLM able to use APIs and tools. Gorilla is a fine-tuned version of LLaMA and surpasses GPT-4 in API call tasks —> Read more. Self-LearningMIT researchers published a paper outlining a self-learning LLM that is able to outperform much larger alternatives. The model uses a technique called self-training where the model used its own predictions to train itself —> Read more. REVEALGoogle Research published a paper introducing Retrieval-Augmented Visual-Language Pre-Training with Multi-Source Multimodal Knowledge Memory(REVEAL), a visual-language model that can answer knowledge intensive queries. The model employs neural representation learning to encode knowledge representations as memory structures that can be easily accessible —> Read more. Differential Privacy in MLGoogle Research published a paper reviewing the current state of ML differential privacy methods. The paper discusses the core DP ML techniques from an engineering and research perspective —> Read more. Improved Mathematical ReasoningOpenAI published a paper detailing a new technique for mathematical problem solving. The method relies on rewarding a model for the reasoning steps instead of the final results —> Read more. 📌 Event: LLMs in Production II — A Virtual Conference From the Forefront of AI - Jun 15-16There’s endless chatter about LLMs these days. But the people who actually have interesting things to say are the ones using LLMs in the wild. Join us at the upcoming LLMs in Production Conference to hear from the experts at the forefront of using LLMs. There will be over 50 speakers from Stripe, Meta, Canva, Databricks, Anthropic, Microsoft, Cohere, Redis, Langchain, Chroma, Humanloop, Jasper, Salesforce, and so many more. Be there on June 15-16! Register for the virtual conference for free, or join in-person workshops (SF). 🤖 Cool AI Tech ReleasesFalcon 40BThe Technology Innovation Institute (TII) open sourced Falcon 40B, an open source LLM trained on one trillion tokens —> Read more. AviaryAnycale, the company behind the Ray platform, open sourced a new LLM serving platform called Aviary —> Read more. 📡AI Radar
You’re on the free list for TheSequence Scope and TheSequence Chat. For the full experience, become a paying subscriber to TheSequence Edge. Trusted by thousands of subscribers from the leading AI labs and universities. |
Key phrases
Older messages
This week on TuringPost
Saturday, June 3, 2023
Hi there, Last week, we introduced Turing Post – the newsletter that complements TheSequence and covers other topics that help you make informed decisions about AI. It is for those who are in the AI
📝 Guest Post: Stop Hallucinations From Hurting your LLM Powered Apps*
Friday, June 2, 2023
LLM hallucinations pose a big threat to the successful adoption of the new wave of LLM apps. In this post, the Galileo team dives into how one can prevent hallucinations from creeping in, as well as
Edge 296: Inside OpenAI's Method to Use GPT-4 to Explain Neuron's Behaviors in GPT-2
Thursday, June 1, 2023
The technique is one of the first attempts to utilize LLMs as a explainability foundation.
The Sequence Chat: Rohan Taori on Stanford's Alpaca, Alpaca Farm and the Future of LLMs
Wednesday, May 31, 2023
Alpaca was one of the first open LLMs to incorporate instruction following capabilities. Now one of the project's main researchers shares his insights about modern LLMs.
Edge 295: Self-Instruct Models
Tuesday, May 30, 2023
What if LLMs could auto improve their own instruction following capabilities?
You Might Also Like
Relief From Tinnitus: Free Discovery Call!
Tuesday, April 30, 2024
Do you suffer from tinnitus or a ringing in your ears? 1 in 3 adults over the age of 65 will suffer from this condition and often don't know there are things you can do to help. Our friends at
WebAIM April 2024 Newsletter
Tuesday, April 30, 2024
WebAIM April 2024 Newsletter Read this newsletter online at https://webaim.org/newsletter/2024/april Feature Web Accessibility in the 2024 Presidential Campaigns WebAIM's John Northup ran the US
👀 Being More Productive on a Smaller Screen — How to Hide Games on Steam Family Sharing
Tuesday, April 30, 2024
Also: What to Expect From Apple's "Let Loose" Event, and More! How-To Geek Logo April 30, 2024 Did You Know The letter J is the only letter that makes no appearance on the Periodic Table.
PEP 686, Lazy Evaluation, Serverless Python, and More
Tuesday, April 30, 2024
PEP 686: Make UTF-8 Mode Default #627 – APRIL 30, 2024 VIEW IN BROWSER The PyCoder's Weekly Logo PEP 686: Make UTF-8 Mode Default This Python Enhancement Proposal outlines making UTF-8 the default
Daily Coding Problem: Problem #1427 [Easy]
Tuesday, April 30, 2024
Daily Coding Problem Good morning! Here's your coding interview problem for today. This problem was asked by Amazon. Given an array and a number k that's smaller than the length of the array,
🎙 My advice for film + TV creatives on the AI wave
Tuesday, April 30, 2024
Learning AI fast + Karate Kid references
Ranked | The Top 20 Countries in Debt to China 💸
Tuesday, April 30, 2024
The 20 nations featured in this graphic each owe billions in debt to China, often posing concerns for their economic future. View Online | Subscribe Presented by Voronoi: The App Where Data Tells the
Noonification: RaspberryPi: Technology Overload - LAMP🕯 to UASP🐝
Tuesday, April 30, 2024
Top Tech Content sent at Noon! The first AI-powered startup unlocking the “billionaire economy” for your benefit How are you, @newsletterest1? 🪐 What's happening in tech this week: The
Issue #50: ScreenFloat by Matthias Gansrigler
Tuesday, April 30, 2024
Today, we're looking at ScreenFloat by Matthias Gansrigler. ScreenFloat is a screenshot management utility that lets you create floating screenshots so you can easily reference them no matter what
You're invited - Taking the Guesswork out of Product Development
Tuesday, April 30, 2024
Taking the Guesswork out of Product Development Tuesday, May 14th @ 2:00PM GMT // 10:00 AM EST Approximately 50% of product budgets are allocated to reworking and optimizing post-launch experiences.