Robotics is Inching Towards it ChatGPT Moment
Was this email forwarded to you? Sign up here Robotics is Inching Towards it ChatGPT MomentMajor developments in robotics from NVIDIA, Meta and MIT.Next Week in The Sequence:
You can subscribe to The Sequence below:📝 Editorial: Robotics is Inching Towards it ChatGPT MomentThe field of AI robotics is currently experiencing a surge in innovation, with researchers developing new techniques and technologies that are pushing the boundaries of what robots can do. One of the most exciting areas of development is the use of large language models (LLMs) to train robots. LLMs are a type of AI that are trained on massive datasets of text and code, and they have shown remarkable ability to generate text, translate languages, and write different kinds of creative content. Researchers are now exploring how to use LLMs to train robots to perform a wide range of tasks, from simple household chores to complex industrial operations. This week we saw several major research contributions in the field of robotics from NVIDIA, MIT and Meta among others. A major challenge in robotics is the heterogeneity of data. Robots generate data from a variety of sources, including vision sensors, robotic arm position encoders, and simulations. These data are often difficult to combine and use to train robots. Researchers at MIT have developed a new technique called Heterogeneous Pretrained Transformers (HPT) that addresses this challenge. HPT aligns data from different sources into a shared "language" that a generative AI model can process. This approach allows robots to be trained on a much larger and more diverse dataset, which can lead to significant improvements in performance. Beyond the technical advancements, the industry is witnessing a growing focus on the integration of touch perception, dexterity, and human-robot interaction. Meta's Fundamental AI Research (FAIR) team is actively working on creating embodied AI agents capable of perceiving and interacting with their surroundings, while also coexisting safely with humans. Their efforts are leading to advancements in areas such as tactile sensing, which allows robots to "feel" and manipulate objects with greater precision. This is exemplified by their development of Meta Sparsh, a general-purpose touch representation that works across various sensors and tasks, and Meta Digit 360, a breakthrough tactile fingertip with human-level multimodal sensing capabilities. The drive towards more versatile and adaptable robots is also evident in the development of new control frameworks. NVIDIA's research on HOVER (Humanoid Versatile Controller) showcases a multi-mode policy distillation framework that consolidates diverse control modes into a unified policy. HOVER allows robots to seamlessly switch between different control modes, such as navigation, manipulation, and human interaction, without the need for retraining.8 This development marks a significant step toward creating more flexible and adaptable robots that can perform a wide range of tasks. The advancements in AI robotics, as highlighted by these recent developments, demonstrate a clear momentum in the field. With the continuous development of new techniques and technologies, we can expect even more impressive progress in the near future. These breakthroughs not only promise to revolutionize industries but also hold the potential to significantly enhance our daily lives. 📍 EventYou’re invited to an exclusive fireside chat with Ben Orkin, VP of Engineering - MLOps at North, hosted by Tecton and Data Science Connect. Discover how this leading fintech company leveraged Tecton to build a system that detects fraud at scale with millisecond-level response times while adapting to emerging fraud patterns. You’ll learn:
Don't miss this deep dive into building mission-critical ML systems that balance speed, scale, and adaptability! –>Register here. 🔎 ML ResearchHOVERNVIDIA, Carnegie Mellon University, UC Berkeley and other AI research labs published the research around HOVER(Humanoid Versatile Controller), a 1.5 million parameter neural network to control humanoid robots. HOVER is based on a distillation method that extracts various control modes under the same policy —> Read more. NotebookLM AudioGoogle DeepMind published some details about the speech generation technologies behind NotebookLM and Illuminate. The solution included audio generation models such as AudioLM or SoundStream as well as specialized transformers for handling audio tokens —> Read more. Advancing Embodied AIMeta FAIR published several papers and research artifacts advancing different areas of embodied AI. The research includes areas such as perception, dexterity, and human-robot interaction —> Read more. Stealing User Prompts from MoEsGoogle DeepMind published a paper proposing an attack against MoE models that can unveil the user’s input prompt. The core of the technique centers on manipulating the expert routing system within the MoE model to capture the entire input —> Read more. LLMs as Data ScientistsSnowflake AI Research published a paper proposing FeatEng, a benchmark designed to evaluate LLMs in data science tasks such as feature engineering code. The benchmark presents a model with a dataset and a series of prompts and scores the generated code —> Read more. Memorization in LLMsResearchers from Princeton University, Google, Allen AI and University of Illinois published a paper proposing a quantitative approach to measure memorization in LLMs. The paper proposes a bechmark based on Knights and Knaves (K&K) puzzles to evaluate memorization in reasoning tasks —> Read more. 🤖 AI Tech ReleasesChatGPT SearchOpenAI unveiled ChatGPT Search allowing it to search web sources —> Read more. MobileLLMMeta AI open sourced MobileLLM, a foundation model optimized for on-device scenarios —> Read more. TensorFlow 2.18The new version of TensorFlow is out —> Read more. SmolLM2HuggingFace open sourced a series of small models optimized for edge computing —> Read more. 🛠 Real World AIConversational AI at AirbnbAirbnb revealed some details about the architecture powering its conversational AI experiences —> Read more. 📡AI Radar
You’re on the free list for TheSequence Scope and TheSequence Chat. For the full experience, become a paying subscriber to TheSequence Edge. Trusted by thousands of subscribers from the leading AI labs and universities. |
Older messages
📽 Fully Virtual: Agents in Production
Friday, November 1, 2024
Must-see event! ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏
Edge 444: Learn About Movie Gen: Meta AI's Amazing Audio-Video Generation Model
Thursday, October 31, 2024
The new model represents an important milestone open source video and audio generation. ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏
The Sequence Chat: Thinking About Transformers as Computers
Wednesday, October 30, 2024
A different way to reflect about the capabilities of transformers. ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏
Edge 443: EVERYTHING you Need to Know About State Space Models
Tuesday, October 29, 2024
A summary of our series about the most viable alternative to transformers. ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏
Anthropic, WOW
Sunday, October 27, 2024
New models, an agent that can interact with your computer and a new code generation tool. ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏
You Might Also Like
LW 157 - Better Collaboration Building for Merchants
Tuesday, November 5, 2024
Better Collaboration Building for Merchants Shopify Development news and articles Issue 157 - 11/
Vlt client & registry; ESMeta
Tuesday, November 5, 2024
We have 2 links for you - Stay up-to-date on JavaScript and tools Two vlt products: a better npm client and a serverless package registry www.vlt.sh @vltpkg@fosstodon.org vlt has launched two products:
Want a programming job? Learn this
Tuesday, November 5, 2024
Stop background Android apps; AI election tracker; Early Black Friday sales -- ZDNET ZDNET Tech Today - US November 5, 2024 Python code Your dream programming job demands this language, every site
⚙️ The battle for AI regulation
Tuesday, November 5, 2024
Plus: Perplexity's AI hub
Post from Syncfusion Blogs on 11/05/2024
Tuesday, November 5, 2024
New blogs from Syncfusion 6 Effective Ways to Merge PDF Files Using C# By Chinnu Muniyappan This blog explains the six effective methods to merge PDF files using C# and the Syncfusion .NET PDF Library
Google Warns of Actively Exploited CVE-2024-43093 Vulnerability in Android System
Tuesday, November 5, 2024
THN Daily Updates Newsletter cover The Data Science Workshop: Learn how you can build machine learning models and create your own real-world data science projects, Second Edition ($35.99 Value) FREE
Issue 161
Tuesday, November 5, 2024
🤖👮 Atlanta prison introduces 6-foot tall AI-powered robot guards. Meta's blood money: how Facebook turns tragedy into profit. OpenAI safety expert quits: "We're not ready for what's
Edge 445: A New Series About Knowledge Distillation
Tuesday, November 5, 2024
In this issue: ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏
New Blogs on ThomasMaurer.ch for 11/05/2024
Tuesday, November 5, 2024
View this email in your browser Thomas Maurer Cloud & Datacenter Update This is the update for blog posts on ThomasMaurer.ch. Honored to Receive the YouTube Silver Creator Award By Thomas Maurer on
📱 I Tried Running Ubuntu on My Phone — Samsung's One UI Is How Android Should Be
Monday, November 4, 2024
Also: The Most Realistic Game Simulations, and More! How-To Geek Logo November 4, 2024 Did You Know Peter Weller, best known for his role as Robocop, is an accomplished academic and actor. He has a