Edge 388: Google DeepMind's SIMA can Follow Language Instructions in 3D Games Just Like Humans
Was this email forwarded to you? Sign up here Edge 388: Google DeepMind's SIMA can Follow Language Instructions in 3D Games Just Like HumansThe AI agent represents a major improvement relative to expensive reinforcement learning methods.Video games have long served as some of the best environments for training AI agents. Since their early days, AI labs like OpenAI and DeepMind have built agents that excel at mastering video games such as Atari, Dota 2, StarCraft, and many others. The principles of many of these agents have been applied in areas such as embodied AI, self-driving cars, and many other domains that require taking action in different environments. However, most of the AI breakthroughs in 3D game environments have been constrained to one or a small number of games. Building that type of AI is really hard, but imagine if we could build agents that can understand many gaming worlds at once and follow instructions like a human player? A few weeks ago, Google DeepMind unveiled their work on the Scalable, Instructable, Multiworld Agent (SIMA). The goal of the project was to develop instructable agents that can interact with any 3D environment just like a human by following simple language instructions. This might not seem like a big deal until we consider that the standard way to communicate instructions has been with super-expensive reinforcement learning models. Language is the most powerful and yet simple abstraction for communicating instructions about the world or, in this case, a 3D virtual world. The magic of SIMA is its ability to translate those abstract instructions into mouse and keyboard actions used to navigate an environment... Subscribe to TheSequence to read the rest.Become a paying subscriber of TheSequence to get access to this post and other subscriber-only content. A subscription gets you:
|
Older messages
Edge 387: Tool Learning in Autonomous Agents
Tuesday, April 16, 2024
Agents that master tools and APIs, UC Berkeley's Gorilla and Microsoft's TaskWeaver ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏
Neuro-Symbolic Models are Making a Comeback
Sunday, April 14, 2024
A new startup called Symbolica comes out of stealth with a very different value proposition. ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏
Edge 386: Inside Yi, 01's Model Leading the Chinese LLM Movement
Saturday, April 13, 2024
Yi has achieved remarkable performance across language and image tasks. ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏
Edge 385: The Two Big Schools for Building Autonomous Agents
Tuesday, April 9, 2024
Language or computer-vision based agents? ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏
Generative Audio Models Just Had a Great Week
Sunday, April 7, 2024
Three major generative audio released in the last seven days. ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏
You Might Also Like
JSter #218 - Libraries and more
Wednesday, May 1, 2024
All JavaScript is good JavaScript. I'm close to done with my SurviveJS rework. The new site will have more content while being much lighter and faster to compile so that's all good. Libraries
BetterDev #258 - Build an 8-bit computer from scratch and Home automation with ESP8266
Wednesday, May 1, 2024
Better Dev #258 Apr 30, 2024 Hi all, We come back with a new issue this week. If you like BetterDev, please help spead word out by refer to your friends. Buy me a coffee would be great too. Build an 8-
Interface Interference 👎
Wednesday, May 1, 2024
Amid the AI device dunking, should everything “just be an app”? Here's a version for your browser. Hunting for the end of the long tail • April 30, 2024 Interface Interference The problem
Some Tesla Supercharger jobs get a jolt
Tuesday, April 30, 2024
Plus: Amazon CodeWhisperer changes its name and Arc gets a Windows version View this email online in your browser By Christine Hall Tuesday, April 30, 2024 Welcome to TechCrunch PM, bringing you the
Relief From Tinnitus: Free Discovery Call!
Tuesday, April 30, 2024
Do you suffer from tinnitus or a ringing in your ears? 1 in 3 adults over the age of 65 will suffer from this condition and often don't know there are things you can do to help. Our friends at
WebAIM April 2024 Newsletter
Tuesday, April 30, 2024
WebAIM April 2024 Newsletter Read this newsletter online at https://webaim.org/newsletter/2024/april Feature Web Accessibility in the 2024 Presidential Campaigns WebAIM's John Northup ran the US
👀 Being More Productive on a Smaller Screen — How to Hide Games on Steam Family Sharing
Tuesday, April 30, 2024
Also: What to Expect From Apple's "Let Loose" Event, and More! How-To Geek Logo April 30, 2024 Did You Know The letter J is the only letter that makes no appearance on the Periodic Table.
PEP 686, Lazy Evaluation, Serverless Python, and More
Tuesday, April 30, 2024
PEP 686: Make UTF-8 Mode Default #627 – APRIL 30, 2024 VIEW IN BROWSER The PyCoder's Weekly Logo PEP 686: Make UTF-8 Mode Default This Python Enhancement Proposal outlines making UTF-8 the default
Daily Coding Problem: Problem #1427 [Easy]
Tuesday, April 30, 2024
Daily Coding Problem Good morning! Here's your coding interview problem for today. This problem was asked by Amazon. Given an array and a number k that's smaller than the length of the array,
🎙 My advice for film + TV creatives on the AI wave
Tuesday, April 30, 2024
Learning AI fast + Karate Kid references