Edge 402: UC Berkeley's Large World Model Can Understand Really Long Videos
Was this email forwarded to you? Sign up here Edge 402: UC Berkeley's Large World Model Can Understand Really Long VideosOne of the most impressive research in generative video of the last year.Video understanding might become the next frontier for generative AI. Building AI models and agents that fully understand complex environments have long been one of the goals of AI. The recent generative AI revolution have expanded the horizons of AI models in order to understand environments using language, video and images. Obviously, video understanding seems to be the key to unlock this capability as videos include features such as object interaction, physics and other key characteristics of real world settings. A group of AI researchers from UC Berkeley that include AI legend Peiter Abbeel published a paper proposing a model that can learn complex representations from images and videos in seuqences of up to one million tokens. They named the model: large world model(LWM). The ProblemToday’s language models have difficulty grasping world aspects that are challenging to encapsulate solely through text, especially when it comes to managing intricate, extended tasks. Videos provide a rich source of temporal information that static images and text cannot offer, highlighting the potential benefits of integrating video with language in model training. This integration aims to create models that comprehend both textual knowledge and the physical world, broadening AI’s potential to assist humans. Nevertheless, the ambition to learn from millions of tokens spanning video and language sequences is hampered by significant hurdles such as memory limitations, computational challenges, and the scarcity of comprehensive datasets... Subscribe to TheSequence to read the rest.Become a paying subscriber of TheSequence to get access to this post and other subscriber-only content. A subscription gets you:
|
Older messages
Edge 401: Reflection and Refinement Planning Methods in Autonomous Agents
Tuesday, June 4, 2024
Can LLM agents handle planning errorts effectively? ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏
Generative AI Unicorn Capitulation
Monday, June 3, 2024
Adept and Humane are looking for buyers. ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏
Edge 399: Understanding External-Aid Planning and Autonomous Agents
Monday, June 3, 2024
How do we supply an agents with external help to improve its planning capabilities? ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏
Edge 400: Inside AlphaFold 3: Google DeepMind's Amazing BioScience Model
Monday, June 3, 2024
The model expands from its predecessors and is able to predict the structure of many of the life's molecules. ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏
Mistral Codestral is the Newest AI Model in the Code Generation Race
Monday, June 3, 2024
Plus updates from Elon Musk's xAI , several major funding rounds and intriguing research publications. ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏
You Might Also Like
📳 Galaxy Z Flip 6 Review — How to Watch the 2024 Summer Olympics for Free
Friday, July 26, 2024
Also: Fixing Spotify's Repeating Ads, and More! How-To Geek Logo July 26, 2024 Did You Know The rectangular area of a flag found in the upper left corner (top hoist corner) of the flag, such as the
Your monthly update has arrived
Friday, July 26, 2024
What's new in Google Play and Android July 2024 The Collections surface engages users with content Introducing Collections, a new on-device surface for your content Collections present users with
iOS Dev Weekly - Issue 671
Friday, July 26, 2024
There are two types of apps on the visionOS App Store. Will you create an app that makes people reach for the headset? 🥽 View on the Web Archives ISSUE 671 July 26th 2024 Comment In the last two weeks
Ranked | The 10 Busiest Ports in the World, by Cargo Traffic 🚢
Friday, July 26, 2024
As critical nodes for trade and commercial activity, we show the top 10 busiest ports in the world by cargo volume. View Online | Subscribe Presented by: Is Your Portfolio Powering the Future? >>
Let the Games Begin
Friday, July 26, 2024
Week of July 22, 2024 Let the Games Begin Week of July 22, 2024 By MG Siegler • 26 Jul 2024 View in browser View in browser Mark Zuckerberg loves two things above all else right now: llamas and
Daily Coding Problem: Problem #1508 [Hard]
Friday, July 26, 2024
Daily Coding Problem Good morning! Here's your coding interview problem for today. This problem was asked by Uber. Given an array of integers, return a new array such that each element at index i
OpenAI announces SearchGPT - Weekly News Roundup - Issue #477
Friday, July 26, 2024
Plus: Will billionaires live forever; a police robot dog jamming wireless networks; Alphabet to invest $5B into Waymo; warnings about “model collapse”; a new partnership for AI security; and more! ͏ ͏
Using Data as a Product Manager
Friday, July 26, 2024
If you had your choice between a little data or a lot of data on which to guide decisions, which would you pick?
Last Mile of Blockchains: RPC and Node-as-a-Service
Friday, July 26, 2024
Top Tech Content sent at Noon! Find the hottest jobs from top tech companies Read this email in your browser How are you, @newsletterest1? 🪐 What's happening in tech today, July 26, 2024? The
⚙️ Generative AI is making workers less productive
Friday, July 26, 2024
Plus: Runway trained video generator on thousands of YouTube videos