Edge 402: UC Berkeley's Large World Model Can Understand Really Long Videos
Was this email forwarded to you? Sign up here Edge 402: UC Berkeley's Large World Model Can Understand Really Long VideosOne of the most impressive research in generative video of the last year.Video understanding might become the next frontier for generative AI. Building AI models and agents that fully understand complex environments have long been one of the goals of AI. The recent generative AI revolution have expanded the horizons of AI models in order to understand environments using language, video and images. Obviously, video understanding seems to be the key to unlock this capability as videos include features such as object interaction, physics and other key characteristics of real world settings. A group of AI researchers from UC Berkeley that include AI legend Peiter Abbeel published a paper proposing a model that can learn complex representations from images and videos in seuqences of up to one million tokens. They named the model: large world model(LWM). The ProblemToday’s language models have difficulty grasping world aspects that are challenging to encapsulate solely through text, especially when it comes to managing intricate, extended tasks. Videos provide a rich source of temporal information that static images and text cannot offer, highlighting the potential benefits of integrating video with language in model training. This integration aims to create models that comprehend both textual knowledge and the physical world, broadening AI’s potential to assist humans. Nevertheless, the ambition to learn from millions of tokens spanning video and language sequences is hampered by significant hurdles such as memory limitations, computational challenges, and the scarcity of comprehensive datasets... Subscribe to TheSequence to read the rest.Become a paying subscriber of TheSequence to get access to this post and other subscriber-only content. A subscription gets you:
|
Older messages
Edge 401: Reflection and Refinement Planning Methods in Autonomous Agents
Tuesday, June 4, 2024
Can LLM agents handle planning errorts effectively? ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏
Generative AI Unicorn Capitulation
Monday, June 3, 2024
Adept and Humane are looking for buyers. ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏
Edge 399: Understanding External-Aid Planning and Autonomous Agents
Monday, June 3, 2024
How do we supply an agents with external help to improve its planning capabilities? ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏
Edge 400: Inside AlphaFold 3: Google DeepMind's Amazing BioScience Model
Monday, June 3, 2024
The model expands from its predecessors and is able to predict the structure of many of the life's molecules. ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏
Mistral Codestral is the Newest AI Model in the Code Generation Race
Monday, June 3, 2024
Plus updates from Elon Musk's xAI , several major funding rounds and intriguing research publications. ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏
You Might Also Like
💡 Can Renters Have a Smart Home? — Getting the SteamOS Beta on Steam Deck
Saturday, September 28, 2024
Also: Your Google Doc Can Now Have a Stylish Cover, and More! How-To Geek Logo September 28, 2024 Did You Know If Johnny Depp hadn't been available to play Willy Wonka in Charlie and the Chocolate
Meta's new empire: VR, AR and AI - Sync #486
Saturday, September 28, 2024
Plus: Mira Murati leaves OpenAI; Microsoft to revive a nuclear plant for its AI data centre; bioengineered trees that capture more carbon; stem cell therapy for diabetes; and more! ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏
The Paywall’s Final Frontier 🔒
Saturday, September 28, 2024
Could CNN's planned paywall be a harbinger for free online news? Here's a version for your browser. Hunting for the end of the long tail • September 28, 2024 The Paywall's Final Frontier
Feature | The Best Visualizations from September on Voronoi 🏆
Saturday, September 28, 2024
See the most popular, most discussed, and most liked visualizations on our new data storytelling app Voronoi from September View Online | Subscribe In December 2023, we publicly launched Voronoi, our
Daily Coding Problem: Problem #1570 [Medium]
Saturday, September 28, 2024
Daily Coding Problem Good morning! Here's your coding interview problem for today. This problem was asked by Epic. The "look and say" sequence is defined as follows: beginning with the
Will Data Centers Ruin Your Neighborhood?
Saturday, September 28, 2024
Top Tech Content sent at Noon! A dev conference with discussions, workshops, and 1:1 feedback sessions Read this email in your browser How are you, @newsletterest1? 🪐 What's happening in tech today
🐍 New Python tutorials on Real Python
Saturday, September 28, 2024
Hey there, There's always something going on over at Real Python as far as Python tutorials go. Here's what you may have missed this past week: Python Virtual Environments: A Primer In this
ALERT - Critical Linux Printing System Flaws Could Allow Remote Command Execution
Saturday, September 28, 2024
THN Daily Updates Newsletter cover [Watch LIVE] Building a Successful Data Security Posture Management Program Learn From the Leaders: Early DSPM Adopters Reveal Their Data Security Success Secrets
Monitor Your Heart Health Every Day
Saturday, September 28, 2024
Withings is reducing the price of BPM Connect to $99.95 in the US, reaffirming our dedication to accessible health tech. With nearly half the adult population affected by high blood pressure, we're
📧 Breaking It Down: How to Migrate Your Modular Monolith to Microservices
Saturday, September 28, 2024
Breaking It Down: How to Migrate Your Modular Monolith to Microservices Read on: my website / Read time: 9 minutes The .NET Weekly is brought to you by: Integrate e-signatures into your workflows