Introduction

Current AI architectures like Transformers are powerful and have achieved impressive results, including generalizing on previously unseen data and emergent abilities as the models are scaled. At the same time, they are still constrained compared to humans and animals who don’t need to see millions of data points before making the right conclusions or learning new skills like speaking. Examples include crows that can solve puzzles as five-year-olds, orcas’ sophisticated hunting methods that often deploy brutal, coordinated attacks, and elephants' cooperative abilities.

The Moravec paradox highlights that tasks that are difficult for humans, such as computing, are simple for computers to handle because they can be described and modeled easily. However, perception and sensory processing, which are natural for humans, are challenging for machines to master. Simply scaling the model and providing it with more data might not be a viable solution. Some argue that this approach will not lead to qualitatively different results that would enable AI models to reach a new level of reasoning or world perception. Therefore, alternative methods must be explored to enable AI to attain human-level intelligence. Yann LeCun (one of AI Godfathers) insists that JEPA is the first step.

In today’s episode, we will cover:

Yann LeCun is the one who was always rational about the latest models and exposed their limitations and educated the public that the fear of AGI and AI taking over humans is an unreasonable fear. In February 2022, Yann LeCun proposed his vision of achieving human-level reasoning by AI. And Joint Embedding Predictive Architecture (JEPA) was at the core of his vision. Let’s figure out what is it!

What are limitations of LLMs?

Yann LeCun also gave several talks presenting his vision for objective-driven AI: talk 1 (March 28, 2024), and talk 2 (September 9, 2023). There, he extensively discussed the limitations of large language models (LLMs):

So what’s the potential solution?