If you were forwarded this newsletter, and you'd like to receive it in the future, subscribe here.​

​The Year Algorithms Learn to Act​

In the bustling tech campuses of 2024, the age of passive AI – systems that merely respond to our queries – is giving way to something far more profound: the era of AI agents.

As we look to 2025, we’re about to discover what happens when algorithms learn to act.

At the heart of these emerging agents lies a trinity of learning approaches :

• supervised learning : like reading a book to a child, humans provide clear guidance to AI labeling cat & dog, sheep & cow.
• unsupervised learning : AI discovers hidden patterns in data ; an ecommerce site recommends products you might like by clustering similar users’ purchasing patterns.
• reinforcement learning : an AI learns to play a video game by playing thousands of times, just the way a gamer might.

Deep learning means using the neural networks architecture to calculate an answer like what will the weather be tomorrow or summarize the Knicks game last night.

I remember studying deep learning in graduate school as the last chapter in a textbook - a professor’s offhand remark : “Here’s an idea that is fascinating but impractical!”

The transformer architecture changed everything. Like a printing press for neural nettworks, AI could process vast amounts of data, growing more capable with each gigabyte. More than its accuracy, its versatility grows : the same fundamentals that summarizes an article generates art, composes music, & translates.

Just as humans do, agents will face lots of uncertainty. A user asks to book a ticket for Moana 2 for the holidays but the time & location are booked. What should it do?

AI trained using DRL creates a mental model of the world & then strives to find the best answer considering time, computational expense, & other parameters.

Is it better to find the next nearest theater at the same time or find another time or ask the user?

The better the tools we provide to agents to model & explore the problem space, the better agents will act on our behalf. We’ve come a long way from that textbook chapter.

Miguel Morales, author of Grokking Deep Reinforcement Learning, produced the images above. It’s a wonderful book to understand the topic at a deeper level.

​