In this issue: 

• we start a new series about high scale ML training; 

• we explain SeedRL, an architecture for massively scaling the training of reinforcement learning agents; 

• we overview Horovod, an open-source framework created by Uber to streamline the parallelization of deep learning training workflows.   

Enjoy the learning!  

💡 ML Concept of the Day: A New Series About High Scale ML Training  

Training is one of the most important aspects of the lifecycle of ML models. In an ecosystem dominated by supervised learning techniques, having proper architectures for training is paramount for building robust ML systems. In the context of ML models, training is one of those aspects that is relatively simple to master at a small scale. But its complexity grows exponentially (really) with the size and complexity of a neural network. Over the last few years, the ML community has made significant advancements in both the research and implementation of high scale ML training methods. We will dedicate the next few weeks of TheSequence Edges to exploring the latest ML training methods and architectures powering some of the largest ML models in production.  

The complexity of ML training architectures grows with the size and number of models. What does it take to train a large transformer network like GPT-3? How to distribute the training process across many servers while ensuring the models can generalize? Similarly, what types of training architectures are better suited to train a large number of ML models?  

Not surprisingly, the vast majority of research in high scale ML training architectures has come from the AI labs in large technology companies like Microsoft, Google, Uber, and Facebook. These companies are building massively large ML models that require incredibly complex training infrastructures. As part of those efforts, tech incumbents have open-sourced many of the frameworks powering the training of those super large models. Systematizing the training of ML models requires as much ML research as engineering sophistication. In the next few weeks, we will dive deeper into some of the most advanced frameworks and research methods that can be used to train ML models at scale.  

🔎 ML Research You Should Know: SeedRL is an Architecture for Massively Scaling the Training of Reinforcement Learning Agents ...