📝 Editorial 

The frantic pace of machine learning (ML) research is pushing the complexity of neural networks to new highs. Larger neural networks seem to be the state-of-the-art these days, reaching new milestones in areas such as natural language processing (NLP),  computer vision, and speech analysis. Despite the progress, one of the most significant limitations for adopting big ML models remains how little we know about the way they generalize knowledge. Arguably, one of the biggest mysteries of ML is understanding why functions learned by neural networks generalize to unseen data. We are all impressed with the performance of GPT-3, but we can’t quite explain it. The future of ML has to be based on explainable ML and, to get there, we might have to go back to the first principles.  

A few days ago, researchers from Berkeley AI Research (BAIR) quietly published what I think could be one of the most important ML papers of the last few years. Behind the weird title of “Neural Tangent Kernel Eigenvalues Accurately Predict Generalization”, BAIR tries to formulate a first-principles theory of generalization. In a nutshell, the BAIR research reformulates subjective why-questions with a quantitative problem: given a network architecture, a target function f, and a training set of n random examples, can we efficiently predict the generalization performance of the network’s learned function f? The research shows that the incomprehensible complexity of neural networks is ruled by relatively simple rules. Even though BAIR’s research can’t be considered a complete theory of neural network generalization, it’s certainly an encouraging step in that direction.

Share

🔺🔻TheSequence Scope – our Sunday edition with the industry’s development overview – is free. To receive high-quality content about the most relevant developments in the ML world every Tuesday and Thursday, please subscribe to TheSequence Edge 🔺🔻

🗓 Next week in TheSequence Edge:

Edge#137: Detailed recap of our self-supervising (SSL) series.

Edge#138: Deep dive into Toloka App Services

Now, let’s review the most important developments in the AI industry this week

🔎 ML Research

Grouping Tasks in Multi-Task Models 

Which types of tasks should a neural network learn together? Google Research published a paper proposing a task-grouping technique for multi-task networks →read more on Google Research blog

Learning and Evolution 

Researchers from Stanford University published a paper proposing a technique called “deep evolutionary reinforcement learning,” or DERL, which uses complex virtual environments to simulate evolutionary dynamics and improve the learning of agents →read more in the original paper on Nature 

A First-Principles Theory of Generalization  

Berkeley AI Research (BAIR) published a fascinating paper outlining a quantitative theory for neural network generalization →read more on BAIR blog

Advances in Model-Based Optimization  

Berkeley AI Research (BAIR) published a blog post summarizing recent advancements in model-based optimization methods, which are actively used in design problems →read more on BAIR blog

🛠 Real World ML

Grammar Corrections in Pixel 6 

Google Research published some details about the models powering grammar correction capabilities on Gboard on Pixel 6 →read more on Google Research blog

Adapting LinkedIn’s ML Talent Solutions to COVID times 

The LinkedIn engineering team published a blog post detailing some of the building blocks used to improve ML solutions based on the dynamics of the job market during the pandemic →read more on LinkedIn blog

🤖 Cool AI Tech Releases

Metaflow UI 

Netflix open-sourced a new UI for its Metaflow ML platform →read more on Netflix blog

EC2 DL1 Instances 

AWS announced the general availability of DL1 instances, which improve the training of deep learning models by up to 40% →read more in this press release from AWS 

💎 We recommend

Watch a powerhouse panel of executives from Starbucks, HubSpot, and WestCap, talk about how today’s low-code/no-code trends, including automated predictive analytics, are applied to their unique business case and how they’re using it to drive business strategy.

Watch the panel

💸 Money in AI

ML&AI:

• AI model development platform Abacus.ai raised $50 million in a Series C round led by Tiger Global. Hiring.

• Data quality platform Anomalo raised a $33 million Series A round led by Norwest Venture Partners. Hiring remote.

AI-powered:

• Logging and security analytics platform Devo Technology raised $250 million in a series E funding round led by TCV. Hiring in Madrid, Spain.

• Cybersecurity startup Dragos raised $200 million in a Series D funding round led by Koch Disruptive Technologies. Hiring remote/USA/Canada.

• Network automation and security service provider BackBox raised $32 million in a Series A round led by venture firm Elsewhere Partners. Hiring in Dallas, US/Israel/Remote.

• Anti-phishing platform SlashNext raised $26 million in Series B venture capital funding. Hiring.

• No-code visual intelligence platform Cogniac raised $20 million in a Series B1 financing round led by National Grid Partners. Hiring remote/USA.

Acquisitions:

• Content moderation provider Two Hat was acquired by Microsoft for an undisclosed amount.

• Business automation platform Clear Software was acquired by Microsoft on undisclosed terms.

• Business intelligence platform Momentive was acquired by Zendesk.

• Conversational cloud platform LivePerson acquired speech recognition and conversational analytics startup VoiceBase and customer engagement platform Tenfold for an undisclosed sum. Hiring globally.

Follow us on Twitter