TheSequence - 🔂 Edge#217: ML Testing Series – Recap
Was this email forwarded to you? Sign up here Last week we finished our mini-series about ML testing, one of the most critical elements of the ML models’ lifecycle. Here is a full recap for you to catch up with the topics we covered. As the proverb (and many ML people) says: Repetition is the mother of learning ;) The essence of ML testing is to execute explicit checks that validate the behavior of an ML model. This approach contrasts with testing in traditional software applications. In a web or mobile application, users provide tests in the form of logic and data and fine-tune the system’s behavior. The cycle is different in ML, where a test starts with the expected behavior and the corresponding dataset, with the model’s logic as an output. Plenty of taxonomies can be used to organize ML testing techniques. A very general approach segments testing techniques into two main groups relative to the ML model lifecycle:
Typically, both types of tests should be incorporated into an MLOps pipeline. Also, tests should include both code and data. Over the years, there have been a lot of different ML test techniques that have been widely covered in research. Examples include invariance tests, minimum functionality tests, directional tests and many others. Forward this email to those who might benefit from reading it or give a gift subscription. → In Edge#209 (read it without a subscription): we explore how Uber backtests time-series forecasting models at scale; and discuss Deepchecks, an ML testing platform you should know about. → In Edge#211: we discuss what to test in ML models; explain how Meta uses A/B testing to improve Facebook’s newsfeed algorithm; and explore Meta’s Ax, a framework for A/B testing in PyTorch. → In Edge#213: we overview the fundamental types of tests to be applied to trained models; explain how Meta uses Bayesian Optimization to conduct better experiments in ML models; and explore TensorFlow’s What-If Tool, one of the most commonly used testing tools in the machine learning space. → In Edge#215: we discuss Pre-Train Model Testing; overview the pillars of robust machine learning; and explore Great Expectations, one of the most complete data validation frameworks used in ML pipelines. Next week we are going back to deep learning theory. Our next mini-series will cover a new generation of text-image models and their underlying techniques. Fascinating! You’re on the free list for TheSequence Scope and TheSequence Chat. For the full experience, become a paying subscriber to TheSequence Edge. Trusted by thousands of subscribers from the leading AI labs and universities. |
Older messages
📙 Free book: Meet the Data Science Innovators
Monday, August 15, 2022
Learn from top data science leaders, who share their insights on their groundbreaking innovations, their careers, and the data science profession. Who's doing the most innovative things in data
😴 ❌ Don’t Sleep on JAX
Sunday, August 14, 2022
Weekly news digest curated by the industry insiders
📌 Event: Last chance to register for conference on scalable AI – Aug 23-24 in San Francisco!
Friday, August 12, 2022
The world's top minds in AI and distributed computing are coming to Ray Summit — August 23-24 in San Francisco. Join the global Ray community for two days of keynotes, training, and technical
🐈⬛ Edge#216: DeepMind’s New Super Model can Generalize Across Multiple Tasks on Different Domains
Friday, August 12, 2022
Gato is able to master tasks such as image classification, question answering or controlling a robotic arm
🏛 Edge#215: Pre-Train Model Testing and the Pillars of Robust ML
Tuesday, August 9, 2022
In this issue: we discuss Pre-Train Model Testing; we overview the pillars of robust machine learning; we explore Great Expectations
You Might Also Like
Edge 389: Understanding Large Action Models
Tuesday, April 23, 2024
One of the most important concepts in autonomous agents. ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏
Apple World Cup bid ⚽, Meta opens VR OS 🌎, Anthropic's prompt library 🤖
Tuesday, April 23, 2024
Apple is working to secure an exclusive TV deal with FIFA Sign Up |Advertise|View Online TLDR Together With WorkOS TLDR 2024-04-23 WorkOS is the only auth provider your B2B SaaS app needs to start
New Blogs on ThomasMaurer.ch for 04/23/2024
Tuesday, April 23, 2024
View this email in your browser Thomas Maurer Cloud & Datacenter Update This is the update for blog posts on ThomasMaurer.ch. Cloud operations for Windows Server through Azure Arc By Thomas Maurer
Post-Post 🗨️
Tuesday, April 23, 2024
Assessing the post-Twitter climate amid Post.News' shutdown. Here's a version for your browser. Hunting for the end of the long tail • April 22, 2024 Post-Post The demise of Post, one of the
BetterDev #257 - Building a GPS Receiver
Monday, April 22, 2024
Better Dev #257 Apr 22, 2024 Hi all, We come back with a new issue this week. If you like BetterDev, please help spead word out by refer to your friends. Buy me a coffee would be great too. This week I
Tomorrow's Free Notes Class: How to sign up!
Monday, April 22, 2024
Hi there, Tomorrow we will be hosting a Free Notes App Class. This is your last chance to register for tomorrow's live class and learn how to get the most out of your Notes app. Our experienced
Elon’s ‘balls to the wall’ autonomy push
Monday, April 22, 2024
Plus: Amazon ends California drone deliveries and Rippling's founder has a brand-new bag View this email online in your browser By Christine Hall Monday, April 22, 2024 Image Credits: Toru Hanai/
📱 Your iPhone is Now Discoverable by Others — Tips for Building Your First PC
Monday, April 22, 2024
Also: How to Play Windows Games on Your Mac, and More! How-To Geek Logo April 22, 2024 📩 Get expert reviews, the hottest deals, how-to's, breaking news, and more delivered directly to your inbox by
JSK Daily for Apr 22, 2024
Monday, April 22, 2024
JSK Daily for Apr 22, 2024 View this email in your browser A community curated daily e-mail of JavaScript news It Is so Cool to Develop React Native With Expo 1. What are the benefits of Expo?. "
😺 The social walkie-talkie
Monday, April 22, 2024
Hi, hi! It's Monday and it's Earth Day! Don't miss the Cat Nips section below for innovative products in the... Product Hunt Read in browser This newsletter is brought to you by YOU MIGHT