🟢 ⚪️Edge#158: A Deep Dive Into Aporia, the ML Observability Platform
This is an example of TheSequence Edge, a Premium newsletter that our subscribers receive every Tuesday and Thursday. On Thursdays, we do deep dives into one of the freshest research papers or technology frameworks that is worth your attention. 🤿 Deep Dive: Aporia: The Full-Stack ML Observability PlatformFinalizing our MLOps series, we’d like to circle back to the importance of ML Observability and give an overview of Aporia, the Full-Stack ML Observability Platform that is worth your attention. Aporia offers a completely customizable solution that includes visibility, monitoring and automation, advanced investigation tools, and explainability. The platform is designed to empower data science and ML engineering teams to build their own in-house observability platform, customized to their specific models and use cases. The Importance of Customized ML ObservabilityEvery model is unique. Even when organizations have the same use case for their model, such as fraud detection, the data used to train the model and the model itself is tailor-made to fit the needs of the organization. A data scientist’s job is to build these models to achieve the goals of the organization and ensure they work effectively. Data scientists train these models in a controlled environment. After deploying those models into production, they then enter an uncontrolled environment, where they are affected by real-world data and events that impact the model’s performance. To be able to maintain model performance in production, it’s critical to detect real-world changes that impact models such as drift, performance degradation, data integrity issues, and unexpected bias. Given that each model is unique, it’s essential to have a customizable ML observability solution that can be tailored by the experts behind the models to best support the specific use case and business needs. Customized Observability with AporiaAporia’s customizable ML observability enables data science and ML teams to visualize their models in production, analyze the behavior of different data segments, and compare model behavior to training or over time. This is done using common evaluation metrics, in addition to customized performance metrics, which a data scientist can customize to their specific models. Aporia’s monitor builder allows teams to easily create monitors for their models to detect a wide variety of behaviors, including drift, bias, data integrity issues, performance degradation, and more. Each monitor can run against different baselines – from the training set, a specific data segment comparison, or anomaly detection over time. With all these combinations, data science teams have access to over 50 customizable monitors, including a Python code-based monitor. Visibility & Investigation for Root Cause AnalysisWith Aporia, data science and ML teams can see all their models in one place, in real-time. Using the solution’s Time-Series and Datapoints Investigation tools, teams can also understand the root cause of any issue, discover when an issue actually started (before it reached the threshold or after), what was impacted by that issue, compare prediction distribution to training, feature importance and more. Furthermore, data scientists can see which features impacted predictions the most using advanced explainability tools. Combined, these tools enable a data scientist to answer the question: How do I resolve this particular issue and improve my model’s performance and business results? Improving Models in Production & Monitoring for DriftAporia’s focus on customization makes it easy for data scientists to go beyond drift detection and improve their models in production in a number of ways.
Explainability in AporiaAporia’s explainability tools help data scientists understand which features impact model predictions the most and explain model results to business owners. Explainability can also be used to simulate “What if” scenarios – quickly debugging prediction results by changing feature content and re-explaining the model. Together, these capabilities allow data scientists to better understand their model results in production and communicate those results to stakeholders. Customized Monitoring in Less Than 5 Minutes & Self-Hosted DeploymentWith Aporia’s self-hosted solution, the organization’s data remains secure. Integrating new models takes less than 3 minutes (see documentation). To try, you can sign up to Aporia’s free Community Edition, start with the demo model provided, or integrate your own production models to start monitoring and improving your models in production. Conclusion:ML systems play a central role in real-world production settings, from biomedical domains to space exploration. Hence, ensuring the consistent reliability of ML models is of utmost importance in the industry. ML Observability is, thus, an essential part of the production process for data science teams to monitor, validate and update the models deployed to users. Robust and diverse observability tools like Aporia can aid in identifying gaps in the deployed model or the production data, help visualize performance in every stage of the development, and recommend solutions to the detected problems. You can go over our full MLOps series here. Next week we will start a new fascinating series about Generative Models. Stay tuned! You’re on the free list for TheSequence Scope and TheSequence Chat. For the full experience, become a paying subscriber to TheSequence Edge. Trusted by thousands of subscribers from the leading AI labs and universities. |
Older messages
📝 Guest post: Data Labeling and Its Role in E-commerce Today – Recent Use Cases*
Wednesday, January 26, 2022
No subscription is needed
🔄🔄 Edge#159: MLOps Full Recap
Tuesday, January 25, 2022
Dive in!
👷♀️🧑🏻🎓👩💻👨🏻🏫 The MoE Momentum
Sunday, January 23, 2022
Weekly news digest curated by the industry insiders
📌 Learn from 40+ AI experts at mlcon 2.0 ML dev conf <Feb22-23>
Friday, January 21, 2022
Our partner cnvrg.io is hosting another incredible virtual conference mlcon 2.0! It is FREE
🥸 Edge#158: Microsoft KEAR is a Deep Learning Model for Common Sense Reasoning
Thursday, January 20, 2022
What's New in AI, a deep dive into one of the freshest research papers or technology frameworks that are worth your attention. Our goal is to keep you up to date with new developments in AI in a
You Might Also Like
Scoop: Tiger Global-backed Innovaccer in talks to raise $250M
Wednesday, May 1, 2024
Plus: An update on Google's layoffs and the social platform X didn't see coming View this email online in your browser By Christine Hall Wednesday, May 1, 2024 Welcome to TechCrunch PM. Today,
🖥️ Why I'm Never Going Back to a Windows PC — Tips Before You Buy a Smart Ring
Wednesday, May 1, 2024
Also: How to Clear the Moisture Detected Warning on Samsung Phones, and More How-To Geek Logo May 1, 2024 Did You Know A single 1 oz shot of espresso only has approximately 40 mg of caffeine, whereas a
Daily Coding Problem: Problem #1428 [Hard]
Wednesday, May 1, 2024
Daily Coding Problem Good morning! Here's your coding interview problem for today. This problem was asked by Microsoft. Given an array of positive integers, divide the array into two subsets such
Top Tech Deals 👀 Samsung Gaming Monitor, Pixel Watch 2, MacBook Air, and More
Wednesday, May 1, 2024
Get a discounted M3 MacBook Air or expand your Xbox storage. How-To Geek Logo May 1, 2024 Top Tech Deals: Samsung Gaming Monitor, Pixel Watch 2, MacBook Air, and More Get a discounted M3 MacBook Air or
Infographic | Visualizing Global Gold Production in 2023 🏅
Wednesday, May 1, 2024
Gold production in 2023 was led by China, Australia, and Russia, with each outputting over 300 tonnes. View Online | Subscribe Presented by: Access European benchmarks with a trusted 25-year history
⚙️ GPT-5 may be releasing sooner than expected
Wednesday, May 1, 2024
Plus: Amazon rebrands AI branch
Noonification: How to Create a CI/CD Pipeline Using GitHub and AWS EC2
Wednesday, May 1, 2024
Top Tech Content sent at Noon! Get Algolia: AI Search that understands How are you, @newsletterest1? 🪐 What's happening in tech today, May 1, 2024? The HackerNoon Newsletter brings the HackerNoon
Arc for Windows is better than Chrome
Wednesday, May 1, 2024
Adobe bug bounty; Rabbit's first R1 software update; Dream podcaster mic -- ZDNET ZDNET Tech Today - US May 1, 2024 placeholder Arc browser is now available for Windows and it's so much better
Is TikTok trying to get banned from the App Store early?
Wednesday, May 1, 2024
TikTok is offering some users a way to buy its in-app tipping tokens outside of Apple's App Store. View this email online in your browser By Alex Wilhelm Wednesday, May 1, 2024 Good morning, and
Get Compliant in 2024 - Download Ultimate PAM Policy Template Today
Wednesday, May 1, 2024
Privileged Access Management Policy Template What are your PAM policies for 2024? Get ready for the New Year Is your approach to Privileged Access Management as current and effective as it could be? In