• Home
  • About
  • Contact
  • Privacy
  • Terms
  • DCMA
  • Write For Us / Submit
Tech News, Magazine & Review WordPress Theme 2017
  • Tech
    transfer whatsapp from android to iphone

    How to Transfer WhatsApp from Android to iPhone!

    Understanding the Role of Humidity Chambers in Climate Testing

    Understanding the Role of Humidity Chambers in Climate Testing

    Everything You Get to Know About Movember Beard Memes

    Everything You Get to Know About Movember Beard Memes

    Best Book Recommendation Apps

    11 Best Book Recommendation Apps

    How to quickly divide or Split PDF files

    How to Quickly Divide or Split PDF Files

    Enhance your omnichannel service strategies with data-driven insights.

    Black Market for Fraudulent Ad Accounts Found on Facebook

    Sen. Mark Warner and Sen. Marco Rubio at a hearing.

    Trump’s TikTok ban might have been right after all

    14 Gift Ideas for the Home Chef (2022): Vitamix, Cookbooks, and More

    14 Gift Ideas for the Home Chef (2022): Vitamix, Cookbooks, and More

    6 Android Apps for Budget Management

    Twitter, Facebook, Lyft layoffs spark fears of dotcom crash 2.0

  • Gear
    • All
    • Camera
    • Laptop
    • Smartphone
    Latest Smart Home Gadgets for a Connected Life

    Latest Smart Home Gadgets for a Connected Life

    Eco-Friendly Products for Students

    Eco-Friendly Products for Students

    Essential Photography Equipment and Gadgets

    Mastering Your Shots: Essential Photography Equipment and Gadgets

    Level Up Your Game: Must-Have Gaming Gear!

    Level Up Your Game: Must-Have Gaming Gear!

    Pixel Watch in the test

    Google Pixel Watch in the Test

    Top 5 Websites To Buy Camera Lenses At Reasonable Price

    Top 5 Websites To Buy Camera Lenses At Reasonable Price

    Trending Tags

    • Best iPhone 7 deals
    • Apple Watch 2
    • Nintendo Switch
    • CES 2017
    • Playstation 4 Pro
    • iOS 10
    • iPhone 7
    • Sillicon Valley
  • Gaming
    9 Best Alternative Games Like Kahoot

    9 Best Alternative Games Like Kahoot

    The Top 8 Free Bubble Shooter Games for Endless Entertainment

    The Top 8 Free Bubble Shooter Games for Endless Entertainment

    Cloud Gaming Revolution: How Streaming is Changing the Future of Gaming

    Cloud Gaming Revolution: How Streaming is Changing the Future of Gaming

    Level Up Your Game: Must-Have Gaming Gear!

    Level Up Your Game: Must-Have Gaming Gear!

    Game Streaming Platforms: Evolution & Impact on Industry

    Game Streaming Platforms: Evolution & Impact on Industry

    The Rise of Esports Games

    Esports Games: Exploring the Ecosystem, Growth, and Impact

  • Crypto
    transfer whatsapp from android to iphone

    How to Transfer WhatsApp from Android to iPhone!

    Understanding the Role of Humidity Chambers in Climate Testing

    Understanding the Role of Humidity Chambers in Climate Testing

    Everything You Get to Know About Movember Beard Memes

    Everything You Get to Know About Movember Beard Memes

    Best Book Recommendation Apps

    11 Best Book Recommendation Apps

    How to quickly divide or Split PDF files

    How to Quickly Divide or Split PDF Files

    Enhance your omnichannel service strategies with data-driven insights.

    Black Market for Fraudulent Ad Accounts Found on Facebook

    Sen. Mark Warner and Sen. Marco Rubio at a hearing.

    Trump’s TikTok ban might have been right after all

    14 Gift Ideas for the Home Chef (2022): Vitamix, Cookbooks, and More

    14 Gift Ideas for the Home Chef (2022): Vitamix, Cookbooks, and More

    6 Android Apps for Budget Management

    Twitter, Facebook, Lyft layoffs spark fears of dotcom crash 2.0

  • Business
    transfer whatsapp from android to iphone

    How to Transfer WhatsApp from Android to iPhone!

    Understanding the Role of Humidity Chambers in Climate Testing

    Understanding the Role of Humidity Chambers in Climate Testing

    Everything You Get to Know About Movember Beard Memes

    Everything You Get to Know About Movember Beard Memes

    Best Book Recommendation Apps

    11 Best Book Recommendation Apps

    How to quickly divide or Split PDF files

    How to Quickly Divide or Split PDF Files

    Enhance your omnichannel service strategies with data-driven insights.

    Black Market for Fraudulent Ad Accounts Found on Facebook

    Sen. Mark Warner and Sen. Marco Rubio at a hearing.

    Trump’s TikTok ban might have been right after all

    14 Gift Ideas for the Home Chef (2022): Vitamix, Cookbooks, and More

    14 Gift Ideas for the Home Chef (2022): Vitamix, Cookbooks, and More

    6 Android Apps for Budget Management

    Twitter, Facebook, Lyft layoffs spark fears of dotcom crash 2.0

No Result
View All Result
Geeky Insider
  • Tech
    transfer whatsapp from android to iphone

    How to Transfer WhatsApp from Android to iPhone!

    Understanding the Role of Humidity Chambers in Climate Testing

    Understanding the Role of Humidity Chambers in Climate Testing

    Everything You Get to Know About Movember Beard Memes

    Everything You Get to Know About Movember Beard Memes

    Best Book Recommendation Apps

    11 Best Book Recommendation Apps

    How to quickly divide or Split PDF files

    How to Quickly Divide or Split PDF Files

    Enhance your omnichannel service strategies with data-driven insights.

    Black Market for Fraudulent Ad Accounts Found on Facebook

    Sen. Mark Warner and Sen. Marco Rubio at a hearing.

    Trump’s TikTok ban might have been right after all

    14 Gift Ideas for the Home Chef (2022): Vitamix, Cookbooks, and More

    14 Gift Ideas for the Home Chef (2022): Vitamix, Cookbooks, and More

    6 Android Apps for Budget Management

    Twitter, Facebook, Lyft layoffs spark fears of dotcom crash 2.0

  • Gear
    • All
    • Camera
    • Laptop
    • Smartphone
    Latest Smart Home Gadgets for a Connected Life

    Latest Smart Home Gadgets for a Connected Life

    Eco-Friendly Products for Students

    Eco-Friendly Products for Students

    Essential Photography Equipment and Gadgets

    Mastering Your Shots: Essential Photography Equipment and Gadgets

    Level Up Your Game: Must-Have Gaming Gear!

    Level Up Your Game: Must-Have Gaming Gear!

    Pixel Watch in the test

    Google Pixel Watch in the Test

    Top 5 Websites To Buy Camera Lenses At Reasonable Price

    Top 5 Websites To Buy Camera Lenses At Reasonable Price

    Trending Tags

    • Best iPhone 7 deals
    • Apple Watch 2
    • Nintendo Switch
    • CES 2017
    • Playstation 4 Pro
    • iOS 10
    • iPhone 7
    • Sillicon Valley
  • Gaming
    9 Best Alternative Games Like Kahoot

    9 Best Alternative Games Like Kahoot

    The Top 8 Free Bubble Shooter Games for Endless Entertainment

    The Top 8 Free Bubble Shooter Games for Endless Entertainment

    Cloud Gaming Revolution: How Streaming is Changing the Future of Gaming

    Cloud Gaming Revolution: How Streaming is Changing the Future of Gaming

    Level Up Your Game: Must-Have Gaming Gear!

    Level Up Your Game: Must-Have Gaming Gear!

    Game Streaming Platforms: Evolution & Impact on Industry

    Game Streaming Platforms: Evolution & Impact on Industry

    The Rise of Esports Games

    Esports Games: Exploring the Ecosystem, Growth, and Impact

  • Crypto
    transfer whatsapp from android to iphone

    How to Transfer WhatsApp from Android to iPhone!

    Understanding the Role of Humidity Chambers in Climate Testing

    Understanding the Role of Humidity Chambers in Climate Testing

    Everything You Get to Know About Movember Beard Memes

    Everything You Get to Know About Movember Beard Memes

    Best Book Recommendation Apps

    11 Best Book Recommendation Apps

    How to quickly divide or Split PDF files

    How to Quickly Divide or Split PDF Files

    Enhance your omnichannel service strategies with data-driven insights.

    Black Market for Fraudulent Ad Accounts Found on Facebook

    Sen. Mark Warner and Sen. Marco Rubio at a hearing.

    Trump’s TikTok ban might have been right after all

    14 Gift Ideas for the Home Chef (2022): Vitamix, Cookbooks, and More

    14 Gift Ideas for the Home Chef (2022): Vitamix, Cookbooks, and More

    6 Android Apps for Budget Management

    Twitter, Facebook, Lyft layoffs spark fears of dotcom crash 2.0

  • Business
    transfer whatsapp from android to iphone

    How to Transfer WhatsApp from Android to iPhone!

    Understanding the Role of Humidity Chambers in Climate Testing

    Understanding the Role of Humidity Chambers in Climate Testing

    Everything You Get to Know About Movember Beard Memes

    Everything You Get to Know About Movember Beard Memes

    Best Book Recommendation Apps

    11 Best Book Recommendation Apps

    How to quickly divide or Split PDF files

    How to Quickly Divide or Split PDF Files

    Enhance your omnichannel service strategies with data-driven insights.

    Black Market for Fraudulent Ad Accounts Found on Facebook

    Sen. Mark Warner and Sen. Marco Rubio at a hearing.

    Trump’s TikTok ban might have been right after all

    14 Gift Ideas for the Home Chef (2022): Vitamix, Cookbooks, and More

    14 Gift Ideas for the Home Chef (2022): Vitamix, Cookbooks, and More

    6 Android Apps for Budget Management

    Twitter, Facebook, Lyft layoffs spark fears of dotcom crash 2.0

Submit
Geeky Insider
No Result
View All Result

The importance of monitoring machine learning models

by Staff Writer
July 21, 2023
in Education
Reading Time: 5 mins read
security monitoring
Share on FacebookShare on Twitter

 

Agile development teams must ensure that microservices, applications, and databases are observable, have monitoring in place to identify operational issues, and use AIops to correlate alerts into manageable incidents. When users and business stakeholders want enhancements, many DevOps teams follow agile methodologies to process feedback and deploy new versions.

Even if there are few requests, DevOps teams know they must upgrade apps and patch underlying components; otherwise, the software developed today will become tomorrow’s technical debt.

Table of Contents

    • READ ALSO
    • TigerGraph Cloud adds graph analytics, machine learning tools
    • Uvalde acting chief delayed after learning children were alive | Gun Violence News
  • Why monitor machine learning models?
  • Best practices in ML monitoring

READ ALSO

Study: Autistic People Recognize Emotions Better Than Neurotypical in Cartoons

Study: Autistic People Recognize Emotions Better Than Neurotypical in Cartoons

November 17, 2022
Feeding the world by AI, machine learning and the cloud

Feeding the world by AI, machine learning and the cloud

November 16, 2022

The agile transformation in machine learning models’ life-cycle management presents unique challenges. Andy Dang, co-founder and head of engineering at WhyLabs, highlights the complexity of the process. While similar to software development, ML models rely on complex and unpredictable data, making the transformation even more intricate.

In addition to code, components, and infrastructure, models are built using algorithms, configuration, and training data sets. These are selected and optimized at design time but need updating as assumptions and the data change over time.

Why monitor machine learning models?

Like monitoring applications for performance, reliability, and error conditions, machine learning model monitoring provides data scientists visibility on model performance. ML monitoring is especially important when models are used for predictions or when the ML runs on datasets with high volatility.

Dmitry Petrov, cofounder and CEO of Iterative, says, “The main goals around model monitoring focus on performance and troubleshooting as ML teams want to be able to improve on their models and ensure everything is running as intended.”

Rahul Kayala, principal product manager at Moveworks, shares this explanation on ML model monitoring. “Monitoring can help businesses balance the benefits of AI predictions with their need for predictable outcomes,” he says. “Automated alerts can help ML operations teams detect outliers in real time, giving them time to respond before any harm occurs.”

Stu Bailey, cofounder of ModelOp, adds, “Coupling robust monitoring with automated remediation accelerates time to resolution, which is key for maximizing business value and reducing risk.”

In particular, data scientists need to be notified of unexpected outliers. “AI models are often probabilistic, meaning they can generate a range of results,” says Kayala. “Sometimes, models can produce an outlier, an outcome significantly outside the normal range. Outliers can be disruptive to business outcomes and often have major negative consequences if they go unnoticed. To ensure AI models are impactful in the real world, ML teams should also monitor trends and fluctuations in product and business metrics that AI impacts directly.”

For example, let’s consider predicting a stock’s daily price. When there’s low market volatility, algorithms such as the long short-term memory (LSTM) can provide rudimentary predictions, and more comprehensive deep learning algorithms can improve accuracy. But most models will struggle to make accurate predictions when markets are highly volatile, and model monitoring can alert for these conditions.

Another type of ML model performs classifications, and precision and recall metrics can help track accuracy. Precision measures the true positives against the ones the model selected, while recall tracks a model’s sensitivity. ML monitoring can also alert on ML model drift, such as concept drift when the underlying statistics of what’s being predicted change, or data drift when the input data changes.

A third concern is explainable ML, where models are stressed to determine which input features contribute most significantly to the results. This issue relates to model bias, where the training data has statistical flaws that skew the model to make erroneous predictions.

These issues can erode trust and create significant business issues. Model performance management aims to address them across the development, training, deployment, and monitoring phases.

Krishnaram Kenthapadi, chief scientist at Fiddler, believes that explainable ML with reduced risk of biases requires model performance management. “To ensure ML models are not unduly discriminating, enterprises need solutions that deliver context and visibility into model behaviors throughout the entire life cycle—from model training and validation to analysis and improvement,” says Kenthapadi. “Model performance management ensures models are trustworthy and helps engineers and data scientists identify bias, monitor the root cause, and provide explanations for why those instances occurred in a timely manner.”

Best practices in ML monitoring

Modelops, ML monitoring, and model performance management are terms for practices and tools to ensure machine learning models operate as expected and provide trustworthy predictions. What underlying practices should data science and devops teams consider in their implementations?

Josh Poduska, chief field data scientist at Domino Data Lab, says, “Model monitoring is a critical, ongoing process. To improve future accuracy for a model that has drifted, retrain it with fresher data, along with its associated ground truth labels that are more representative of the current reality.“

Ira Cohen, chief data scientist and cofounder at Anodot, shares important factors in ML model monitoring. “First, models should monitor output and input features’ behavior, as shifts in the input features can cause issues,” he says. He suggests using proxy measures when model performance cannot be measured directly or quickly enough.

Cohen says data scientists need tools for model monitoring. He says, “Monitoring models manually is not scalable, and dashboards and reports are not equipped to handle the complexity and volume of the monitoring data generated when many AI models are deployed.”

Here are some recommended practices for ML model monitoring and performance management:

  • Petrov recommends, “Ensure you have the tools and automation in place upstream at the beginning of the model development life cycle to support your monitoring needs.”
  • Dang says, “Data engineers and scientists should run preliminary validations to ensure their data is in the expected format. As the data and the code move through a CI/CD pipeline, they should enable data unit testing through validations and constraint checks.”
  • Cohen suggests, “Use scalable anomaly detection algorithms that learn the behavior of each model’s inputs and outputs to alert when they deviate from the norm, effectively using AI to monitor AI.”
  • Kayala says, “Track the drift in the distribution of features. A large change in distribution indicates the need to retrain our models to achieve optimal performance.”
  • Bailey adds, “Increasingly, organizations are looking to monitor model risk and ROI as part of more comprehensive model governance programs, ensuring that models meet business and technical KPIs.”

Software development largely focuses on maintaining the code, monitoring application performance, improving reliability, and responding to operational and security incidents. In machine learning, ever-changing data, volatility, bias, and other factors require data science teams to manage models across their life cycle and monitor them in production.

Copyright © 2022 IDG Communications, Inc.

Source by www.infoworld.com

Related Posts

Business Lessons You Should Learn
Business

The 8 Business Lessons You Should Learn

February 28, 2023
How to quickly divide or Split PDF files
Apps And Softwares

How to Quickly Divide or Split PDF Files

February 28, 2023
How to protect yourself from phishing
How To

Phishing: The nasty tricks of the cybercriminals – this is how they protect themselves

July 28, 2023
Idaho university murders - latest: Police admit errors and backtrack as they reveal other roommates were in home
Education

Idaho university murders – latest: Police admit errors and backtrack as they reveal other roommates were in home

November 17, 2022

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Recommended.

6 Android Apps for Budget Management

AMD Launches RDNA 3, RX 7900 XT and XTX, Its Next-Gen GPU and Tech

November 3, 2022
6 Android Apps for Budget Management

Microsoft Introduces Microsoft Defender – A 365 Online Security App

June 30, 2022

Trending.

MicroSD card in hand.

How to Move Apps to SD Card on an Amazon Fire Tablet

October 5, 2022
How to Downgrade Google Chrome to an Older Version

How to Downgrade Google Chrome to an Older Version

September 7, 2022
Google's No Internet Dino game running on a laptop computer

How to Hack the Hidden Google Chrome Dinosaur Game

October 23, 2022
Employee Payroll Management Software Simplifying Processes

Employee Payroll Management Software: Simplifying Processes

August 23, 2023
Symbolic Link Header Image. A Windows folder on a blue background.

The Complete Guide to Creating Symbolic Links (aka Symlinks) on Windows

July 27, 2022
Geeky Insider

Geeky Insider is Web based independent Tech Magazine covering Topics Related to Technology, Gadgets, Software, Gaming, Crypto, Business And So on. Read More...

Submit a Techy Post | Write For Us

Feel free to contact us for submission queries. via contact form or email us at [email protected].

Categories

  • Android
  • Apple
  • Apps And Softwares
  • Articles
  • Business
  • Camera
  • Crypto
  • Education
  • Entertainment
  • Gadget and Gear
  • Gaming
  • How To
  • Laptop
  • Marketing
  • Microsoft
  • Review
  • Security
  • Smartphone
  • Technology
00-samsung FRP unlock tool

Unlock Samsung in Minutes: 5 Proven Hacks with the Ultimate FRP Unlock Tool!

September 19, 2023
transfer whatsapp from android to iphone

How to Transfer WhatsApp from Android to iPhone!

September 19, 2023
Understanding the Role of Humidity Chambers in Climate Testing

Understanding the Role of Humidity Chambers in Climate Testing

September 4, 2023
  • Home
  • About
  • Contact
  • Privacy
  • Terms
  • DCMA
  • Write For Us / Submit

© 2022 GeekyInsider.com

No Result
View All Result
  • Home
  • Review
  • Apple
  • Gaming
  • Gadget and Gear
    • Camera
    • Smartphone
  • Microsoft
  • Security