ALL YOU NEED IS…. ANOMALY DETECTION.
Within data management for predictive analytics, Anomaly Detection has emerged as one of the most widely used approaches, for being simple and intuitive, as well as effective.
It originated as a valid method for detecting intruders in computer systems and is widely used in fields as diverse as predictive maintenance or email classification as spam.
ARE CLASSIFICATION ALGORITHMS NOT ENOUGH?
Before continuing, a question: couldn’t we simply use, for example, in the case of preventive maintenance, a classification algorithm that allows us to categorize events as “correct” or “error”?
The answer is not easy to affirm or deny “a priori,” but it is true that despite the powerful tools that classification algorithms are, when it comes to categorizing events, these models present certain limitations.
PROBLEM 1: OVER-REPRESENTATION
The first of these, very present in real life, deals with the over-representation of one class compared to others.
Let’s consider, for example, an algorithm designed to predict failures in a certain industrial machine. To do this, and based on the supervised learning classification algorithm scheme, we need past samples of events categorized as “ok” and past events categorized as “failure,” to feed our algorithm so that it can, with experience, satisfactorily discern between “failure” and “ok” for future unclassified events.
If we have many more samples of the “ok” state than the “failure” state (as is common), the algorithm will tend to overemphasize the dominant class, not producing satisfactory results (the “failure” state is what we want to predict).
PROBLEM 2: DO WE KNOW ALL POSSIBLE FAILURES?
Furthermore, this programming structure requires knowing all situations in which the “failure” class appears, making it difficult to predict such an outcome when that failure has not occurred previously.
So, generally speaking, classification algorithms can have poor performance in these scenarios.
WHAT TO USE THEN?
This type of problem requires a different approach: Anomaly Detection.
In Anomaly Detection (everything sounds better in English), the idea becomes clear that to know when something is wrong, two approaches can be taken: either knowing when the parameters indicate that the event under study is going wrong, or knowing what the parameters are like when the event under study is functioning correctly.
The first is not always accessible, and with this second approach, when the parameters indicate that the event deviates from its normal parameters under optimal conditions, it will therefore alert to an anomaly.
Returning to the case of our industrial machine, it will be enough to parameterize the values when it is functioning correctly and, by establishing an appropriate moving range (which learns with experience), indicate an anomaly when we are out of range.
Providing this algorithm with new information, verifying the correct classification of events, can increase its accuracy (for example, by expanding its definition range for correct events).
ONE STEP CLOSER TO INDUSTRY 4.0
Although this approach helps us overcome the over-representation of the dominant class (something that other algorithms like SMOTE could address), the true power of anomaly detection lies in the fact that we can predict anomalies…
even if we have never seen one!!!
This makes it ideal for prediction tasks based on deterioration and wear, such as in maintenance tasks.
Currently, machines across all industries are generating data; they never stop communicating.
Gradually, tools are increasing to “listen” to them and allow them, for example, to tell us if they are well or unwell, if they need maintenance and what kind.
The Anomaly Detection algorithm is another support for advancing efficient resource and data management, and it allows companies to gain that competitive cost advantage, achieving a good position in one of the fastest-growing fields (and will continue to be, according to Gartner or Forrester reports): the Internet of Things (IoT).
Companies are striving to embark on this new wave of technology that forms Industry 4.0, and the efficient use of resources is key to achieving it.