A data-driven approach for multivariate contextualized anomaly detection: industry use case

 Anomaly detection is the process of discovering some anomalous behaviour in the real-time operation of a system. It is a difficult task, since in a general case (multivariate anomaly detection, an anomaly can be related to the behaviour of several parameters which are not necessarily behaving anomalously per se, but their (complex) relation is anomalous (not usual/normal). This implies the need for a very efficient modeling of the normal behavior in order to know what should be treated as anomalous/outlier/unusual. Consequently, classical model-driven approaches, due to their focusing on the selected parameters for creating models, are not able to model the behavior if the whole system. This is why data-driven approaches for anomaly detection are getting ever more important for the industry use cases where hundreds (thousands) of parameter should be taken into account. However, current approaches are usually focused on the univariate anomaly detection (or some variations of it), so without going into observing the entire space of relations (computation very difficult). 

In this paper we present a novel approach for the multivariate anomaly detection that is based on modeling and managing the streams of variations in a multidimensional space. The main advantage of this approach is the possibility to observe the relations between variations of a large set of parameters and create clusters of “normal/usual” variations. In order to ensure scaling, which is one of the most challenging requirements, the approach is based on the usage of the big data technologies for realizing data analytics tasks/calculations. 

The approach is realized as a part of D2Lab (Data Diagnostics Laboratory) framework and has been applied in several industrial use cases. In this paper we present a very interesting usage for the anomaly detection in the process of functional testing of home appliances devices (in particular case refrigerators) after manufacturing/assembling process. It has been done for a bug vendor (Whirlpool), who expects huge saving in testing and improved customer satisfaction from this approach.