Anomaly detection

Last updated

In data analysis, anomaly detection (also referred to as outlier detection and sometimes as novelty detection) is generally understood to be the identification of rare items, events or observations which deviate significantly from the majority of the data and do not conform to a well defined notion of normal behavior. [1] Such examples may arouse suspicions of being generated by a different mechanism, [2] or appear inconsistent with the remainder of that set of data. [3]

Contents

Anomaly detection finds application in many domains including cybersecurity, medicine, machine vision, statistics, neuroscience, law enforcement and financial fraud to name only a few. Anomalies were initially searched for clear rejection or omission from the data to aid statistical analysis, for example to compute the mean or standard deviation. They were also removed to better predictions from models such as linear regression, and more recently their removal aids the performance of machine learning algorithms. However, in many applications anomalies themselves are of interest and are the observations most desirous in the entire data set, which need to be identified and separated from noise or irrelevant outliers.

Three broad categories of anomaly detection techniques exist. [1] Supervised anomaly detection techniques require a data set that has been labeled as "normal" and "abnormal" and involves training a classifier. However, this approach is rarely used in anomaly detection due to the general unavailability of labelled data and the inherent unbalanced nature of the classes. Semi-supervised anomaly detection techniques assume that some portion of the data is labelled. This may be any combination of the normal or anomalous data, but more often than not, the techniques construct a model representing normal behavior from a given normal training data set, and then test the likelihood of a test instance to be generated by the model. Unsupervised anomaly detection techniques assume the data is unlabelled and are by far the most commonly used due to their wider and relevant application.

Definition

Many attempts have been made in the statistical and computer science communities to define an anomaly. The most prevalent ones include the following, and can be categorised into three groups: those that are ambiguous, those that are specific to a method with pre-defined thresholds usually chosen empirically, and those that are formally defined:

Ill defined

Specific

History

Intrusion detection

The concept of intrusion detection, a critical component of anomaly detection, has evolved significantly over time. Initially, it was a manual process where system administrators would monitor for unusual activities, such as a vacationing user's account being accessed or unexpected printer activity. This approach was not scalable and was soon superseded by the analysis of audit logs and system logs for signs of malicious behavior. [4]

By the late 1970s and early 1980s, the analysis of these logs was primarily used retrospectively to investigate incidents, as the volume of data made it impractical for real-time monitoring. The affordability of digital storage eventually led to audit logs being analyzed online, with specialized programs being developed to sift through the data. These programs, however, were typically run during off-peak hours due to their computational intensity. [4]

The 1990s brought the advent of real-time intrusion detection systems capable of analyzing audit data as it was generated, allowing for immediate detection of and response to attacks. This marked a significant shift towards proactive intrusion detection. [4]

As the field has continued to develop, the focus has shifted to creating solutions that can be efficiently implemented across large and complex network environments, adapting to the ever-growing variety of security threats and the dynamic nature of modern computing infrastructures. [4]

Applications

Anomaly detection is applicable in a very large number and variety of domains, and is an important subarea of unsupervised machine learning. As such it has applications in cyber-security, intrusion detection, fraud detection, fault detection, system health monitoring, event detection in sensor networks, detecting ecosystem disturbances, defect detection in images using machine vision, medical diagnosis and law enforcement. [5]

Intrusion detection

Anomaly detection was proposed for intrusion detection systems (IDS) by Dorothy Denning in 1986. [6] Anomaly detection for IDS is normally accomplished with thresholds and statistics, but can also be done with soft computing, and inductive learning. [7] Types of features proposed by 1999 included profiles of users, workstations, networks, remote hosts, groups of users, and programs based on frequencies, means, variances, covariances, and standard deviations. [8] The counterpart of anomaly detection in intrusion detection is misuse detection.

Fintech fraud detection

Anomaly detection is vital in fintech for fraud prevention. [9] [10]

Preprocessing

Preprocessing data to remove anomalies can be an important step in data analysis, and is done for a number of reasons. Statistics such as the mean and standard deviation are more accurate after the removal of anomalies, and the visualisation of data can also be improved. In supervised learning, removing the anomalous data from the dataset often results in a statistically significant increase in accuracy. [11] [12]

Video surveillance

Anomaly detection has become increasingly vital in video surveillance to enhance security and safety. [13] [14] With the advent of deep learning technologies, methods using Convolutional Neural Networks (CNNs) and Simple Recurrent Units (SRUs) have shown significant promise in identifying unusual activities or behaviors in video data. [13] These models can process and analyze extensive video feeds in real-time, recognizing patterns that deviate from the norm, which may indicate potential security threats or safety violations. [13]

IT infrastructure

In IT infrastructure management, anomaly detection is crucial for ensuring the smooth operation and reliability of services. [15] Techniques like the IT Infrastructure Library (ITIL) and monitoring frameworks are employed to track and manage system performance and user experience. [15] Detection anomalies can help identify and pre-empt potential performance degradations or system failures, thus maintaining productivity and business process effectiveness. [15]

IoT systems

Anomaly detection is critical for the security and efficiency of Internet of Things (IoT) systems. [16] It helps in identifying system failures and security breaches in complex networks of IoT devices. [16] The methods must manage real-time data, diverse device types, and scale effectively. Garbe et al. [17] have introduced a multi-stage anomaly detection framework that improves upon traditional methods by incorporating spatial clustering, density-based clustering, and locality-sensitive hashing. This tailored approach is designed to better handle the vast and varied nature of IoT data, thereby enhancing security and operational reliability in smart infrastructure and industrial IoT systems. [17]

Petroleum industry

Anomaly detection is crucial in the petroleum industry for monitoring critical machinery. [18] Martí et al. used a novel segmentation algorithm to analyze sensor data for real-time anomaly detection. [18] This approach helps promptly identify and address any irregularities in sensor readings, ensuring the reliability and safety of petroleum operations. [18]

Oil and gas pipeline monitoring

In the oil and gas sector, anomaly detection is not just crucial for maintenance and safety, but also for environmental protection. [19] Aljameel et al. propose an advanced machine learning-based model for detecting minor leaks in oil and gas pipelines, a task traditional methods may miss. [19]

Methods

Many anomaly detection techniques have been proposed in literature. [1] [20] The performance of methods usually depend on the data sets. For example, some may be suited to detecting local outliers, while others global, and methods have little systematic advantages over another when compared across many data sets. [21] [22] Almost all algorithms also require the setting of non-intuitive parameters critical for performance, and usually unknown before application. Some of the popular techniques are mentioned below and are broken down into categories:

Statistical

Parameter-free

Also referred to as frequency-based or counting-based, the simplest non-parametric anomaly detection method is to build a histogram with the training data or a set of known normal instances, and if a test point does not fall in any of the histogram bins mark it as anomalous, or assign an anomaly score to test data based on the height of the bin it falls in [1] . The size of bins are key to the effectiveness of this technique but must be determined by the implementer.

A more sophisticated technique uses kernel functions to approximate the distribution of the normal data. Instances in low probability areas of the distribution are then considered anomalies [23] .

Parametric-based

Density

Neural networks

Cluster-based

Ensembles

Others

Histogram-based Outlier Score (HBOS) uses value histograms and assumes feature independence for fast predictions. [51]

Anomaly detection in dynamic networks

Dynamic networks, such as those representing financial systems, social media interactions, and transportation infrastructure, are subject to constant change, making anomaly detection within them a complex task. Unlike static graphs, dynamic networks reflect evolving relationships and states, requiring adaptive techniques for anomaly detection.

Types of anomalies in dynamic networks

  1. Community anomalies
  2. Compression anomalies
  3. Decomposition anomalies
  4. Distance anomalies
  5. Probabilistic model anomalies

Explainable anomaly detection

Many of the methods discussed above only yield an anomaly score prediction, which often can be explained to users as the point being in a region of low data density (or relatively low density compared to the neighbor's densities). In explainable artificial intelligence, the users demand methods with higher explainability. Some methods allow for more detailed explanations:

Software

Datasets

See also

Related Research Articles

An intrusion detection system (IDS) is a device or software application that monitors a network or systems for malicious activity or policy violations. Any intrusion activity or violation is typically either reported to an administrator or collected centrally using a security information and event management (SIEM) system. A SIEM system combines outputs from multiple sources and uses alarm filtering techniques to distinguish malicious activity from false alarms.

<span class="mw-page-title-main">Outlier</span> Observation far apart from others in statistics and data science

In statistics, an outlier is a data point that differs significantly from other observations. An outlier may be due to a variability in the measurement, an indication of novel data, or it may be the result of experimental error; the latter are sometimes excluded from the data set. An outlier can be an indication of exciting possibility, but can also cause serious problems in statistical analyses.

Machine learning (ML) is a field of study in artificial intelligence concerned with the development and study of statistical algorithms that can learn from data and generalize to unseen data, and thus perform tasks without explicit instructions. Advances in the field of deep learning have allowed neural networks to surpass many previous approaches in performance.

Unsupervised learning is a framework in machine learning where, in contrast to supervised learning, algorithms learn patterns exclusively from unlabeled data. Other frameworks in the spectrum of supervisions include weak- or semi-supervision, where a small portion of the data is tagged, and self-supervision. Some researchers consider self-supervised learning a form of unsupervised learning.

A recommender system (RecSys), or a recommendation system (sometimes replacing system with terms such as platform, engine, or algorithm), is a subclass of information filtering system that provides suggestions for items that are most pertinent to a particular user. Recommender systems are particularly useful when an individual needs to choose an item from a potentially overwhelming number of items that a service may offer.

<span class="mw-page-title-main">Cluster analysis</span> Grouping a set of objects by similarity

Cluster analysis or clustering is the task of grouping a set of objects in such a way that objects in the same group are more similar to each other than to those in other groups (clusters). It is a main task of exploratory data analysis, and a common technique for statistical data analysis, used in many fields, including pattern recognition, image analysis, information retrieval, bioinformatics, data compression, computer graphics and machine learning.

The curse of dimensionality refers to various phenomena that arise when analyzing and organizing data in high-dimensional spaces that do not occur in low-dimensional settings such as the three-dimensional physical space of everyday experience. The expression was coined by Richard E. Bellman when considering problems in dynamic programming. The curse generally refers to issues that arise when the number of datapoints is small relative to the intrinsic dimension of the data.

In predictive analytics, data science, machine learning and related fields, concept drift or drift is an evolution of data that invalidates the data model. It happens when the statistical properties of the target variable, which the model is trying to predict, change over time in unforeseen ways. This causes problems because the predictions become less accurate as time passes. Drift detection and drift adaptation are of paramount importance in the fields that involve dynamically changing data and data models.

Density-based spatial clustering of applications with noise (DBSCAN) is a data clustering algorithm proposed by Martin Ester, Hans-Peter Kriegel, Jörg Sander, and Xiaowei Xu in 1996. It is a density-based clustering non-parametric algorithm: given a set of points in some space, it groups together points that are closely packed, and marks as outliers points that lie alone in low-density regions . DBSCAN is one of the most commonly used and cited clustering algorithms.

In machine learning, one-class classification (OCC), also known as unary classification or class-modelling, tries to identify objects of a specific class amongst all objects, by primarily learning from a training set containing only the objects of that class, although there exist variants of one-class classifiers where counter-examples are used to further refine the classification boundary. This is different from and more difficult than the traditional classification problem, which tries to distinguish between two or more classes with the training set containing objects from all the classes. Examples include the monitoring of helicopter gearboxes, motor failure prediction, or the operational status of a nuclear plant as 'normal': In this scenario, there are few, if any, examples of catastrophic system states; only the statistics of normal operation are known.

Ordering points to identify the clustering structure (OPTICS) is an algorithm for finding density-based clusters in spatial data. It was presented by Mihael Ankerst, Markus M. Breunig, Hans-Peter Kriegel and Jörg Sander. Its basic idea is similar to DBSCAN, but it addresses one of DBSCAN's major weaknesses: the problem of detecting meaningful clusters in data of varying density. To do so, the points of the database are (linearly) ordered such that spatially closest points become neighbors in the ordering. Additionally, a special distance is stored for each point that represents the density that must be accepted for a cluster so that both points belong to the same cluster. This is represented as a dendrogram.

Fraud represents a significant problem for governments and businesses and specialized analysis techniques for discovering fraud using them are required. Some of these methods include knowledge discovery in databases (KDD), data mining, machine learning and statistics. They offer applicable and successful solutions in different areas of electronic fraud crimes.

<span class="mw-page-title-main">ELKI</span> Data mining framework

ELKI is a data mining software framework developed for use in research and teaching. It was originally created by the database systems research unit at the Ludwig Maximilian University of Munich, Germany, led by Professor Hans-Peter Kriegel. The project has continued at the Technical University of Dortmund, Germany. It aims at allowing the development and evaluation of advanced data mining algorithms and their interaction with database index structures.

In anomaly detection, the local outlier factor (LOF) is an algorithm proposed by Markus M. Breunig, Hans-Peter Kriegel, Raymond T. Ng and Jörg Sander in 2000 for finding anomalous data points by measuring the local deviation of a given data point with respect to its neighbours.

Hans-Peter Kriegel is a German computer scientist and professor at the Ludwig Maximilian University of Munich and leading the Database Systems Group in the Department of Computer Science. He was previously professor at the University of Würzburg and the University of Bremen after habilitation at the Technical University of Dortmund and doctorate from Karlsruhe Institute of Technology.

In network theory, link analysis is a data-analysis technique used to evaluate relationships between nodes. Relationships may be identified among various types of nodes, including organizations, people and transactions. Link analysis has been used for investigation of criminal activity, computer security analysis, search engine optimization, market research, medical research, and art.

Massive Online Analysis (MOA) is a free open-source software project specific for data stream mining with concept drift. It is written in Java and developed at the University of Waikato, New Zealand.

Approximate computing is an emerging paradigm for energy-efficient and/or high-performance design. It includes a plethora of computation techniques that return a possibly inaccurate result rather than a guaranteed accurate result, and that can be used for applications where an approximate result is sufficient for its purpose. One example of such situation is for a search engine where no exact answer may exist for a certain search query and hence, many answers may be acceptable. Similarly, occasional dropping of some frames in a video application can go undetected due to perceptual limitations of humans. Approximate computing is based on the observation that in many scenarios, although performing exact computation requires large amount of resources, allowing bounded approximation can provide disproportionate gains in performance and energy, while still achieving acceptable result accuracy. For example, in k-means clustering algorithm, allowing only 5% loss in classification accuracy can provide 50 times energy saving compared to the fully accurate classification.

Arthur Zimek is a professor in data mining, data science and machine learning at the University of Southern Denmark in Odense, Denmark.

References

  1. 1 2 3 4 5 Chandola, V.; Banerjee, A.; Kumar, V. (2009). "Anomaly detection: A survey". ACM Computing Surveys . 41 (3): 1–58. doi:10.1145/1541880.1541882. S2CID   207172599.
  2. 1 2 Hawkins, Douglas M. (1980). Identification of Outliers. Springer. ISBN   978-0-412-21900-9. OCLC   6912274.
  3. 1 2 Barnett, Vic; Lewis, Lewis (1978). Outliers in statistical data. Wiley. ISBN   978-0-471-99599-9. OCLC   1150938591.
  4. 1 2 3 4 Kemmerer, R.A.; Vigna, G. (April 2002). "Intrusion detection: a brief history and overview". Computer. 35 (4): supl27–supl30. doi:10.1109/mc.2002.1012428. ISSN   0018-9162.
  5. ↑ Aggarwal, Charu (2017). Outlier Analysis. Springer Publishing Company, Incorporated. ISBN   978-3319475776.
  6. ↑ Denning, D. E. (1987). "An Intrusion-Detection Model" (PDF). IEEE Transactions on Software Engineering . SE-13 (2): 222–232. CiteSeerX   10.1.1.102.5127 . doi:10.1109/TSE.1987.232894. S2CID   10028835. Archived (PDF) from the original on June 22, 2015.
  7. ↑ Teng, H. S.; Chen, K.; Lu, S. C. (1990). "Adaptive real-time anomaly detection using inductively generated sequential patterns". Proceedings. 1990 IEEE Computer Society Symposium on Research in Security and Privacy (PDF). pp. 278–284. doi:10.1109/RISP.1990.63857. ISBN   978-0-8186-2060-7. S2CID   35632142.
  8. ↑ Jones, Anita K.; Sielken, Robert S. (2000). "Computer System Intrusion Detection: A Survey". Computer Science Technical Report. Department of Computer Science, University of Virginia: 1–25}.
  9. ↑ Stojanović, Branka; Božić, Josip; Hofer-Schmitz, Katharina; Nahrgang, Kai; Weber, Andreas; Badii, Atta; Sundaram, Maheshkumar; Jordan, Elliot; Runevic, Joel (January 2021). "Follow the Trail: Machine Learning for Fraud Detection in Fintech Applications". Sensors. 21 (5): 1594. Bibcode:2021Senso..21.1594S. doi: 10.3390/s21051594 . ISSN   1424-8220. PMC   7956727 . PMID   33668773.
  10. ↑ Ahmed, Mohiuddin; Mahmood, Abdun Naser; Islam, Md. Rafiqul (February 2016). "A survey of anomaly detection techniques in financial domain". Future Generation Computer Systems. 55: 278–288. doi:10.1016/j.future.2015.01.001. ISSN   0167-739X. S2CID   204982937.
  11. ↑ Tomek, Ivan (1976). "An Experiment with the Edited Nearest-Neighbor Rule". IEEE Transactions on Systems, Man, and Cybernetics . 6 (6): 448–452. doi:10.1109/TSMC.1976.4309523.
  12. ↑ Smith, M. R.; Martinez, T. (2011). "Improving classification accuracy by identifying and removing instances that should be misclassified" (PDF). The 2011 International Joint Conference on Neural Networks. p. 2690. CiteSeerX   10.1.1.221.1371 . doi:10.1109/IJCNN.2011.6033571. ISBN   978-1-4244-9635-8. S2CID   5809822.
  13. 1 2 3 4 5 6 Qasim, Maryam; Verdu, Elena (2023-06-01). "Video anomaly detection system using deep convolutional and recurrent models". Results in Engineering. 18: 101026. doi: 10.1016/j.rineng.2023.101026 . ISSN   2590-1230. S2CID   257728239.
  14. ↑ Zhang, Tan; Chowdhery, Aakanksha; Bahl, Paramvir (Victor); Jamieson, Kyle; Banerjee, Suman (2015-09-07). "The Design and Implementation of a Wireless Video Surveillance System". Proceedings of the 21st Annual International Conference on Mobile Computing and Networking. MobiCom '15. New York, NY, USA: Association for Computing Machinery. pp. 426–438. doi:10.1145/2789168.2790123. ISBN   978-1-4503-3619-2. S2CID   12310150.
  15. 1 2 3 Gow, Richard; Rabhi, Fethi A.; Venugopal, Srikumar (2018). "Anomaly Detection in Complex Real World Application Systems". IEEE Transactions on Network and Service Management. 15: 83–96. doi:10.1109/TNSM.2017.2771403. hdl: 1959.4/unsworks_73660 . S2CID   3883483 . Retrieved 2023-11-08.
  16. 1 2 Chatterjee, Ayan; Ahmed, Bestoun S. (August 2022). "IoT anomaly detection methods and applications: A survey". Internet of Things. 19: 100568. arXiv: 2207.09092 . doi: 10.1016/j.iot.2022.100568 . ISSN   2542-6605. S2CID   250644468.
  17. 1 2 Garg, Sahil; Kaur, Kuljeet; Batra, Shalini; Kaddoum, Georges; Kumar, Neeraj; Boukerche, Azzedine (2020-03-01). "A multi-stage anomaly detection scheme for augmenting the security in IoT-enabled applications". Future Generation Computer Systems. 104: 105–118. doi:10.1016/j.future.2019.09.038. ISSN   0167-739X. S2CID   204077191.
  18. 1 2 3 Martí, Luis; Sanchez-Pi, Nayat; Molina, José Manuel; Garcia, Ana Cristina Bicharra (February 2015). "Anomaly Detection Based on Sensor Data in Petroleum Industry Applications". Sensors. 15 (2): 2774–2797. Bibcode:2015Senso..15.2774M. doi: 10.3390/s150202774 . ISSN   1424-8220. PMC   4367333 . PMID   25633599.
  19. 1 2 Aljameel, Sumayh S.; Alomari, Dorieh M.; Alismail, Shatha; Khawaher, Fatimah; Alkhudhair, Aljawharah A.; Aljubran, Fatimah; Alzannan, Razan M. (August 2022). "An Anomaly Detection Model for Oil and Gas Pipelines Using Machine Learning". Computation. 10 (8): 138. doi: 10.3390/computation10080138 . ISSN   2079-3197.
  20. ↑ Zimek, Arthur; Filzmoser, Peter (2018). "There and back again: Outlier detection between statistical reasoning and data mining algorithms" (PDF). Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery. 8 (6): e1280. doi:10.1002/widm.1280. ISSN   1942-4787. S2CID   53305944. Archived from the original (PDF) on 2021-11-14. Retrieved 2019-12-09.
  21. ↑ Campos, Guilherme O.; Zimek, Arthur; Sander, Jörg; Campello, Ricardo J. G. B.; Micenková, Barbora; Schubert, Erich; Assent, Ira; Houle, Michael E. (2016). "On the evaluation of unsupervised outlier detection: measures, datasets, and an empirical study". Data Mining and Knowledge Discovery. 30 (4): 891. doi:10.1007/s10618-015-0444-8. ISSN   1384-5810. S2CID   1952214.
  22. ↑ Anomaly detection benchmark data repository of the Ludwig-Maximilians-Universität München; Mirror Archived 2022-03-31 at the Wayback Machine at University of São Paulo.
  23. ↑ Chandola, Varun; Banerjee, Arindam; Kumar, Vipin (2009-07-30). "Anomaly detection: A survey". ACM Comput. Surv. 41 (3): 15:1–15:58. doi:10.1145/1541880.1541882. ISSN   0360-0300.
  24. ↑ Knorr, E. M.; Ng, R. T.; Tucakov, V. (2000). "Distance-based outliers: Algorithms and applications". The VLDB Journal the International Journal on Very Large Data Bases. 8 (3–4): 237–253. CiteSeerX   10.1.1.43.1842 . doi:10.1007/s007780050006. S2CID   11707259.
  25. ↑ Ramaswamy, S.; Rastogi, R.; Shim, K. (2000). Efficient algorithms for mining outliers from large data sets. Proceedings of the 2000 ACM SIGMOD international conference on Management of data – SIGMOD '00. p. 427. doi:10.1145/342009.335437. ISBN   1-58113-217-4.
  26. ↑ Angiulli, F.; Pizzuti, C. (2002). Fast Outlier Detection in High Dimensional Spaces. Principles of Data Mining and Knowledge Discovery. Lecture Notes in Computer Science. Vol. 2431. p. 15. doi: 10.1007/3-540-45681-3_2 . ISBN   978-3-540-44037-6.
  27. ↑ Breunig, M. M.; Kriegel, H.-P.; Ng, R. T.; Sander, J. (2000). LOF: Identifying Density-based Local Outliers (PDF). Proceedings of the 2000 ACM SIGMOD International Conference on Management of Data. SIGMOD. pp. 93–104. doi:10.1145/335191.335388. ISBN   1-58113-217-4.
  28. ↑ Liu, Fei Tony; Ting, Kai Ming; Zhou, Zhi-Hua (December 2008). "Isolation Forest". 2008 Eighth IEEE International Conference on Data Mining. pp. 413–422. doi:10.1109/ICDM.2008.17. ISBN   9780769535029. S2CID   6505449.
  29. ↑ Liu, Fei Tony; Ting, Kai Ming; Zhou, Zhi-Hua (March 2012). "Isolation-Based Anomaly Detection". ACM Transactions on Knowledge Discovery from Data. 6 (1): 1–39. doi:10.1145/2133360.2133363. S2CID   207193045.
  30. ↑ Schubert, E.; Zimek, A.; Kriegel, H. -P. (2012). "Local outlier detection reconsidered: A generalized view on locality with applications to spatial, video, and network outlier detection". Data Mining and Knowledge Discovery. 28: 190–237. doi:10.1007/s10618-012-0300-z. S2CID   19036098.
  31. 1 2 Kriegel, H. P.; Kröger, P.; Schubert, E.; Zimek, A. (2009). Outlier Detection in Axis-Parallel Subspaces of High Dimensional Data. Advances in Knowledge Discovery and Data Mining. Lecture Notes in Computer Science. Vol. 5476. p. 831. doi:10.1007/978-3-642-01307-2_86. ISBN   978-3-642-01306-5.
  32. 1 2 Kriegel, H. P.; Kroger, P.; Schubert, E.; Zimek, A. (2012). Outlier Detection in Arbitrarily Oriented Subspaces. 2012 IEEE 12th International Conference on Data Mining. p. 379. doi:10.1109/ICDM.2012.21. ISBN   978-1-4673-4649-8.
  33. ↑ Fanaee-T, H.; Gama, J. (2016). "Tensor-based anomaly detection: An interdisciplinary survey". Knowledge-Based Systems. 98: 130–147. doi:10.1016/j.knosys.2016.01.027. S2CID   16368060.
  34. ↑ Zimek, A.; Schubert, E.; Kriegel, H.-P. (2012). "A survey on unsupervised outlier detection in high-dimensional numerical data". Statistical Analysis and Data Mining. 5 (5): 363–387. doi:10.1002/sam.11161. S2CID   6724536.
  35. ↑ Schölkopf, B.; Platt, J. C.; Shawe-Taylor, J.; Smola, A. J.; Williamson, R. C. (2001). "Estimating the Support of a High-Dimensional Distribution". Neural Computation. 13 (7): 1443–71. CiteSeerX   10.1.1.4.4106 . doi:10.1162/089976601750264965. PMID   11440593. S2CID   2110475.
  36. 1 2 3 Hawkins, Simon; He, Hongxing; Williams, Graham; Baxter, Rohan (2002). "Outlier Detection Using Replicator Neural Networks". Data Warehousing and Knowledge Discovery. Lecture Notes in Computer Science. Vol. 2454. pp. 170–180. CiteSeerX   10.1.1.12.3366 . doi:10.1007/3-540-46145-0_17. ISBN   978-3-540-44123-6. S2CID   6436930.
  37. ↑ An, J.; Cho, S. (2015). "Variational autoencoder based anomaly detection using reconstruction probability" (PDF). Special Lecture on IE. 2 (1): 1–18. SNUDM-TR-2015-03.
  38. ↑ Malhotra, Pankaj; Vig, Lovekesh; Shroff, Gautman; Agarwal, Puneet (22–24 April 2015). Long Short Term Memory Networks for Anomaly Detection in Time Series. ESANN 2015: 23rd European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning. pp. 89–94. ISBN   978-2-87587-015-5.
  39. ↑ Hubert, Mia; Debruyne, Michiel; Rousseeuw, Peter J. (2018). "Minimum covariance determinant and extensions". WIREs Computational Statistics. 10 (3). arXiv: 1709.07045 . doi: 10.1002/wics.1421 . ISSN   1939-5108. S2CID   67227041.
  40. ↑ Hubert, Mia; Debruyne, Michiel (2010). "Minimum covariance determinant". WIREs Computational Statistics. 2 (1): 36–43. doi:10.1002/wics.61. ISSN   1939-0068. S2CID   123086172.
  41. ↑ Alzubaidi, Laith; Zhang, Jinglan; Humaidi, Amjad J.; Al-Dujaili, Ayad; Duan, Ye; Al-Shamma, Omran; Santamaría, J.; Fadhel, Mohammed A.; Al-Amidie, Muthana; Farhan, Laith (2021-03-31). "Review of deep learning: concepts, CNN architectures, challenges, applications, future directions". Journal of Big Data. 8 (1): 53. doi: 10.1186/s40537-021-00444-8 . ISSN   2196-1115. PMC   8010506 . PMID   33816053.
  42. ↑ Belay, Mohammed Ayalew; Blakseth, Sindre Stenen; Rasheed, Adil; Salvo Rossi, Pierluigi (January 2023). "Unsupervised Anomaly Detection for IoT-Based Multivariate Time Series: Existing Solutions, Performance Analysis and Future Directions". Sensors. 23 (5): 2844. Bibcode:2023Senso..23.2844B. doi: 10.3390/s23052844 . ISSN   1424-8220. PMC   10007300 . PMID   36905048.
  43. ↑ He, Z.; Xu, X.; Deng, S. (2003). "Discovering cluster-based local outliers". Pattern Recognition Letters. 24 (9–10): 1641–1650. Bibcode:2003PaReL..24.1641H. CiteSeerX   10.1.1.20.4242 . doi:10.1016/S0167-8655(03)00003-5.
  44. ↑ Campello, R. J. G. B.; Moulavi, D.; Zimek, A.; Sander, J. (2015). "Hierarchical Density Estimates for Data Clustering, Visualization, and Outlier Detection". ACM Transactions on Knowledge Discovery from Data. 10 (1): 5:1–51. doi:10.1145/2733381. S2CID   2887636.
  45. ↑ Lazarevic, A.; Kumar, V. (2005). "Feature bagging for outlier detection". Proceedings of the eleventh ACM SIGKDD international conference on Knowledge discovery in data mining. pp. 157–166. CiteSeerX   10.1.1.399.425 . doi:10.1145/1081870.1081891. ISBN   978-1-59593-135-1. S2CID   2054204.
  46. ↑ Nguyen, H. V.; Ang, H. H.; Gopalkrishnan, V. (2010). Mining Outliers with Ensemble of Heterogeneous Detectors on Random Subspaces. Database Systems for Advanced Applications. Lecture Notes in Computer Science. Vol. 5981. p. 368. doi:10.1007/978-3-642-12026-8_29. ISBN   978-3-642-12025-1.
  47. ↑ Kriegel, H. P.; Kröger, P.; Schubert, E.; Zimek, A. (2011). Interpreting and Unifying Outlier Scores. Proceedings of the 2011 SIAM International Conference on Data Mining. pp. 13–24. CiteSeerX   10.1.1.232.2719 . doi:10.1137/1.9781611972818.2. ISBN   978-0-89871-992-5.
  48. ↑ Schubert, E.; Wojdanowski, R.; Zimek, A.; Kriegel, H. P. (2012). On Evaluation of Outlier Rankings and Outlier Scores. Proceedings of the 2012 SIAM International Conference on Data Mining. pp. 1047–1058. doi:10.1137/1.9781611972825.90. ISBN   978-1-61197-232-0.
  49. ↑ Zimek, A.; Campello, R. J. G. B.; Sander, J. R. (2014). "Ensembles for unsupervised outlier detection". ACM SIGKDD Explorations Newsletter. 15: 11–22. doi:10.1145/2594473.2594476. S2CID   8065347.
  50. ↑ Zimek, A.; Campello, R. J. G. B.; Sander, J. R. (2014). Data perturbation for outlier detection ensembles. Proceedings of the 26th International Conference on Scientific and Statistical Database Management – SSDBM '14. p. 1. doi:10.1145/2618243.2618257. ISBN   978-1-4503-2722-0.
  51. ↑ Goldstein, Markus; Dengel, Andreas (2012). "Histogram-based Outlier Score (HBOS): A fast Unsupervised Anomaly Detection Algorithm" (PDF). Personal page of Markus Goldstein. (Poster only at KI 2012 conference, not in proceedings)
  52. ↑ Zhao, Yue; Nasrullah, Zain; Li, Zheng (2019). "Pyod: A python toolbox for scalable outlier detection" (PDF). Journal of Machine Learning Research. 20. arXiv: 1901.01588 .
  53. ↑ "FindAnomalies". Mathematica documentation.
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.