Knee of a curve

Last updated
Explained variance. The "elbow" is indicated by the red circle. The number of clusters chosen should therefore be 4. DataClustering ElbowCriterion.JPG
Explained variance. The "elbow" is indicated by the red circle. The number of clusters chosen should therefore be 4.
Photovoltaic solar cell I-V curves where a line intersects the knee of the curves where the maximum power transfer point is located. Solar-Cell-IV-curve-with-MPP.png
Photovoltaic solar cell I-V curves where a line intersects the knee of the curves where the maximum power transfer point is located.

In mathematics, a knee of a curve (or elbow of a curve) is a point where the curve visibly bends, specifically from high slope to low slope (flat or close to flat), or in the other direction. This is particularly used in optimization, where a knee point is the optimum point for some decision, for example when there is an increasing function and a trade-off between the benefit (vertical y axis) and the cost (horizontal x axis): the knee is where the benefit is no longer increasing rapidly, and is no longer worth the cost of further increases – a cutoff point of diminishing returns.

Contents

In heuristic use, the term may be used informally, and a knee point identified visually, but in more formal use an explicit objective function is used, and depends on the particular optimization problem. A knee may also be defined purely geometrically, in terms of the curvature or the second derivative.

Definitions

The knee of a curve can be defined as a vertex of the graph. This corresponds with the graphical intuition (it is where the curvature has a maximum), but depends on the choice of scale.

The term "knee" as applied to curves dates at least to the 1910s, [1] and is found more commonly by the 1940s, [2] being common enough to draw criticism. [3] [4] The unabridged Webster's Dictionary (1971 edition) gives definition 3h of knee as: [5]

an abrupt change in direction in a curve (as on a graph); esp one approaching a right angle in shape.

Criticism

Graphical notions of a "knee" of a curve, based on curvature, are criticized due to their dependence on the coordinate scale: different choices of scale result in different points being the "knee". This criticism dates at least to the 1940s, being found in Worthing & Geffner (1943 , Preface), who criticize: [4]

references to the significance of a so-called knee of a curve when the location of the knee was a function of the chosen coordinate scales

Detection methods

The Kneedle algorithm The algorithm detects the best balanced tradeoff based on the mathematical curvature concept, which is defined and well studied for continuous functions. [6] [7] Alternatively, the kneepointDetection() function [8] from the SamSPECTRAL [9] R package can be used to find the knee point, where is a "phase change" in the data, by fitting two lines using linear regression.

Applications

Related Research Articles

<span class="mw-page-title-main">Supervised learning</span> Paradigm in machine learning

Supervised learning (SL) is a paradigm in machine learning where input objects and a desired output value train a model. The training data is processed, building a function that maps new data to expected output values. An optimal scenario will allow for the algorithm to correctly determine output values for unseen instances. This requires the learning algorithm to generalize from the training data to unseen situations in a "reasonable" way. This statistical quality of an algorithm is measured through the so-called generalization error.

<span class="mw-page-title-main">Nonlinear dimensionality reduction</span> Projection of data onto lower-dimensional manifolds

Nonlinear dimensionality reduction, also known as manifold learning, is any of various related techniques that aim to project high-dimensional data, potentially existing across non-linear manifolds which cannot be adequately captured by linear decomposition methods, onto lower-dimensional latent manifolds, with the goal of either visualizing the data in the low-dimensional space, or learning the mapping itself. The techniques described below can be understood as generalizations of linear decomposition methods used for dimensionality reduction, such as singular value decomposition and principal component analysis.

<span class="mw-page-title-main">Time series</span> Sequence of data points over time

In mathematics, a time series is a series of data points indexed in time order. Most commonly, a time series is a sequence taken at successive equally spaced points in time. Thus it is a sequence of discrete-time data. Examples of time series are heights of ocean tides, counts of sunspots, and the daily closing value of the Dow Jones Industrial Average.

In mathematics, a piecewise linear or segmented function is a real-valued function of a real variable, whose graph is composed of straight-line segments.

<span class="mw-page-title-main">Curve fitting</span> Process of constructing a curve that has the best fit to a series of data points

Curve fitting is the process of constructing a curve, or mathematical function, that has the best fit to a series of data points, possibly subject to constraints. Curve fitting can involve either interpolation, where an exact fit to the data is required, or smoothing, in which a "smooth" function is constructed that approximately fits the data. A related topic is regression analysis, which focuses more on questions of statistical inference such as how much uncertainty is present in a curve that is fitted to data observed with random errors. Fitted curves can be used as an aid for data visualization, to infer values of a function where no data are available, and to summarize the relationships among two or more variables. Extrapolation refers to the use of a fitted curve beyond the range of the observed data, and is subject to a degree of uncertainty since it may reflect the method used to construct the curve as much as it reflects the observed data.

<span class="mw-page-title-main">Receiver operating characteristic</span> Diagnostic plot of binary classifier ability

A receiver operating characteristic curve, or ROC curve, is a graphical plot that illustrates the performance of a binary classifier model at varying threshold values.

In machine learning and pattern recognition, a feature is an individual measurable property or characteristic of a phenomenon. Choosing informative, discriminating and independent features is a crucial element of effective algorithms in pattern recognition, classification and regression. Features are usually numeric, but structural features such as strings and graphs are used in syntactic pattern recognition. The concept of "feature" is related to that of explanatory variable used in statistical techniques such as linear regression.

In statistics, the k-nearest neighbors algorithm (k-NN) is a non-parametric supervised learning method first developed by Evelyn Fix and Joseph Hodges in 1951, and later expanded by Thomas Cover. It is used for classification and regression. In both cases, the input consists of the k closest training examples in a data set. The output depends on whether k-NN is used for classification or regression:

In computer vision and image processing, a feature is a piece of information about the content of an image; typically about whether a certain region of the image has certain properties. Features may be specific structures in the image such as points, edges or objects. Features may also be the result of a general neighborhood operation or feature detection applied to the image. Other examples of features are related to motion in image sequences, or to shapes defined in terms of curves or boundaries between different image regions.

<span class="mw-page-title-main">Scree plot</span> Diagnostic plot

In multivariate statistics, a scree plot is a line plot of the eigenvalues of factors or principal components in an analysis. The scree plot is used to determine the number of factors to retain in an exploratory factor analysis (FA) or principal components to keep in a principal component analysis (PCA). The procedure of finding statistically significant factors or components using a scree plot is also known as a scree test. Raymond B. Cattell introduced the scree plot in 1966.

In mathematics, a graph partition is the reduction of a graph to a smaller graph by partitioning its set of nodes into mutually exclusive groups. Edges of the original graph that cross between the groups will produce edges in the partitioned graph. If the number of resulting edges is small compared to the original graph, then the partitioned graph may be better suited for analysis and problem-solving than the original. Finding a partition that simplifies graph analysis is a hard problem, but one that has applications to scientific computing, VLSI circuit design, and task scheduling in multiprocessor computers, among others. Recently, the graph partition problem has gained importance due to its application for clustering and detection of cliques in social, pathological and biological networks. For a survey on recent trends in computational methods and applications see Buluc et al. (2013). Two common examples of graph partitioning are minimum cut and maximum cut problems.

In statistics, signal processing, and time series analysis, a sinusoidal model is used to approximate a sequence Yi to a sine function:

<span class="mw-page-title-main">Plot (graphics)</span> Graphical technique for data sets

A plot is a graphical technique for representing a data set, usually as a graph showing the relationship between two or more variables. The plot can be drawn by hand or by a computer. In the past, sometimes mechanical or electronic plotters were used. Graphs are a visual representation of the relationship between variables, which are very useful for humans who can then quickly derive an understanding which may not have come from lists of values. Given a scale or ruler, graphs can also be used to read off the value of an unknown variable plotted as a function of a known one, but this can also be done with data presented in tabular form. Graphs of functions are used in mathematics, sciences, engineering, technology, finance, and other areas.

Determining the number of clusters in a data set, a quantity often labelled k as in the k-means algorithm, is a frequent problem in data clustering, and is a distinct issue from the process of actually solving the clustering problem.

The principal curvature-based region detector, also called PCBR is a feature detector used in the fields of computer vision and image analysis. Specifically the PCBR detector is designed for object recognition applications.

<span class="mw-page-title-main">HeuristicLab</span> Software environment

HeuristicLab is a software environment for heuristic and evolutionary algorithms, developed by members of the Heuristic and Evolutionary Algorithm Laboratory (HEAL) at the University of Applied Sciences Upper Austria, in Hagenberg im Mühlkreis. HeuristicLab has a strong focus on providing a graphical user interface so that users are not required to have comprehensive programming skills to adjust and extend the algorithms for a particular problem. In HeuristicLab algorithms are represented as operator graphs and changing or rearranging operators can be done by drag-and-drop without actually writing code. The software thereby tries to shift algorithm development capability from the software engineer to the user and practitioner. Developers can still extend the functionality on code level and can use HeuristicLab's plug-in mechanism that allows them to integrate custom algorithms, solution representations or optimization problems.

<span class="mw-page-title-main">Elbow method (clustering)</span> Heuristic used in computer science

In cluster analysis, the elbow method is a heuristic used in determining the number of clusters in a data set. The method consists of plotting the explained variation as a function of the number of clusters and picking the elbow of the curve as the number of clusters to use. The same method can be used to choose the number of parameters in other data-driven models, such as the number of principal components to describe a data set.

The following outline is provided as an overview of and topical guide to machine learning:

References

  1. ↑ Terrell, John Alan (1913). An Experimental Investigation of a New System for Automatically Regulating the Voltage of an Alternating Current Circuit. Rensselaer Polytechnic Institute. p.  10. ... enables one to tell how near the "knee" of the curve the iron is ...
  2. ↑ NACA Wartime Report. L. National Advisory Committee for Aeronautics. 1943. p.  21. ... the knee of the curve lies in the region of the critical load ...
  3. ↑ Worthing & Geffner 1943, Preface.
  4. 1 2 Kiokemeister, Fred L. (1949). "An Analysis of Functions Describing Experimental Data". Psychophysical Research: 5.
  5. ↑ Thomas & Sheldon 1999, p. 18.
  6. ↑ Satopaa, Ville; Albrecht, Jeannie; Irwin, David; Raghavan, Barath (June 2011). "Finding a "Kneedle" in a Haystack: Detecting Knee Points in System Behavior". 2011 31st International Conference on Distributed Computing Systems Workshops. pp. 166–171. doi:10.1109/ICDCSW.2011.20. ISBN   978-1-4577-0384-3.
  7. ↑ Kleine, Daniel (20 January 2023). "Detecting knee- / elbow points in a graph". Medium.
  8. ↑ "kneepointDetection: Fits 2 regression lines to data to estimate the knee (or... in SamSPECTRAL: Identifies cell population in flow cytometry data". rdrr.io.
  9. ↑ "SamSPECTRAL". Bioconductor.
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.