Linear combination

Last updated

In mathematics, a linear combination or superposition is an expression constructed from a set of terms by multiplying each term by a constant and adding the results (e.g. a linear combination of x and y would be any expression of the form ax + by, where a and b are constants). [1] [2] [3] [4] The concept of linear combinations is central to linear algebra and related fields of mathematics. Most of this article deals with linear combinations in the context of a vector space over a field, with some generalizations given at the end of the article.

Contents

Definition

Let V be a vector space over the field K. As usual, we call elements of V vectors and call elements of K scalars . If v1,...,vn are vectors and a1,...,an are scalars, then the linear combination of those vectors with those scalars as coefficients is

There is some ambiguity in the use of the term "linear combination" as to whether it refers to the expression or to its value. In most cases the value is emphasized, as in the assertion "the set of all linear combinations of v1,...,vn always forms a subspace". However, one could also say "two different linear combinations can have the same value" in which case the reference is to the expression. The subtle difference between these uses is the essence of the notion of linear dependence: a family F of vectors is linearly independent precisely if any linear combination of the vectors in F (as value) is uniquely so (as expression). In any case, even when viewed as expressions, all that matters about a linear combination is the coefficient of each vi; trivial modifications such as permuting the terms or adding terms with zero coefficient do not produce distinct linear combinations.

In a given situation, K and V may be specified explicitly, or they may be obvious from context. In that case, we often speak of a linear combination of the vectorsv1,...,vn, with the coefficients unspecified (except that they must belong to K). Or, if S is a subset of V, we may speak of a linear combination of vectors in S, where both the coefficients and the vectors are unspecified, except that the vectors must belong to the set S (and the coefficients must belong to K). Finally, we may speak simply of a linear combination, where nothing is specified (except that the vectors must belong to V and the coefficients must belong to K); in this case one is probably referring to the expression, since every vector in V is certainly the value of some linear combination.

Note that by definition, a linear combination involves only finitely many vectors (except as described in the § Generalizations section. However, the set S that the vectors are taken from (if one is mentioned) can still be infinite; each individual linear combination will only involve finitely many vectors. Also, there is no reason that n cannot be zero; in that case, we declare by convention that the result of the linear combination is the zero vector in V.

Examples and counterexamples

Euclidean vectors

Let the field K be the set R of real numbers, and let the vector space V be the Euclidean space R3. Consider the vectors e1 = (1,0,0), e2 = (0,1,0) and e3 = (0,0,1). Then any vector in R3 is a linear combination of e1, e2, and e3.

To see that this is so, take an arbitrary vector (a1,a2,a3) in R3, and write:

Functions

Let K be the set C of all complex numbers, and let V be the set CC(R) of all continuous functions from the real line R to the complex plane C. Consider the vectors (functions) f and g defined by f(t) := eit and g(t) := eit. (Here, e is the base of the natural logarithm, about 2.71828..., and i is the imaginary unit, a square root of −1.) Some linear combinations of f and g are:

On the other hand, the constant function 3 is not a linear combination of f and g. To see this, suppose that 3 could be written as a linear combination of eit and eit. This means that there would exist complex scalars a and b such that aeit + beit = 3 for all real numbers t. Setting t = 0 and t = π gives the equations a + b = 3 and a + b = −3, and clearly this cannot happen. See Euler's identity.

Polynomials

Let K be R, C, or any field, and let V be the set P of all polynomials with coefficients taken from the field K. Consider the vectors (polynomials) p1 := 1, p2 := x + 1, and p3 := x2 + x + 1.

Is the polynomial x2  1 a linear combination of p1, p2, and p3? To find out, consider an arbitrary linear combination of these vectors and try to see when it equals the desired vector x2  1. Picking arbitrary coefficients a1, a2, and a3, we want

Multiplying the polynomials out, this means

and collecting like powers of x, we get

Two polynomials are equal if and only if their corresponding coefficients are equal, so we can conclude

This system of linear equations can easily be solved. First, the first equation simply says that a3 is 1. Knowing that, we can solve the second equation for a2, which comes out to −1. Finally, the last equation tells us that a1 is also −1. Therefore, the only possible way to get a linear combination is with these coefficients. Indeed,

so x2  1 is a linear combination of p1, p2, and p3.

On the other hand, what about the polynomial x3  1? If we try to make this vector a linear combination of p1, p2, and p3, then following the same process as before, we get the equation

However, when we set corresponding coefficients equal in this case, the equation for x3 is

which is always false. Therefore, there is no way for this to work, and x3  1 is not a linear combination of p1, p2, and p3.

The linear span

Take an arbitrary field K, an arbitrary vector space V, and let v1,...,vn be vectors (in V). It is interesting to consider the set of all linear combinations of these vectors. This set is called the linear span (or just span) of the vectors, say S = {v1, ..., vn}. We write the span of S as span(S) [5] [6] or sp(S):

Linear independence

Suppose that, for some sets of vectors v1,...,vn, a single vector can be written in two different ways as a linear combination of them:

This is equivalent, by subtracting these (), to saying a non-trivial combination is zero: [7] [8]

If that is possible, then v1,...,vn are called linearly dependent ; otherwise, they are linearly independent. Similarly, we can speak of linear dependence or independence of an arbitrary set S of vectors.

If S is linearly independent and the span of S equals V, then S is a basis for V.

Affine, conical, and convex combinations

By restricting the coefficients used in linear combinations, one can define the related concepts of affine combination, conical combination, and convex combination, and the associated notions of sets closed under these operations.

Type of combinationRestrictions on coefficientsName of setModel space
Linear combinationno restrictions Vector subspace
Affine combination Affine subspace Affine hyperplane
Conical combination Convex cone Quadrant, octant, or orthant
Convex combination and Convex set Simplex

Because these are more restricted operations, more subsets will be closed under them, so affine subsets, convex cones, and convex sets are generalizations of vector subspaces: a vector subspace is also an affine subspace, a convex cone, and a convex set, but a convex set need not be a vector subspace, affine, or a convex cone.

These concepts often arise when one can take certain linear combinations of objects, but not any: for example, probability distributions are closed under convex combination (they form a convex set), but not conical or affine combinations (or linear), and positive measures are closed under conical combination but not affine or linear – hence one defines signed measures as the linear closure.

Linear and affine combinations can be defined over any field (or ring), but conical and convex combination require a notion of "positive", and hence can only be defined over an ordered field (or ordered ring), generally the real numbers.

If one allows only scalar multiplication, not addition, one obtains a (not necessarily convex) cone; one often restricts the definition to only allowing multiplication by positive scalars.

All of these concepts are usually defined as subsets of an ambient vector space (except for affine spaces, which are also considered as "vector spaces forgetting the origin"), rather than being axiomatized independently.

Operad theory

More abstractly, in the language of operad theory, one can consider vector spaces to be algebras over the operad (the infinite direct sum, so only finitely many terms are non-zero; this corresponds to only taking finite sums), which parametrizes linear combinations: the vector for instance corresponds to the linear combination . Similarly, one can consider affine combinations, conical combinations, and convex combinations to correspond to the sub-operads where the terms sum to 1, the terms are all non-negative, or both, respectively. Graphically, these are the infinite affine hyperplane, the infinite hyper-octant, and the infinite simplex. This formalizes what is meant by being or the standard simplex being model spaces, and such observations as that every bounded convex polytope is the image of a simplex. Here suboperads correspond to more restricted operations and thus more general theories.

From this point of view, we can think of linear combinations as the most general sort of operation on a vector space – saying that a vector space is an algebra over the operad of linear combinations is precisely the statement that all possible algebraic operations in a vector space are linear combinations.

The basic operations of addition and scalar multiplication, together with the existence of an additive identity and additive inverses, cannot be combined in any more complicated way than the generic linear combination: the basic operations are a generating set for the operad of all linear combinations.

Ultimately, this fact lies at the heart of the usefulness of linear combinations in the study of vector spaces.

Generalizations

If V is a topological vector space, then there may be a way to make sense of certain infinite linear combinations, using the topology of V. For example, we might be able to speak of a1v1 + a2v2 + a3v3 + ⋯, going on forever. Such infinite linear combinations do not always make sense; we call them convergent when they do. Allowing more linear combinations in this case can also lead to a different concept of span, linear independence, and basis. The articles on the various flavors of topological vector spaces go into more detail about these.

If K is a commutative ring instead of a field, then everything that has been said above about linear combinations generalizes to this case without change. The only difference is that we call spaces like this V modules instead of vector spaces. If K is a noncommutative ring, then the concept still generalizes, with one caveat: since modules over noncommutative rings come in left and right versions, our linear combinations may also come in either of these versions, whatever is appropriate for the given module. This is simply a matter of doing scalar multiplication on the correct side.

A more complicated twist comes when V is a bimodule over two rings, KL and KR. In that case, the most general linear combination looks like

where a1,...,an belong to KL, b1,...,bn belong to KR, and v1,…,vn belong to V.

See also

Citations

  1. Strang (2016) p. 3, § 1.1
  2. Lay, Lay & McDonald (2016) p. 28, ch. 1
  3. Axler (2015) p. 28, § 2.3
  4. nLab (2015) Linear combinations.
  5. Axler (2015) pp. 29-30, §§ 2.5, 2.8
  6. Katznelson & Katznelson (2008) p. 9, § 1.2.3
  7. Axler (2015) pp. 32-33, §§ 2.17, 2.19
  8. Katznelson & Katznelson (2008) p. 14, § 1.3.2

Related Research Articles

In mathematics, and more specifically in linear algebra, a linear map is a mapping between two vector spaces that preserves the operations of vector addition and scalar multiplication. The same names and the same definition are also used for the more general case of modules over a ring; see Module homomorphism.

<span class="mw-page-title-main">Basis (linear algebra)</span> Set of vectors used to define coordinates

In mathematics, a set B of vectors in a vector space V is called a basis if every element of V may be written in a unique way as a finite linear combination of elements of B. The coefficients of this linear combination are referred to as components or coordinates of the vector with respect to B. The elements of a basis are called basis vectors.

<span class="mw-page-title-main">Linear algebra</span> Branch of mathematics

Linear algebra is the branch of mathematics concerning linear equations such as:

In linear algebra, the rank of a matrix A is the dimension of the vector space generated by its columns. This corresponds to the maximal number of linearly independent columns of A. This, in turn, is identical to the dimension of the vector space spanned by its rows. Rank is thus a measure of the "nondegenerateness" of the system of linear equations and linear transformation encoded by A. There are multiple equivalent definitions of rank. A matrix's rank is one of its most fundamental characteristics.

<span class="mw-page-title-main">Vector space</span> Algebraic structure in linear algebra

In mathematics and physics, a vector space is a set whose elements, often called vectors, can be added together and multiplied ("scaled") by numbers called scalars. Scalars are often real numbers, but can be complex numbers or, more generally, elements of any field. The operations of vector addition and scalar multiplication must satisfy certain requirements, called vector axioms. Real vector spaces and complex vector spaces are kinds of vector spaces based on different kinds of scalars: real numbers and complex numbers.

<span class="mw-page-title-main">Affine transformation</span> Geometric transformation that preserves lines but not angles nor the origin

In Euclidean geometry, an affine transformation or affinity is a geometric transformation that preserves lines and parallelism, but not necessarily Euclidean distances and angles.

<span class="mw-page-title-main">Linear span</span> In linear algebra, generated subspace

In mathematics, the linear span (also called the linear hull or just span) of a set S of vectors (from a vector space), denoted span(S), is defined as the set of all linear combinations of the vectors in S. For example, two linearly independent vectors span a plane. The linear span can be characterized either as the intersection of all linear subspaces that contain S, or as the smallest subspace containing S. The linear span of a set of vectors is therefore a vector space itself. Spans can be generalized to matroids and modules.

<span class="mw-page-title-main">Linear subspace</span> In mathematics, vector subspace

In mathematics, and more specifically in linear algebra, a linear subspace or vector subspace is a vector space that is a subset of some larger vector space. A linear subspace is usually simply called a subspace when the context serves to distinguish it from other types of subspaces.

<span class="mw-page-title-main">Row and column spaces</span> Vector spaces associated to a matrix

In linear algebra, the column space of a matrix A is the span of its column vectors. The column space of a matrix is the image or range of the corresponding matrix transformation.

<span class="mw-page-title-main">Linear independence</span> Vectors whose linear combinations are nonzero

In the theory of vector spaces, a set of vectors is said to be linearly independent if there exists no nontrivial linear combination of the vectors that equals the zero vector. If such a linear combination exists, then the vectors are said to be linearly dependent. These concepts are central to the definition of dimension.

In mathematics, a linear form is a linear map from a vector space to its field of scalars.

<span class="mw-page-title-main">Affine space</span> Euclidean space without distance and angles

In mathematics, an affine space is a geometric structure that generalizes some of the properties of Euclidean spaces in such a way that these are independent of the concepts of distance and measure of angles, keeping only the properties related to parallelism and ratio of lengths for parallel line segments. Affine space is the setting for affine geometry.

In mathematics, an affine combination of x1, ..., xn is a linear combination

<span class="mw-page-title-main">Convex combination</span> Linear combination of points where all coefficients are non-negative and sum to 1

In convex geometry and vector algebra, a convex combination is a linear combination of points where all coefficients are non-negative and sum to 1. In other words, the operation is equivalent to a standard weighted average, but whose weights are expressed as a percent of the total weight, instead of as a fraction of the count of the weights as in a standard weighted average.

<span class="mw-page-title-main">Real coordinate space</span> Space formed by the n-tuples of real numbers

In mathematics, the real coordinate space or real coordinate n-space, of dimension n, denoted Rn or , is the set of all ordered n-tuples of real numbers, that is the set of all sequences of n real numbers, also known as coordinate vectors. Special cases are called the real lineR1, the real coordinate planeR2, and the real coordinate three-dimensional spaceR3. With component-wise addition and scalar multiplication, it is a real vector space.

In mathematics, an operad is a structure that consists of abstract operations, each one having a fixed finite number of inputs (arguments) and one output, as well as a specification of how to compose these operations. Given an operad , one defines an algebra over to be a set together with concrete operations on this set which behave just like the abstract operations of . For instance, there is a Lie operad such that the algebras over are precisely the Lie algebras; in a sense abstractly encodes the operations that are common to all Lie algebras. An operad is to its algebras as a group is to its group representations.

<span class="mw-page-title-main">Convex cone</span> Mathematical set closed under positive linear combinations

In linear algebra, a cone—sometimes called a linear cone for distinguishing it from other sorts of cones—is a subset of a vector space that is closed under positive scalar multiplication; that is, C is a cone if implies for every positive scalar s. A cone need not be convex, or even look like a cone in Euclidean space.

Affine geometry, broadly speaking, is the study of the geometrical properties of lines, planes, and their higher dimensional analogs, in which a notion of "parallel" is retained, but no metrical notions of distance or angle are. Affine spaces differ from linear spaces in that they do not have a distinguished choice of origin. So, in the words of Marcel Berger, "An affine space is nothing more than a vector space whose origin we try to forget about, by adding translations to the linear maps." Accordingly, a complex affine space, that is an affine space over the complex numbers, is like a complex vector space, but without a distinguished point to serve as the origin.

Given a finite number of vectors in a real vector space, a conical combination, conical sum, or weighted sum of these vectors is a vector of the form

This glossary of linear algebra is a list of definitions and terms relevant to the field of linear algebra, the branch of mathematics concerned with linear equations and their representations as vector spaces.

References

Textbook

Web