Partially linear model

Last updated

A partially linear model is a form of semiparametric model, since it contains parametric and nonparametric elements. Application of the least squares estimators is available to partially linear model, if the hypothesis of the known of nonparametric element is valid. Partially linear equations were first used in the analysis of the relationship between temperature and usage of electricity by Engle, Granger, Rice and Weiss (1986). [1] Typical application of partially linear model in the field of Microeconomics is presented by Tripathi in the case of profitability of firm's production in 1997. Also, partially linear model applied successfully in some other academic field. In 1994, Zeger and Diggle introduced partially linear model into biometrics. In environmental science, Parda-Sanchez et al. used partially linear model to analysis collected data in 2000. So far, partially linear model was optimized in many other statistic methods. In 1988, Robinson applied Nadaraya-Waston kernel estimator to test the nonparametric element to build a least-squares estimator  After that, in 1997, local linear method was found by Truong.

Contents

Partially linear model

Regression,2009-12-01.jpg

Synopsis

Algebra equation

The algebra expression of partially linear model is written as:

[2]

Equation components outline

and : Vectors of explanatory variables. Independently random or fixed distributed variables.

: To be measured Parameter.

: The random error in statistics with 0 mean.

: To be measured part in partially linear model.

Assumption [2]

Wolfgang, Hua Liang and Jiti Gao consider the assumptions and remarks of partially linear model under fixed and random design conditions.

When randomly distributed, introduce

and (1)

is smaller than positive infinity when t value between 0 and 1, and the sum of covariance of  is positive. The random errors μ are independent of ,

When and Ti are fixed distributed, valued between 0 and 1, and satisfies , where factor i values between 1 and n, and factor j value between 1 and p, Error factor satisfies, .

The least square (LS) estimators [2]

The precondition of application of the least squares estimators is the existence of nonparametric component, and running at random distributed and fixed distributed cases.

Engle, Granger, Rice and Weiss's (1986) smoothing model should be first introduced, before applying the least squares estimators. The algebra function of their model is expressed as (2).

Wolfgang, Liang and Gao (1988) make an assumption that the pair (ß,g) satisfies (3).

This means that for all , .

So, and .

Under random distributed case, Wolfgang, Hua Liang and Jiti Gao assume that for all 1 ≤ i ≤ n, (4)

so, , due to the fact that is a positive number, as proved by function (1). So, established for all 1≤i≤n and j equals to 1 and 2 when .

Under fixed distributed case, By parameterizing factor  from smoothing model (2) as where .

By making same assumption as (4), which follows from assumption (1), and under the fact of .

Assuming factors (i here are positive integers) satisfies and establish positive weight functions . Any estimators of , for every , we have . By applying LS criterion, the LS estimator of . The nonparametric estimator of   is expressed as . So, When the random errors are identically distributed, the estimators of variance  is expressed as, .

History and applications of partially linear model

The real-world application of partially linear model was first considered for analyzing data by Engle, Granger, Rice and Weiss in 1986. [2]

In their point of view, the relevance between temperature and the consumption of electricity cannot be expressed in a linear model, because there are massive of confounding factors, such as average income, goods price, consumer purchase ability and some other economic activities. Some of the factors are relevance with each other and might influence the observed result. Therefore, they introduced partially linear model, which contained both with parametric and nonparametric factors. The partially linear model enables and simplifies the linear transformation of data (Engle, Granger, Rice and Weiss, 1986). They also applied the smoothing spline technique for their research.

There was a case of application of partially linear model in biometrics by Zeger and Diggle in 1994. The research objective of their paper is the evolution period cycle of CD4 cell amounts in HIV (Human immune-deficiency virus) seroconverters (Zeger and Diggle, 1994). [3] CD4 cell plays a significant role in immune function in human body. Zeger and Diggle aimed to assess the proceed of disease by measuring the changing amount of CD4 cells. The number of CD4 cell is associated with body age and smoking behavior and so on. To clear the group of observation data in their experiment, Zeger and Diggle applied partially linear model for their work. Partially linear model primarily contributes to the estimation of average loss time of CD4 cells and adjusts the time dependence of some other covariables in order to simplify the proceed of data comparison, and also, the partially linear model characterizes the deviation of typical curve for their observed group to estimate the progression curve of the changing amount of CD4 cell. The deviation, granted by partially linear model, potentially helps to recognize the observed targets who had a slow progression on the amounting change of CD4 cells. 

In 1999, Schmalensee and Stoker (1999) have used partially linear model in the field of economics. The independent variable of their research is the demand for gasoline in The United States. The primary research target in their paper is the relationship between gasoline consumption and long-run income elasticity in the U.S. Similarly, there are also massive of confounding variables, which might mutually affect. Hence, Schmalemsee and Stoker chose to deal with the issues of linear transformation of data between parametric and nonparametric by applying partially linear model. [4]

In the field of environment science, Prada-Sanchez used partially linear model to predict the sulfur dioxide pollution in 2000 (Prada-Sanchez, 2000), [5] and in the next year, Lin and Carroll applied partially linear model for clustered data (Lin and Carroall, 2001). [6]

Development of partially linear model

According to Liang's paper in 2010 (Liang, 2010), The smoothing spline technique was introduced in partially linear model by Engle, Heckman and Rice in 1986. After that, Robinson found an available LS estimator for nonparametric factors in partially linear model in 1988. At the same year, profile LS method was recommended by Speckman. [7]

Other econometrics tools in partially linear model

Kernel regression also was introduced in partially linear model. The local constant method, which is developed by Speckman, and local linear techniques, which was found by Hamilton and Truong in 1997 and was revised by Opsomer and Ruppert in 1997, are all included in kernel regression. Green et al., Opsomer and Ruppert found that one of the significant characteristic of kernel-based methods is that under-smoothing has been taken in order to find root-n estimator of beta. However, Speckman's research in 1988 and Severini's and Staniswalis's research in 1994 proved that those restriction might be canceled.

Bandwidth selection in partially linear model [7]

Bandwidth selection in partially linear model is a confusing issue. Liang addressed a possible solution for this bandwidth selection in his literature by applying profile-kernel based method and backfitting methods. Also the necessity of undersmoothing for backfitting method and the reason why profile-kernel based method can work out the optimal bandwidth selection were justified by Liang. The general computation strategy is applied in Liang's literature for estimating nonparametric function. Moreover, the penalized spline method for partially linear models and intensive simulation experiments were introduced to discover the numerical feature of the penalized spline method, profile and backfitting methods.

Kernel-based profile and backfitting method [7]

By introducing

Following with

The intuitive estimator of ß can be defined as the LS estimator after appropriately estimating and .

Then, for all random vector variable , assume is a kernel regression estimator of . Let . For example, . Denote X,g and T similarly. Let . So

The profile-kernel based estimators solves,

where are kernel estimators of mx and my.

The penalized spline method [7]

The penalized spline method was developed by Eilers and Marx in 1996. Ruppert and Carroll in 2000 and Brumback, Ruppert and Wand in 1999 employed this method in LME framework.

Assuming function can be estimated by

where is an integer, and are fixed knots, Denote Consider . The penalized spline estimator is defined as follow

Where is a smoothing parameter.

As Brumback et al. mentioned in 1999, [8] the estimator is same as the estimator of based on LME model.

,

where ,

Where , and . The matrix shows the penalized spline smoother for up above framework.

Related Research Articles

<span class="mw-page-title-main">Dirac delta function</span> Generalized function whose value is zero everywhere except at zero

In mathematical analysis, the Dirac delta function, also known as the unit impulse, is a generalized function on the real numbers, whose value is zero everywhere except at zero, and whose integral over the entire real line is equal to one. Since there is no function having this property, modelling the delta "function" rigorously involves the use of limits or, as is common in mathematics, measure theory and the theory of distributions.

<span class="mw-page-title-main">Least squares</span> Approximation method in statistics

The method of least squares is a parameter estimation method in regression analysis based on minimizing the sum of the squares of the residuals made in the results of each individual equation.

In physics, a Langevin equation is a stochastic differential equation describing how a system evolves when subjected to a combination of deterministic and fluctuating ("random") forces. The dependent variables in a Langevin equation typically are collective (macroscopic) variables changing only slowly in comparison to the other (microscopic) variables of the system. The fast (microscopic) variables are responsible for the stochastic nature of the Langevin equation. One application is to Brownian motion, which models the fluctuating motion of a small particle in a fluid.

<span class="mw-page-title-main">Fokker–Planck equation</span> Partial differential equation

In statistical mechanics and information theory, the Fokker–Planck equation is a partial differential equation that describes the time evolution of the probability density function of the velocity of a particle under the influence of drag forces and random forces, as in Brownian motion. The equation can be generalized to other observables as well. The Fokker-Planck equation has multiple applications in information theory, graph theory, data science, finance, economics etc.

In physics, the Hamilton–Jacobi equation, named after William Rowan Hamilton and Carl Gustav Jacob Jacobi, is an alternative formulation of classical mechanics, equivalent to other formulations such as Newton's laws of motion, Lagrangian mechanics and Hamiltonian mechanics.

<span class="mw-page-title-main">Dirac comb</span> Periodic distribution ("function") of "point-mass" Dirac delta sampling

In mathematics, a Dirac comb is a periodic function with the formula for some given period . Here t is a real variable and the sum extends over all integers k. The Dirac delta function and the Dirac comb are tempered distributions. The graph of the function resembles a comb, hence its name and the use of the comb-like Cyrillic letter sha (Ш) to denote the function.

In statistics, the theory of minimum norm quadratic unbiased estimation (MINQUE) was developed by C. R. Rao. MINQUE is a theory alongside other estimation methods in estimation theory, such as the method of moments or maximum likelihood estimation. Similar to the theory of best linear unbiased estimation, MINQUE is specifically concerned with linear regression models. The method was originally conceived to estimate heteroscedastic error variance in multiple linear regression. MINQUE estimators also provide an alternative to maximum likelihood estimators or restricted maximum likelihood estimators for variance components in mixed effects models. MINQUE estimators are quadratic forms of the response variable and are used to estimate a linear function of the variances.

<span class="mw-page-title-main">Simple linear regression</span> Linear regression model with a single explanatory variable

In statistics, simple linear regression (SLR) is a linear regression model with a single explanatory variable. That is, it concerns two-dimensional sample points with one independent variable and one dependent variable and finds a linear function that, as accurately as possible, predicts the dependent variable values as a function of the independent variable. The adjective simple refers to the fact that the outcome variable is related to a single predictor.

In physics and fluid mechanics, a Blasius boundary layer describes the steady two-dimensional laminar boundary layer that forms on a semi-infinite plate which is held parallel to a constant unidirectional flow. Falkner and Skan later generalized Blasius' solution to wedge flow, i.e. flows in which the plate is not parallel to the flow.

In mathematics, and more specifically in partial differential equations, Duhamel's principle is a general method for obtaining solutions to inhomogeneous linear evolution equations like the heat equation, wave equation, and vibrating plate equation. It is named after Jean-Marie Duhamel who first applied the principle to the inhomogeneous heat equation that models, for instance, the distribution of heat in a thin plate which is heated from beneath. For linear evolution equations without spatial dependency, such as a harmonic oscillator, Duhamel's principle reduces to the method of variation of parameters technique for solving linear inhomogeneous ordinary differential equations. It is also an indispensable tool in the study of nonlinear partial differential equations such as the Navier–Stokes equations and nonlinear Schrödinger equation where one treats the nonlinearity as an inhomogeneity.

In statistics, semiparametric regression includes regression models that combine parametric and nonparametric models. They are often used in situations where the fully nonparametric model may not perform well or when the researcher wants to use a parametric model but the functional form with respect to a subset of the regressors or the density of the errors is not known. Semiparametric regression models are a particular type of semiparametric modelling and, since semiparametric models contain a parametric component, they rely on parametric assumptions and may be misspecified and inconsistent, just like a fully parametric model.

Linear dynamical systems are dynamical systems whose evolution functions are linear. While dynamical systems, in general, do not have closed-form solutions, linear dynamical systems can be solved exactly, and they have a rich set of mathematical properties. Linear systems can also be used to understand the qualitative behavior of general dynamical systems, by calculating the equilibrium points of the system and approximating it as a linear system around each such point.

In many-body theory, the term Green's function is sometimes used interchangeably with correlation function, but refers specifically to correlators of field operators or creation and annihilation operators.

In optics, the term soliton is used to refer to any optical field that does not change during propagation because of a delicate balance between nonlinear and dispersive effects in the medium. There are two main kinds of solitons:

<span class="mw-page-title-main">Quantile regression</span> Statistics concept

Quantile regression is a type of regression analysis used in statistics and econometrics. Whereas the method of least squares estimates the conditional mean of the response variable across values of the predictor variables, quantile regression estimates the conditional median of the response variable. Quantile regression is an extension of linear regression used when the conditions of linear regression are not met.

<span class="mw-page-title-main">Errors-in-variables models</span> Regression models accounting for possible errors in independent variables

In statistics, errors-in-variables models or measurement error models are regression models that account for measurement errors in the independent variables. In contrast, standard regression models assume that those regressors have been measured exactly, or observed without error; as such, those models account only for errors in the dependent variables, or responses.

<span class="mw-page-title-main">Peregrine soliton</span> Analytic solution of the nonlinear Schrödinger equation

The Peregrine soliton is an analytic solution of the nonlinear Schrödinger equation. This solution was proposed in 1983 by Howell Peregrine, researcher at the mathematics department of the University of Bristol.

In numerical analysis, the local linearization (LL) method is a general strategy for designing numerical integrators for differential equations based on a local (piecewise) linearization of the given equation on consecutive time intervals. The numerical integrators are then iteratively defined as the solution of the resulting piecewise linear equation at the end of each consecutive interval. The LL method has been developed for a variety of equations such as the ordinary, delayed, random and stochastic differential equations. The LL integrators are key component in the implementation of inference methods for the estimation of unknown parameters and unobserved variables of differential equations given time series of observations. The LL schemes are ideals to deal with complex models in a variety of fields as neuroscience, finance, forestry management, control engineering, mathematical statistics, etc.

In statistics, the Innovation Method provides an estimator for the parameters of stochastic differential equations given a time series of observations of the state variables. In the framework of continuous-discrete state space models, the innovation estimator is obtained by maximizing the log-likelihood of the corresponding discrete-time innovation process with respect to the parameters. The innovation estimator can be classified as a M-estimator, a quasi-maximum likelihood estimator or a prediction error estimator depending on the inferential considerations that want to be emphasized. The innovation method is a system identification technique for developing mathematical models of dynamical systems from measured data and for the optimal design of experiments.

Ball divergence is a non-parametric two-sample statistical test method in metric spaces. It measures the difference between two population probability distributions by integrating the difference over all balls in the space. Therefore, its value is zero if and only if the two probability measures are the same. Similar to common non-parametric test methods, ball divergence calculates the p-value through permutation tests.

References

  1. Engle, Robert F.; Granger, C. W. J.; Rice, John; Weiss, Andrew (1986). "Semiparametric Estimates of the Relation Between Weather and Electricity Sales". Journal of the American Statistical Association. 81 (394): 310–20. doi:10.2307/2289218.
  2. 1 2 3 4 Hardle, Liang, JiTi, WolfGang, Hua, Gao (2000). Partially linear model. PHYSICA-VERLAG.{{cite book}}: CS1 maint: multiple names: authors list (link)
  3. Zeger, Scott L.; Diggle, Peter J. (1994). "Semiparametric Models for Longitudinal Data with Application to CD4 Cell Numbers in HIV Seroconverters". Biometrics. 50 (3): 689–699. doi:10.2307/2532783. ISSN   0006-341X. JSTOR   2532783. PMID   7981395.
  4. Schmalensee, Richard; Stoker, Thomas M. (1999). "Household Gasoline Demand in the United States" (PDF). Econometrica. 67 (3): 645–662. doi:10.1111/1468-0262.00041. hdl: 1721.1/50215 . ISSN   1468-0262.
  5. Prada‐Sánchez, J. M.; Febrero‐Bande, M.; Cotos‐Yáñez, T.; González‐Manteiga, W.; Bermúdez‐Cela, J. L.; Lucas‐Domínguez, T. (2000). "Prediction of SO2 pollution incidents near a power station using partially linear models and an historical matrix of predictor-response vectors". Environmetrics. 11 (2): 209–225. doi:10.1002/(SICI)1099-095X(200003/04)11:2<209::AID-ENV403>3.0.CO;2-Z. ISSN   1099-095X.
  6. Carroll, Raymond J.; Lin, Xihong (2001-12-01). "Semiparametric regression for clustered data". Biometrika. 88 (4): 1179–1185. doi:10.1093/biomet/88.4.1179. ISSN   0006-3444.
  7. 1 2 3 4 Liang, Hua (2006-02-10). "Estimation in Partially Linear Models and Numerical Comparisons". Computational Statistics & Data Analysis. 50 (3): 675–687. doi:10.1016/j.csda.2004.10.007. ISSN   0167-9473. PMC   2824448 . PMID   20174596.
  8. Brumback, Babette A.; Ruppert, David; Wand, M. P. (1999). "Variable Selection and Function Estimation in Additive Nonparametric Regression Using a Data-Based Prior: Comment". Journal of the American Statistical Association. 94 (447): 794–797. doi:10.2307/2669991. ISSN   0162-1459. JSTOR   2669991.