S-estimator

The goal of S-estimators is to have a simple high-breakdown regression estimator, which share the flexibility and nice asymptotic properties of M-estimators. The name "S-estimators" was chosen as they are based on estimators of scale.

We will consider estimators of scale defined by a function ${\displaystyle \rho }$, which satisfy

• R1 – ${\displaystyle \rho }$ is symmetric, continuously differentiable and ${\displaystyle \rho (0)=0}$.
• R2 – there exists ${\displaystyle c>0}$ such that ${\displaystyle \rho }$ is strictly increasing on ${\displaystyle [c,\infty ]}$

For any sample ${\displaystyle \{r_{1},...,r_{n}\}}$ of real numbers, we define the scale estimate ${\displaystyle s(r_{1},...,r_{n})}$ as the solution of

${\textstyle {\frac {1}{n}}\sum _{i=1}^{n}\rho (r_{i}/s)=K}$,

where ${\displaystyle K}$ is the expectation value of ${\displaystyle \rho }$ for a standard normal distribution. (If there are more solutions to the above equation, then we take the one with the smallest solution for s; if there is no solution, then we put ${\displaystyle s(r_{1},...,r_{n})=0}$ .)

Definition:

Let ${\displaystyle (x_{1},y_{1}),...,(x_{n},y_{n})}$ be a sample of regression data with p-dimensional ${\displaystyle x_{i}}$. For each vector ${\displaystyle \theta }$, we obtain residuals ${\displaystyle s(r_{1}(\theta ),...,r_{n}(\theta ))}$ by solving the equation of scale above, where ${\displaystyle \rho }$ satisfy R1 and R2. The S-estimator ${\displaystyle {\hat {\theta }}}$ is defined by

${\displaystyle {\hat {\theta }}=\min _{\theta }\,s(r_{1}(\theta ),...,r_{n}(\theta ))}$

and the final scale estimator ${\displaystyle {\hat {\sigma }}}$ is then

${\displaystyle {\hat {\sigma }}=s(r_{1}({\hat {\theta }}),...,r_{n}({\hat {\theta }}))}$. ^[1]

References

1. P. Rousseeuw and V. Yohai, Robust Regression by Means of S-estimators, from the book: Robust and nonlinear time series analysis, pages 256–272, 1984