Yao Yao on September 29, 2014

This post is written by courtesy of:

## The Residual Sum of Squares (RSS) is the sum of the squared residuals

• RSS: Residual Sum of Squares
• SSR: Sum of Squared Residuals
• SSE: Sum of Squared Errors
$RSS = \sum_{i=1}^{m}{e_i^2} = \sum_{i=1}^{m}{(y_i - \hat{f}(x_i))^2}$

## The Mean Squared Error (MSE) is the mean of RSS

$MSE = \frac{RSS}{m}$

## The Root Mean Squared Error (RMSE) is the square root of MSE

$RMSE = \sqrt{MSE} = \sqrt{\frac{RSS}{m}}$

## The Residual Standard Error (RSE) is the square root of $\frac{RSS}{\text{degrees of freedom}}$

$RSE = \sqrt \frac{RSS}{m - p - 1}$

where

• $p$ is the number of predictors
• i.e. $p+1$ is the number of right-hand-side variables, including the intercept, in a regression model
• $m-p-1$ denotes the degrees of freedom.
$TSS = \sum_{i=1}^{m}{(y_i - \bar y)^2}$

where $\bar y$ is the sample mean.

Further we have $Var = \frac{TSS}{m - 1}$

## $R^2$ and Adjusted $R^2$

\begin{align} & R^2 = 1 - \frac{RSS}{TSS} \newline \text{Adjusted } & R^2 = 1 - \frac{RSS/(m-p-1)}{TSS/(m-1)} = 1 - \frac{m-1}{m-p-1} \frac{RSS}{TSS} \end{align}

## Chain Reaction

• $R^2$ ↑
• Adjusted $R^2$ 不好说（这正是 adjustment 的体现）