RSS, MSE, RMSE, RSE, TSS, R2 and Adjusted R2

Yao Yao on September 29, 2014

This post is written by courtesy of:

以下假设 sample 有 $m$ 个 examples。

The Residual Sum of Squares (RSS) is the sum of the squared residuals

以下三个概念等价 (我无话可说):

  • RSS: Residual Sum of Squares
  • SSR: Sum of Squared Residuals
  • SSE: Sum of Squared Errors

The Mean Squared Error (MSE) is the mean of RSS

The Root Mean Squared Error (RMSE) is the square root of MSE

The Residual Standard Error (RSE) is the square root of $\frac{RSS}{\text{degrees of freedom}}$

where

  • $p$ is the number of predictors
    • i.e. $p+1$ is the number of right-hand-side variables, including the intercept, in a regression model
  • $m-p-1$ denotes the degrees of freedom.

where $\bar y$ is the sample mean.

Further we have $Var = \frac{TSS}{m - 1}$

$R^2$ and Adjusted $R^2$

Chain Reaction

当趋向 overfitting 时(比如 predictor 增多,模型变 flexible 时):

  • RSS ↓
    • MSE ↓
    • RMSE ↓
    • 如果是 predicator 增多,那么 RSE 无法断定是上升还是下降
  • TSS →
  • $R^2$ ↑
  • Adjusted $R^2$ 不好说(这正是 adjustment 的体现)


blog comments powered by Disqus