Linear Regression and Residuals

Artificial Intelligence and Data Science · CHAPTER 2 / 5

Level: introductory undergraduate · Study session: 15–20 minutes including practice.

Learning outcome: Build a baseline and interpret prediction errors.

Before you begin: Complete earlier chapters in this discipline; revise algebra and units as needed.

1. Understand the idea

Linear regression models a numeric target using a weighted combination of features. A residual is observed minus predicted. A low training error does not guarantee useful forecasts. Inspect residual patterns, compare against a simple baseline and evaluate on unseen observations.

2. Write the model

ŷ = b + wx; residual = y − ŷ

Define the symbols from the worked case before substituting values. Keep units consistent, record the assumptions and round only after the calculation.

3. Follow a worked example

For ŷ=2+3x, x=1,2,3 gives predictions 5,8,11. If observed values are 6,7,12, residuals are 1,−1,1 and MAE is 1.

4. Explore the model

Figure: calculated teaching cases; straight segments connect sampled points, not experimental observations. Read exact values below.

Illustrative scenario sweep — synthetic educational data
x	Observed y	Predicted y
1	6	5
2	7	8
3	12	11

Plot observed and predicted values, then compute each absolute residual. Add an outlier and compare MAE with squared error.

5. Reproduce it yourself

Copy the input cases into Excel or a notebook and name each quantity and unit.
Recalculate each result from the stated equation or rule. For qualitative cases, explain the state or partition decision instead.
Change one input, predict the direction of change, then calculate again.
Write one sentence explaining where the model stops being valid.

6. Check your understanding

What is the residual when observed=10 and predicted=12?

Reveal the answer and check your work

−2.

Compare your result, units and reasoning. If they differ, return to the equation and test the worked case first.

7. Common mistake

A fitted association does not by itself establish causation.

Further study and academic note

Explore the discipline reference and further learning resources. This is an original IIMEEE introductory teaching note with synthetic examples; it is not a peer-reviewed research result or a professional design specification. Figures were constructed from the listed teaching cases. Resource links support further study and do not imply endorsement.