Understanding Bias in Linear Least Squares Regression with Simple Test Data

Summary

This article explains why linear least squares regression can appear biased when applied to simple test data, focusing on the underlying assumptions of ordinary least squares (OLS) models. The author discusses how OLS assumes independence between X and error terms, and that X is known with certainty. The analysis shows that when data points perfectly fit a regression line with zero error terms, principal component analysis would align with the regression line direction, matching intuitive expectations about the relationship between variables.