Simple Linear Regression and Correlation

Lesson 15 of STA301 marks the transition from univariate analysis to bivariate statistical modeling, examining Simple Linear Regression and Pearson Product-Moment Correlation. In scientific inquiry, researchers frequently investigate relationships between two quantitative variables, designating one as the independent or explanatory variable X (regressor) and the other as the dependent or response variable Y (regressand). The initial diagnostic step is the Scatter Diagram, plotting pairs of observations (X_i, Y_i) on Cartesian axes to visually inspect linear patterns, clusters, and directional trends. The deterministic simple linear regression model is expressed as Y = a + bX, where a represents the Y-intercept and b denotes the regression slope—the expected change in Y per unit change in X. Using Carl Friedrich Gauss's Principle of Least Squares, which minimizes the sum of squared vertical residual errors sum(Y - Y-hat)^2, the Normal Equations are derived: sum(Y) = n*a + b*sum(X) and sum(X*Y) = a*sum(X) + b*sum(X^2). Solving these yield the computational formulas: b = [n*sum(XY) - sum(X)*sum(Y)] / [n*sum(X^2) - (sum X)^2] and a = Y-bar - b*X-bar. The precision of regression predictions is measured by the Standard Error of Estimate: s_yx = sqrt[sum(Y - Y-hat)^2 / (n - 2)], quantifying residual scatter around the fitted line. The lecture also explores Pearson's Correlation Coefficient r, measuring the strength and direction of linear association: r = [n*sum(XY) - sum(X)*sum(Y)] / sqrt{[n*sum(X^2) - (sum X)^2] * [n*sum(Y^2) - (sum Y)^2]}. Core properties of r are demonstrated: r is unitless, bounded strictly between -1 and +1, invariant under linear changes of origin and scale, and shares the same mathematical sign as regression slope b. Crucially, correlation measures association rather than causal dependency. In Virtual University midterm examinations, deriving normal equations, calculating slope coefficients, and interpreting correlation values are prominent computational questions. Diligent practice with sta301 handouts and solved papers ensures complete exam readiness. In preparing for Virtual University assessments in STA301 Statistics and Probability, students should thoroughly review past paper subjective questions and verify key formulas relating to Simple Linear Regression and Correlation.