Module II: Statistical Foundation of Financial Econometrics

To construct, estimate, and interpret quantitative asset pricing and risk models, the financial econometrician must master the foundational principles of mathematical statistics. Statistical inference bridges the divide between observable sample data and unobservable population parameters. This module covers the statistical pillars of financial econometrics: (1) core probability distributions applied in finance and the properties of statistical moments; (2) classical statistical inference, confidence interval construction, and formal hypothesis testing frameworks; (3) the Classical Linear Regression Model (CLRM), the Ordinary Least Squares (OLS) estimation engine, the Gauss-Markov Theorem, and the BLUE properties; (4) multivariate asset pricing applications, including the Sharpe-Lintner CAPM, Fama-French multi-factor specifications, and the Fama-MacBeth two-pass methodology; and (5) rigorous model evaluation, information criteria, structural break testing, and residual diagnostic testing for violations of Gauss-Markov assumptions.

1. Probability Distributions and Statistical Moments in Finance

Financial asset returns are continuous random variables governed by probability density functions (PDFs). The probabilistic behavior of returns is characterized through four statistical moments.

1.1 Major Probability Distributions in Financial Econometrics

Four primary continuous probability distributions form the operational basis of econometric estimation and hypothesis testing:

1.2 Non-Gaussian Distributions in Modern Risk Management

Because empirical asset returns routinely violate Gaussian assumptions, econometricians deploy flexible parametric alternatives:

• Generalized Error Distribution (GED): Controlled by a shape parameter kappa. When kappa = 2, GED reduces to the standard Normal distribution. When kappa < 2, GED generates leptokurtic distributions with heavier tails, widely used in GARCH volatility estimation.
• Hansen's Skewed Student's t-Distribution: Simultaneously incorporates both tail heaviness (via degrees of freedom nu) and directional asymmetry (via a skewness parameter lambda), capturing the stylized facts of equity returns.
• Extreme Value Theory (EVT) Distributions: Focuses specifically on the asymptotic behavior of the tails (Fréchet, Gumbel, Weibull distributions) to model catastrophic tail risk and black-swan market drawdowns without assuming any specific distribution for the central body of returns.

1.3 Formal Test of Normality: The Jarque-Bera (JB) Statistic

Because financial returns deviate from normality, econometricians formally test whether empirical residuals or returns match a Gaussian distribution using the Jarque-Bera test:

2. Statistical Inference and Classical Hypothesis Testing

Statistical inference encompasses point estimation, confidence interval construction, and hypothesis testing.

2.1 Point Estimation, Unbiasedness, and Efficiency

An estimator is a mathematical rule or formula applied to sample data to compute an estimate of an unobservable population parameter theta:

• Unbiasedness: An estimator theta_hat is unbiased if its expected value across infinite conceptual resamples equals the true population parameter: E(theta_hat) = theta. The bias is E(theta_hat) - theta = 0.
• Efficiency: An unbiased estimator is efficient if it achieves the minimum possible variance among all competing unbiased estimators: Var(theta_hat) ≤ Var(theta_tilde).
• Consistency: As sample size N approaches infinity, the probability that theta_hat deviates from theta collapses to zero. Consistency is the essential asymptotic property.

2.2 Confidence Intervals and Standard Errors

A point estimate provides a single numerical value, but conveys no information regarding sampling precision. Econometricians construct Confidence Intervals (CI):

2.3 Hypothesis Testing Framework: Errors and Power

2.4 Testing Multiple Linear Parameter Restrictions: The F-Test

While the t-test evaluates single hypotheses, researchers frequently test joint hypotheses involving multiple coefficients simultaneously (e.g., testing whether two slope coefficients sum to zero, or whether a group of predictors can be jointly omitted):

2.5 The Classical Hypothesis Testing Triad: LR, Wald, and LM Tests

In advanced financial econometrics, testing non-linear parameter restrictions relies on three asymptotically equivalent estimation principles:

• 1. Likelihood Ratio (LR) Test: Requires estimating both the unconstrained model and the constrained (restricted) model using Maximum Likelihood. Compares the difference in maximized log-likelihood values: LR = 2 * [ln(L_unrestricted) - ln(L_restricted)] ~ Chi-Square(m).
• 2. Wald Test: Requires estimating only the unconstrained model. Evaluates whether the unconstrained parameter estimates satisfy the hypothesized restrictions within sampling error. Widely used for testing linear and non-linear restrictions on regression coefficients.
• 3. Lagrange Multiplier (LM / Score) Test: Requires estimating only the restricted model. Evaluates the gradient (slope) of the log-likelihood function at the restricted parameter values. Ideal for testing model extensions (such as testing for higher-order serial correlation or ARCH effects) without estimating complex full models.
• Finite-Sample Relationship: In linear regression models with normal disturbances, a strict numerical inequality holds: Wald ≥ LR ≥ LM. The Wald test rejects the null most frequently, while the LM test is the most conservative.

3. The Classical Linear Regression Model (CLRM) and OLS

The Classical Linear Regression Model estimated via Ordinary Least Squares (OLS) represents the foundational workhorse of empirical finance.

3.1 Mathematical Derivation of Ordinary Least Squares (OLS)

Consider the bivariate regression model: Y_i = beta_0 + beta_1 * X_i + u_i. The observed value is Y_i, and the fitted value is Y_hat_i = b0 + b1 * X_i. The residual is e_i = Y_i - Y_hat_i. OLS derives the parameter estimates b0 and b1 by strictly minimizing the Residual Sum of Squares (RSS):

3.2 Precision and Sampling Variance of OLS Estimators

The precision of the OLS slope estimator depends on three fundamental factors:

• Disturbance Variance (sigma^2): Var(b1) is directly proportional to error variance sigma^2 = RSS / (N - 2). Greater noise in the data increases estimator uncertainty.
• Variation in Independent Variable X: Var(b1) is inversely proportional to Sum[(X_i - X_bar)^2]. The wider the spread of X values, the more precisely the slope can be estimated.
• Sample Size (N): As sample size grows, the denominator expands, driving the standard error SE(b1) toward zero.

3.3 The Gauss-Markov Theorem and BLUE Properties

The Gauss-Markov Theorem establishes the theoretical justification for why OLS is the premier linear estimation technique:

3.4 The Six Classical Gauss-Markov Assumptions

For OLS estimators to be unbiased, consistent, and efficient, six fundamental assumptions must hold:

• Assumption A1 (Linearity in Parameters): The model is linear in coefficients beta, even if variables enter non-linearly (e.g., Y = beta_0 + beta_1 * ln(X) + u is validly linear in parameters).
• Assumption A2 (Zero Conditional Mean of Errors): E(u_t | X_t) = 0. The expected value of the disturbance term conditional on the explanatory variables is strictly zero. This implies strict exogeneity: Covariance(X, u) = 0.
• Assumption A3 (Homoscedasticity / Constant Variance): Var(u_t | X_t) = sigma^2. The variance of the error term is identical and constant across all observations. Violation is Heteroskedasticity.
• Assumption A4 (No Autocorrelation / Serial Independence): Cov(u_i, u_j | X) = 0 for all i ≠ j. Disturbance terms corresponding to different time periods are completely uncorrelated. Violation is Autocorrelation.
• Assumption A5 (Absence of Perfect Multicollinearity): No exact linear relationship exists among the explanatory variables in a multivariate model. Matrix (X'X) must be non-singular and invertible.
• Assumption A6 (Normality of Disturbances): u_t ~ N(0, sigma^2). Disturbances are normally distributed. Required for exact small-sample hypothesis testing (t-tests and F-tests). Under large samples, Central Limit Theorems ensure asymptotic normality.

4. Multivariate Asset Pricing Applications in Finance

In financial economics, multiple regression is deployed to estimate systematic risk exposures, test factor pricing models, and decompose asset returns into alpha and beta components.

4.1 Capital Asset Pricing Model (CAPM) Empirical Regression

The Sharpe-Lintner CAPM states that the expected excess return on an asset is linearly determined by its exposure to the systematic market portfolio:

4.2 Empirical CAPM Estimation Case Walkthrough

Consider an empirical study estimating the CAPM parameters for an aggressive technology stock (TechCorp) against the benchmark market index over T = 60 monthly returns:

Model Diagnostic Summary: R-Squared = 0.712, F(1, 58) = 143.14, p < 0.001. Durbin-Watson = 2.04.

Economic Interpretation: The stock has a market beta of 1.34, confirming it is an aggressive equity asset exhibiting 34% greater sensitivity to systematic market movements than the average firm. Jensen's Alpha is positive (+0.42% monthly or ~5.04% annualized), but with p = 0.139 > 0.05, we fail to reject the null hypothesis of zero abnormal return, supporting market efficiency.

4.3 Fama-French Multi-Factor Asset Pricing Models

Empirical evidence revealed that market beta alone fails to explain the cross-section of equity returns. Eugene Fama and Kenneth French (1993) expanded CAPM into the Fama-French Three-Factor Model by adding firm size and value factors:

4.4 The Fama-MacBeth Two-Pass Cross-Sectional Methodology

To test whether estimated factor betas carry statistically significant positive risk premiums across the broader market, Eugene Fama and James MacBeth (1973) developed a two-pass methodology:

• Pass 1 (Time-Series Regressions): For each asset or portfolio i, run a time-series regression of returns on factors over T periods to estimate factor betas: beta_{i,m}, beta_{i,SMB}, beta_{i,HML}.
• Pass 2 (Cross-Sectional Regressions): For each time period t (from t = 1 to T), run a cross-sectional regression of asset returns across all N assets against the estimated betas: R_{i,t} = gamma_{0,t} + gamma_{1,t} * beta_{i,m} + ... + e_{i,t}.
• Inference: Compute the time-series mean and standard error of the estimated gamma coefficients. A statistically significant positive gamma confirms that the factor is an economically rewarded systematic risk driver.

5. Model Evaluation, Selection, and Diagnostic Testing

Estimating regression models is insufficient without rigorous diagnostic evaluation to verify whether the model is parsimonious, well-specified, and compliant with Gauss-Markov conditions.

5.1 Goodness of Fit vs. Information Criteria

• R-Squared (R^2): Proportion of variance in the dependent variable explained by the regression. A mathematical limitation is that adding any explanatory variable (even pure random noise) mechanically inflates R^2.
• Adjusted R-Squared: Imposes a degrees-of-freedom penalty for every additional predictor added: Adj R^2 = 1 - [(1 - R^2) * (N - 1) / (N - k - 1)]. Rises only if a new predictor improves model fit beyond what would be expected by random chance.
• Information Criteria (AIC & SIC/BIC): Model selection criteria that balance goodness of fit (maximized log-likelihood) against model complexity (number of parameters k). Econometricians select the model that minimizes the information criterion:
  • Akaike Information Criterion (AIC): AIC = 2*k - 2*ln(L). Emphasizes predictive accuracy; asymptotically efficient.
  • Schwarz Bayesian Information Criterion (SIC / BIC): SIC = k * ln(N) - 2*ln(L). Imposes a harsher penalty for sample size N and parameter count k. Asymptotically consistent; strongly favors parsimonious models.

5.2 Testing for Structural Breaks: The Chow Test

Financial relationships often undergo abrupt structural shifts due to regulatory overhauls, financial crises, or monetary policy regime shifts. The Chow Breakpoint Test evaluates whether regression parameters remain constant across two distinct sub-periods (Period 1 of size N1, Period 2 of size N2):

5.3 Residual Diagnostic Testing: Violations of Gauss-Markov Assumptions

5.4 Endogeneity and Instrumental Variables (Two-Stage Least Squares)

When explanatory variables correlate with the disturbance term (Cov(X, u) ≠ 0)—due to omitted variables, measurement error, or reverse causality—OLS estimators become biased and inconsistent. Econometricians resolve endogeneity through Two-Stage Least Squares (2SLS) Instrumental Variable (IV) estimation:

• Instrument Validity Requirements: An instrument Z must satisfy two conditions: (1) Relevance: Z must be strongly correlated with the endogenous explanatory variable X (First-stage F-statistic > 10 benchmark); and (2) Exogeneity: Z must be completely uncorrelated with the error term u (Cov(Z, u) = 0).
• Sargan-Hansen Overidentification Test: When the number of instruments exceeds endogenous variables, researchers test instrument validity. A non-significant test (p > 0.05) confirms that instruments are validly exogenous.

Summary Synthesis