Building Functions from Data
Given a set of data points, we often want a function that describes them — to fill in gaps, smooth noise, or make predictions. Two related but distinct approaches serve different goals.
Interpolation finds a function that passes exactly through every data point. Given points, there is a unique interpolating polynomial of degree through them. Spline interpolation instead joins the points with low-degree (often cubic) pieces, matching smoothly at each point — the standard choice, because a single high-degree polynomial through many points can oscillate wildly between them (Runge's phenomenon). Interpolation assumes the data are exact and the goal is to reconstruct the underlying function precisely at the given points.
Curve fitting (regression) instead finds a function of a chosen form that comes close to the points without necessarily hitting any of them, by minimizing the total error. Least-squares fitting minimizes the sum of squared vertical distances (residuals) from the points to the curve — the same idea as Unit 7's linear regression, extended to any model form. Fitting assumes the data contain noise, and the goal is to capture the underlying trend, not the individual jitter.
The choice hinges on whether the data are exact or noisy:
- Exact data, want the true function → interpolate (pass through every point).
- Noisy data, want the trend → fit (approximate, minimize error).
Forcing a curve through every noisy point (over-interpolating) chases the noise and gives a wiggly, useless model — the classic overfitting trap. Conversely, fitting a too-simple curve to structured data misses real features (underfitting). Choosing the right approach and model complexity is central to data science and numerical modeling.
Common pitfall: interpolating noisy data — forcing a curve through every point when the data contain measurement noise. This makes the curve chase the random jitter, producing wild oscillations and poor predictions (overfitting). When data are noisy, fit a simpler model by least squares to capture the trend, rather than interpolating every point exactly; interpolation is for data you trust as exact.
Noisy scatter points with an accent wiggly high-degree interpolant threading every point (overfit) beside a smooth least-squares line capturing the trend — interpolation versus fitting on noisy data.