Training turns the knobs to make these errors small.
- Drag the knobs until the squares are small.
- Each square's area is that point's error squared.
- Then let the computer find the best line.
- Raise the flexibility and watch both errors.
- Which degree predicts the new data best?
- Find the first degree that breaks the physics check.
Log scale: each step is 100 times larger.
Physics check failed: stress falls while the rubber is stretched further.
Physics check passed (stress rises everywhere). Passing is necessary, not sufficient.
Error for every degree
Size of each knob θ
More options
How a computer walks downhill: Lecture 2
The bowl of the error, seen from above (step 1): darker means a smaller total area
Instead of the formula, a computer can also walk downhill: every step turns both knobs a little towards a smaller total area.
The options below change step 2.
Table: error for every degree
| Degree | Knobs | Training error | New-data error | Physics check |
|---|
Table: data points
Data source: L. R. G. Treloar (1944), Trans. Faraday Soc. 40, 59–70; 25 measurements of vulcanised rubber in uniaxial tension (nominal stress against stretch).
How the fit is computed
A model with d + 1 knobs is a polynomial f(x) = θ0 + θ1x + … + θdxd. Training chooses the knobs that minimise the mean squared error (MSE) on the training data, plus α times the sum of squared coefficients if a penalty is set. The errors on the main view are the root of the MSE (RMSE), so they carry the unit of the data.
Step 1 uses the same idea with a straight line: the button moves the two knobs to the least-squares solution, which a formula gives at once. The Lecture 2 preview above reaches the same line by gradient descent: 40 small steps downhill, the last one lands exactly on it.
For numerical stability the input is rescaled so that the training points span −1 to 1, and the polynomial is written in Legendre polynomials instead of powers of x. A Householder QR factorisation solves the least-squares problem. With more knobs than training points, the demo takes the solution with the smallest coefficients (minimum norm). A tiny ridge of 10−10 is added only if the factorisation finds the problem numerically singular.
The knobs θj are the coefficients of the powers of the rescaled input t, which runs from −1 to 1 over the training data; the chart shows their size |θj| on a log scale. The penalty acts on exactly these knobs and leaves the height θ0 free, so the total size of θ1 to θd shrinks whenever α grows. A penalty smooths the curve between the data points, but it does not make predictions beyond the training data safe.
The verdict compares the error on the new data with the best degree without a penalty: "about right" means within 50% of that error; a penalty so strong that no degree gets close reads "too stiff".
The physics check samples the fitted curve between stretch 1 and 7.6 and fails if the stress ever decreases. The fits were checked against numpy (numpy.polyfit and a Legendre least-squares fit) on the Treloar data.