- Fit one bent line with the two knobs.
- Then let the computer do it (gradient descent).
- Add units and press Train: the bent lines add up.
- Then try a much larger .
- Watch the signal flow: see each unit switch on.
Thicker line: larger weight.
Typical error over the steps (log scale)
Physics check failed: negative stiffness. The stress falls while the rubber is stretched further.
Physics check passed (stiffness never negative in the measured range). Passing is necessary, not sufficient.
- Tap one measurement in the small plot.
- Press Forward, then Back, then Update.
Measurement flowing through the network
Training in step 2 does this for all 25 measurements at once and averages.
More options
These options change steps 2 and 3.
The tiny network from the lecture, computed by this page
Input x = 2, w1 = 3, bent line (ReLU), w2 = 1.5, measured y = 5, squared error (ŷ − y)2, learning rate 0.01.
Table: every knob of the network
| Unit | Weight in | Bends at stretch | Weight out | In the measured range |
|---|
Table: data points
| Stretch λ [-] | Stress P [MPa] |
|---|
Data source: L. R. G. Treloar (1944), Trans. Faraday Soc. 40, 59–70; 25 measurements of vulcanised rubber in uniaxial tension (nominal stress against stretch).
How it is computed
The network has one input, one layer of H units and one output: P = c + Σj aj max(0, wj s + bj), with the rescaled stretch s = (λ − 1) / 6.6, which runs from 0 to 1 over the measurements. Every unit is a bent line (ReLU); the offset bj (often called bias) sets where it bends, the weight in wj how steeply it rises, the weight out aj how much of it is added. The output offset c is one more knob.
Training minimises the mean squared error over all 25 measurements by plain gradient descent: every step moves every knob against its gradient, times the learning rate. The typical error shown is the root of the mean squared error (RMSE), in MPa. Training pauses by itself when 1000 more steps improve the typical error by less than 1 %, when every unit is switched off, or after 30 000 steps.
The gradients come from an explicit backward pass (backpropagation, the chain rule): for one measurement with squared error L = (P̂ − P)2, ∂L/∂P̂ = 2(P̂ − P), ∂L/∂aj = ∂L/∂P̂ · hj, and for a unit that is switched on ∂L/∂wj = ∂L/∂P̂ · aj · s and ∂L/∂bj = ∂L/∂P̂ · aj; a switched-off unit passes nothing back. Step 3 shows these numbers for one measurement and updates with them (the offsets and the output offset are updated too, but not drawn). Step 2 averages them over all 25 measurements. The page checks the backward pass against finite differences.
Random start (fixed by its number, so every run is repeatable): each unit opens to the right with a steepness wj between 2 and 3; unit j bends at a random place in the j-th of H equal slices of the stretch from 1 to about 8.6, so a unit may bend beyond the last measurement; the weights out start as random numbers with spread √(2/H), the output offset at 0. A unit that is switched off for every measurement gets no gradient and never learns, but it still bends the curve beyond the data.
The trained network under More options is the one of the lecture listing (network_rubber.py): the same plain gradient descent with learning rate 0.05, run for 20 000 steps from the listing's own random start (typical error 0.081 MPa). Its units are shown in the order in which they bend, at stretch 0.98, 5.51, 7.01 and 9.67; this only renumbers them. The dots on its error curve are the errors the listing prints every 5000 steps.
The signal flow of step 2 computes the network for one stretch at a time: the dots on the first edges are sized by wj s, those on the second edges by aj hj, the value each unit adds to the stress; a unit fills in proportion to its output hj, relative to its largest output in the plot, and is hollow while it is switched off. During training, the dots show the forward pass and the dashed pieces the way back, once per 1.4 s; the steps themselves run much faster.
Step 1 uses a single bent line that is zero before the bend. The button walks downhill on the two knobs (the bend moves with four times the step of the slope) until the steps become tiny; this finds a nearby valley, which need not be the deepest one. The physics check computes the slope of the curve on every straight piece between stretch 1 and 7.6 and fails if any slope is negative.