Entropy is the average surprise: a pure group has 0 bits, half and half 1 bit.
- Find a question that leaves two pure groups.
- Try every input: is there a second one?
- Why is water alone a poor question?
The question asks about
Before the question1 bit
After the question (weighted average)
Information gain
First question: how much each threshold reduces the squared error
- Grow the tree. When does the error on new recipes stop falling?
- Which threshold wins the first question? See the small plot.
Typical error [MPa] for every depth
- Add trees. Does the forest beat one tree on new recipes?
- What happens where there is no data?
- Show Abrams' law: which curve continues outside the data?
One tree alone, new recipes
Typical error on new recipes [MPa] for every number of trees
More options
These options change steps 2 and 3.
Table: every question about the eight mixes
| Question | Yes group | No group | Information gain |
|---|
Table: the eight mixes
| Cement [kg/m³] | Water [kg/m³] | Fly ash [kg/m³] | Water / cement | Strength [MPa] | Class |
|---|
Table: typical error for every depth
| Depth | Leaves | Training mixes [MPa] | New recipes [MPa] |
|---|
Data source: I-C. Yeh (1998), Cement and Concrete Research 28, 1797-1808; 1030 tests of 428 concrete recipes from several laboratories (UCI Machine Learning Repository). Steps 2 and 3 use the 425 mixes tested after 28 days, which come from 417 recipes. The eight mixes of step 1 come from these, rounded to whole kg/m³ and 0.1 MPa as in the lecture.