Fit a polynomial to 30 noisy points: how flexible should f̂ be?
polynomial degree (flexibility)
new training set
show fits to 20 other training sets
true f(x) = sin(2πx) + x/2
f̂ from this training set
f̂ from other training sets
Training error, test error, and where the test error comes from (2,000 training sets per degree)
k-nearest neighbours: one knob from very flexible (K = 1) to rigid (K = 150)
K, the number of neighbours that vote
new training set
decision regions (200 training points)
error against K (average of 20 training sets)
class 0
class 1
Bayes boundary
test error
training error
Bayes error rate (dashed)
Past the interpolation threshold: test error of a linear model on the first p of 400 features (n = 40)