Skip to content

Latest commit

 

History

History
21 lines (12 loc) · 1018 Bytes

File metadata and controls

21 lines (12 loc) · 1018 Bytes

Chapter 06 Exercises

  1. In a binary classification task (with fairly balanced classes), in what scenario the AUC is not a reliable performance measure for model selection? We are looking at a general case. As a hint, imagine we plotted the ROC curves of two models -- think about in what case the AUC is too simplistic for comparing these models.

Related but Beyond Scope

Exercises in this section cannot be answered by just reading the chapter and require further reading and research.

  1. In the following paragraph, replace ‘__’ by the letters a-c, which are defined as
    a): Out-of-Bag (OOB) Bootstrap
    b): .632 Bootstrap
    c): .632+ Bootstrap

"For estimating the generalization performance, the _(1) is less optimistically biased than the _, while (1) and _ are less pessimistically biased than the _."

Suggested reading to answer this question: