When are Deep Networks better than Decision Forests at small sample sizes, and how?

Team: DF/DN

Program: Biomedical Engineering

Forests have empirically dominated tabular data scenarios, where the relative position of features is irrelevant.In contrast, networks typically dominate other methods on large sample size structured data scenarios, where the relative position of features is key for sample identification.The relationship between the internal representations that the two approaches learn has not yet been made explicit, to our knowledge. We illustrate the conceptual commonalities of their representations on three different classification tasks.