Книга: The Master Algorithm: How the Quest for the Ultimate Learning Machine Will Remake Our World

Chapter Three

Chapter Three

Hume’s classic formulation of the problem of induction appears in Volume I of A Treatise of Human Nature (1739). David Wolpert derives his “no free lunch” theorem for induction in “The lack of a priori distinctions between learning algorithms”* (Neural Computation, 1996). I discuss the importance of prior knowledge in machine learning in “Toward knowledge-rich data mining”* (Data Mining and Knowledge Discovery, 2007) and misinterpretations of Occam’s razor in “The role of Occam’s razor in knowledge discovery”* (Data Mining and Knowledge Discovery, 1999). Overfitting is one of the main themes of The Signal and the Noise, by Nate Silver (Penguin Press, 2012), who calls it “the most important scientific problem you’ve never heard of.” “Why most published research findings are false,”* by John Ioannidis (PLoS Medicine, 2005), discusses the problem of mistaking chance findings for true ones in science. Yoav Benjamini and Yosef Hochberg propose a way to combat it in “Controlling the false discovery rate: A practical and powerful approach to multiple testing”* (Journal of the Royal Statistical Society, Series B, 1995). The bias-variance decomposition is presented in “Neural networks and the bias/variance dilemma,” by Stuart Geman, Elie Bienenstock, and Ren? Doursat (Neural Computation, 1992). “Machine learning as an experimental science,” by Pat Langley (Machine Learning, 1988), discusses the role of experimentation in machine learning.

William Stanley Jevons first proposed viewing induction as the inverse of deduction in The Principles of Science (1874). The paper “Machine learning of first-order predicates by inverting resolution,”* by Steve Muggleton and Wray Buntine (Proceedings of the Fifth International Conference on Machine Learning, 1988), initiated the use of inverse deduction in machine learning. The book Relational Data Mining,* edited by Sa?o D?eroski and Nada Lavra? (Springer, 2001), is an introduction to the field of inductive logic programming, where inverse deduction is studied. “The CN2 Induction Algorithm,”* by Peter Clark and Tim Niblett (Machine Learning, 1989), summarizes some of the main Michalski-style rule induction algorithms. The rule-mining approach used by retailers is described in “Fast algorithms for mining association rules,”* by Rakesh Agrawal and Ramakrishnan Srikant (Proceedings of the Twentieth International Conference on Very Large Databases, 1994). An example of rule induction for cancer prediction is described in “Carcinogenesis predictions using inductive logic programming,” by Ashwin Srinivasan, Ross King, Stephen Muggleton, and Michael Sternberg (Intelligent Data Analysis in Medicine and Pharmacology, 1997).

The two leading decision tree learners are presented in C4.5: Programs for Machine Learning,* by J. Ross Quinlan (Morgan Kaufmann, 1992), and Classification and Regression Trees,* by Leo Breiman, Jerome Friedman, Richard Olshen, and Charles Stone (Chapman and Hall, 1984). “Real-time human pose recognition in parts from single depth images,”* by Jamie Shotton et al. (Communications of the ACM, 2013), explains how Microsoft’s Kinect uses decision trees to track gamers’ motions. “Competing approaches to predicting Supreme Court decision making,” by Andrew Martin et al. (Perspectives on Politics, 2004), describes how decision trees beat legal experts at predicting Supreme Court votes and shows the decision tree for Justice Sandra Day O’Connor.

Allen Newell and Herbert Simon formulated the hypothesis that all intelligence is symbol manipulation in “Computer science as empirical enquiry: Symbols and search” (Communications of the ACM, 1976). David Marr proposed his three levels of information processing in Vision* (Freeman, 1982). Machine Learning: An Artificial Intelligence Approach,* edited by Ryszard Michalski, Jaime Carbonell, and Tom Mitchell (Tioga, 1983), gives a snapshot of the early days of symbolist research in machine learning. “Connectionist AI, symbolic AI, and the brain,”* by Paul Smolensky (Artificial Intelligence Review, 1987), gives a connectionist view of symbolist models.

Оглавление книги