Книга: The Master Algorithm: How the Quest for the Ultimate Learning Machine Will Remake Our World

Index

Index

Abagnale, Frank, Jr., 177, 306

aboutthedata.com, 272

A/B testing, 227, 309

Accuracy, 75-79, 87, 241, 243

Ackley, David, 103

Action potentials, 95-96, 104-105

Acxiom, 272

Adam, the robot scientist, 16, 84, 299

Adaptive systems, 8. See also Machine learning

AdSense system, 160

AI. See Artificial intelligence (AI)

AIDS vaccine, Bayesian networks and, 159-160

Alchemy, 246-259, 309

Markov logic networks and, 246-250

shortcomings, 255-259

tribes of machine learning and, 250-255

alchemy.cs.washington.edu, 250

Algorithms

classifiers, 86-87

complexity monster and, 5-6

defined, 1

designing, 4-5

further readings, 298-299

genetic, 122-128

overview, 1-6

structure mapping, 199-200

See also Machine learning; individual algorithms

AlphaDog, 21

Amazon, 198, 266, 291

A/B testing and, 227

data gathering, 211, 271, 272

machine learning and, 11, 12

Mechanical Turk, 14

recommendations, 12-13, 42, 184, 268, 286

Analogical reasoning, 179, 197

Analogizers, 51, 53, 54, 172-173

Alchemy and, 253-254

case-based reasoning, 197-200

dimensionality, 186-190

Master Algorithm and, 240-241

nearest-neighbor algorithm, 178-186

similiarity and, 179

support vector machines, 53, 190-196

symbolists vs., 200-202

Analogy, 175-179, 197-200

AND gate, 96

AND operation, 2

Anna Karenina (Tolstoy), 66

Apple, 272

Aristotle, 58, 64, 72, 178, 243

Artificial intelligence (AI)

human control of, 282-284

knowledge engineers and, 35-36

machine learning and, 8, 89-90

ASIC (application-specific integrated circuit) design, 49

Asimov, Isaac, 232, 280

Assumptions

ill-posed problem and, 64

of learners, 44

learning from finite data and, 24-25

prior, 174

simplifying to reduce number of probabilities, 150

symbolists and, 61-62

Atlantic (magazine), 273-274

AT &T, 272

Attribute selection, 186-187, 188-189

Attribute weights, 189

Auditory cortex, 26

Autoencoder, 116-118

Automation, machine learning and, 10

Automaton, 123

The Average American (O’Keefe), 206

Average member, 206

Axon, 95

Babbage, Charles, 28

Backpropagation (backprop), 52, 104, 107-111, 115, 302

Alchemy and, 252

genetic algorithms vs., 128

neural networks and, 112-114

reinforcement learning and, 222

Bagging, 238

Baldwin, J. M., 138-139

Baldwin effect, 139, 140, 304

Bandit problems, 129-130

Barto, Andy, 221

Bayes, Thomas, 144-145

Bayesian learning, 166-170, 174-175

Bayesian methods, cell model and, 114

Bayesian model averaging, 166-167

Bayesian models, tweaking probabilities, 170-173

Bayesian networks, 24, 156-161, 305-306

Alchemy and, 250

gene regulation and, 159

inference problem and, 161-166

Master Algorithm and, 240, 245

relational learning and, 231

Bayesians, 51, 52-53, 54, 143-175

Alchemy and, 253

further reading, 304-305

hidden Markov model, 154-155

If… then… rules and, 155-156

inference problem, 161-166

learning and, 166-170

logic and probability and, 173-175

Markov chain, 153-155

Markov networks, 170-173

Master Algorithm and, 240-241, 242

medical diagnosis and, 149-150

models and, 149-153

nature and, 141

probabilistic inference and, 52, 53

See also Bayesian networks

Bayes’ theorem, 31-32, 52-53, 143-149, 253

Beam search, 135

“Beer and diapers” rule, 69-70

Belief, probability and, 149

Belief propagation, 161-164, 242, 253

Bell Labs, 190

Bellman, Richard, 188, 220

Bellman’s equation, 220

Berkeley, George, 58

Berlin, Isaiah, 41

Bias, 78-79

Bias-free learning, futility of, 64

Bias-variance decomposition, 301

The Bible Code (Drosnin), 72

Big data, 21

A/B testing and, 227

algorithms and, 7

clustering and, 206-207

relational learning and, 232-233

science, machine learning, and, 14-16

scientific truth and, 40

Big-data systems, 258

Bing, 12

Biology, learning algorithms and, 15

Black swans, 38-39, 158, 232

The Black Swan (Taleb), 38

Blessing of nonuniformity, 189

Board games, reinforcement learning and, 219

Bohr, Niels, 178, 199

Boltzmann distribution, 103-104

Boltzmann machines, 103-104, 117, 250

Boole, George, 104, 175

Boolean circuits, 123, 136

Boolean variable, 149

Boosting, 238

Borges, Jorge Luis, 71

Box, George, 151

Brahe, Tycho, 14, 131

Brahe phase of science, 39-40

Brain

learning algorithms and, 26-28

mapping, 118

number of connections in, 94-95

reverse engineering the, 52, 302

S curves and, 105

simulating with computer, 95

spin glasses and, 102-103

BRAIN initiative, 118

Breiman, Leo, 238

Brin, Sergey, 55, 227, 274

Bryson, Arthur, 113

Bucket brigade algorithm, 127

Building blocks, 128-129, 134

Buntine, Wray, 80

Burglar alarms, Bayesian networks and, 157-158

Burks, Arthur, 123

Burns, Bob, 206

Business, machine learning and, 10-13

C. elegans, 118

Cajal, Santiago Ram?n y, 93-94

Caltech, 170

CancerCommons.org, 261

Cancer cure

algorithm for, 53-54

Bayesian learning and, 174

inverse deduction and, 83-85

Markov logic network and, 249

program for (CanceRx), 259-261, 310

Cancer diagnosis, 141

Cancer drugs

predicting efficacy of, 83-84

relational learning and models for, 233

selection of, 41-42

CanceRx, 259-261, 310

Capital One, 272

Carbonell, Jaime, 69

Carnap, Rudolf, 175

Cars

driverless, 113, 166, 172, 306

learning to drive, 113

Case-based reasoning, 198, 307

Catch Me If You Can (film), 177

Cause and effect, Bayes’ theorem and, 145-149

Cell

model of, 114-115

relational learning and workings of, 233

Cell assembly, 94

Cell phone, hidden Markov models and, 155

Centaurs, 277

Central Dogma, 83

Cerebellum, 27, 118

Chance, Bayes and, 145

Chaos, study of, 30

Checkers-playing program, 219

Cholera outbreak, London’s, 182-183

Chomsky, Noam, 36-38

Chrome, 266

Chunking, 223-227, 254, 309

Circuit design, genetic programming and, 135-136

Classes, 86-87, 209, 257

Classifiers, 86-87, 127

Master Algorithm and, 240

Na?ve Bayes, 151-153

nearest-neighbor algorithm and, 183

Clinton, Bill, 18

Clustering, 205-210, 254, 257

hierarchical, 210

Cluster prototypes, 207-208

Clusters, 205-210

“Cocktail party” problem, 215

Cognition, theory of, 226

Coin toss, 63, 130, 167-168

Collaborative filtering systems, 183-184, 306-307

Columbus test, 113

Combinatorial explosion, 73-74

Commoner, Barry, 158

Commonsense reasoning, 35, 118-119, 145, 276-277, 300

Complexity monster, 5-6, 7, 43, 246

Compositionality, 119

Computational biologists, use of hidden

Markov models, 155

Computers

decision making and, 282-286

evolution of, 286-289

human interaction with, 264-267

as learners, 45

logic and, 2

S curves and, 105

as sign of Master Algorithm, 34

simulating brain using, 95

as unifier, 236

writing own programs, 6

Computer science, Master Algorithm and, 32-34

Computer vision, Markov networks and, 172

Concepts, 67

conjunctive, 66-68

set of rules and, 68-69

sets of, 86-87

Conceptual model, 44, 152

Conditional independence, 157-158

Conditional probabilities, 245

Conditional random fields, 172, 306

Conference on Neural Information Processing Systems (NIPS), 170, 172

Conjunctive concepts, 65-68, 74

Connectionists/connectionism, 51, 52, 54, 93-119

Alchemy and, 252

autoencoder and, 116-118

backpropagation and, 52, 107-111

Boltzmann machine and, 103-104

cell model, 114-115

connectomics, 118-119

deep learning and, 115

further reading, 302-303

Master Algorithm and, 240-241

nature and, 137-142

neural networks and, 112-114

perceptron, 96-101, 107-108

S curves and, 104-107

spin glasses and, 102-103

symbolist learning vs., 91, 94-95

Connectomics, 118-119

Consciousness, 96

Consilience (Wilson), 31

Constrained optimization, 193-195, 241, 242

Constraints, support vector machines and, 193-195

Convolutional neural networks, 117-119, 303

Cope, David, 199, 307

Cornell University, Creative Machines Lab, 121-122

Cortex, 118, 138

unity of, 26-28, 299-300

Counterexamples, 67

Cover, Tom, 185

Crawlers, 8-9

Creative Machines Lab, 121-122

Credit-assignment problem, 102, 104, 107, 127

Crick, Francis, 122, 236

Crossover, 124-125, 134-136, 241, 243

Curse of dimensionality, 186-190, 196, 201, 307

Cyber Command, 19

Cyberwar, 19-21, 279-282, 299, 310

Cyc project, 35, 300

DARPA, 21, 37, 113, 121, 255

Darwin, Charles, 28, 30, 131, 235

algorithm, 122-128

analogy and, 178

Hume and, 58

on lack of mathematical ability, 127

on selective breeding, 123-124

variation and, 124

Data

accuracy of held-out, 75-76

Bayes’ theorem and, 31-32

control of, 45

first principal component of the, 214

human intuition and, 39

learning from finite, 24-25

Master Algorithm and, 25-26

patterns in, 70-75

sciences and complex, 14

as strategic asset for business, 13

theory and, 46

See also Big data; Overfitting; Personal data

Database engine, 49-50

Databases, 8, 9

Data mining, 8, 73, 232-233, 298, 306. See also Machine learning

Data science, 8. See also Machine learning

Data scientist, 9

Data sharing, 270-276

Data unions, 274-275

Dawkins, Richard, 284

Decision making, artificial intelligence and, 282-286

Decision theory, 165

Decision tree induction, 85-89

Decision tree learners, 24, 301

Decision trees, 24, 85-90, 181-182, 188, 237-238

Deduction

induction as inverse of, 80-83, 301

Turing machine and, 34

Deductive reasoning, 80-81

Deep learning, 104, 115-118, 172, 195, 241, 302

DeepMind, 222

Democracy, machine learning and, 18-19

Dempster, Arthur, 209

Dendrites, 95

Descartes, Ren?, 58, 64

Descriptive theories, normative theories vs., 141-142, 304

Determinism, Laplace and, 145

Developmental psychology, 203-204, 308

DiCaprio, Leonardo, 177

Diderot, Denis, 63

Diffusion equation, 30

Dimensionality, curse of, 186-190, 307

Dimensionality reduction, 189-190, 211-215, 255

nonlinear, 215-217

Dirty Harry (film), 65

Disney animators, S curves and, 106

“Divide and conquer” algorithm, 77-78, 80, 81, 87

DNA sequencers, 84

Downweighting attributes, 189

Driverless cars, 8, 113, 166, 172, 306

Drones, 21, 281

Drugs, 15, 41-42, 83. See also Cancer drugs

Duhigg, Charles, 223

Dynamic programming, 220

Eastwood, Clint, 65

Echolocation, 26, 299

Eddington, Arthur, 75

Effect, law of, 218

eHarmony, 265

Eigenfaces, 215

80/20 rule, 43

Einstein, Albert, 75, 200

Eldredge, Niles, 127

Electronic circuits, genetic programming and, 133-134

Eliza (help desk), 198

EM (expectation maximization) algorithm, 209-210

Emotions, learning and, 218

Empathy-eliciting robots, 285

Empiricists, 57-58

Employment, effect of machine learning on, 276-279

Enlightenment, rationalism vs. empiricism, 58

Entropy, 87

Epinions, 231

Equations, 4, 50

Essay on Population (Malthus), 178, 235

Ethics, robot armies and, 280-281

Eugene Onegin (Pushkin), 153-154

“Explaining away” phenomenon, 163

Evaluation

learning algorithms and, 283

Markov logic networks and, 249

Master Algorithm and, 239, 241, 243

Evolution, 28-29, 121-142

Baldwinian, 139

Darwin’s algorithm, 122-128

human-directed, 286-289, 311

Master Algorithm and, 28-29

of robots, 121-122, 137, 303

role of sex in, 134-137

technological, 136-137

See also Genetic algorithms

Evolutionaries, 51, 52, 54

Alchemy and, 252-253

exploration-exploitation dilemma, 128-130, 221

further reading, 303-304

genetic programming and, 52

Holland and, 127

Master Algorithm and, 240-241

nature and, 137-139

Evolutionary computation, 121-142

Evolutionary robotics, 121-122, 303

Exclusive-OR function (XOR), 100-101, 112, 195

Exploration-exploitation dilemma, 128-130, 221

Exponential function, machine learning and, 73-74

The Extended Phenotype (Dawkins), 284

Facebook, 44, 291

data and, 14, 274

facial recognition technology, 179-180

machine learning and, 11

relational learning and, 230

sharing via, 271-272

Facial identification, 179-180, 182

False discovery rate, 77, 301

Farming, as analogy for machine learning, 6-7

Feature selection, 188-189

Feature template, 248

Feature weighting, 189

Ferret brain rewiring, 26, 299

Feynman, Richard, 4

Filter bubble, 270

Filtering spam, rule for, 125-127

First principal component of the data, 214

Fisher, Ronald, 122

Fitness

Fisher on, 122

in genetic programming, 132

Master Algorithm and, 243

neural learning and, 138-139

sex and, 135

Fitness function, 123-124

Fitness maximum, genetic algorithms and, 127-128, 129

Fix, Evelyn, 178-179, 186

Fodor, Jerry, 38

Forecasting, S curves and, 106

Foundation Medicine, 41, 261

Foundation (Asimov), 232

Fractal geometry, 30, 300

Freakonomics (Dubner & Levitt), 275

Frequentist interpretation of probability, 149

Freund, Yoav, 238

Friedman, Milton, 151

Frontiers, 185, 187, 191, 196

“Funes the Memorious” (Borges), 71

Futility of bias-free learning, 64

FuturICT project, 258

Galileo, 14, 72

Galois, ?variste, 200

Game theory, machine learning and, 20

Gaming, reinforcement learning and, 222

Gates, Bill, 22, 55, 152

GECCO (Genetic and Evolutionary Computing Conference), 136

Gene expression microarrays, 84-85

Generalizations, choosing, 60, 61

Generative model, Bayesian network as, 159

Gene regulation, Bayesian networks and, 159

Genetic algorithms, 122-128

Alchemy and, 252

backpropagation vs., 128

building blocks and, 128-129, 134

schemas, 129

survival of the fittest programs, 131-134

The Genetical Theory of Natural Selection (Fisher), 122

Genetic programming, 52, 131-133, 240, 244, 245, 252, 303-304

sex and, 134-137

Genetic Programming (Koza), 136

Genetic search, 241, 243, 249

Genome, poverty of, 27

Gentner, Dedre, 199

Ghani, Rayid, 17

The Ghost Map (Johnson), 182-183

Gibson, William, 289

Gift economy, 279

Gleevec, 84

Global Alliance for Genomics and Health, 261

G?del, Escher, Bach (Hofstadter), 200

Good, I. J., 286

Google, 9, 44, 291

A/B testing and, 227

AdSense system, 160

communication with learner, 266-267

data gathering, 272

DeepMind and, 222

knowledge graph, 255

Master Algorithm and, 282

Na?ve Bayes and, 152

PageRank and, 154, 305

problem of induction and, 61

relational learning and, 227-228

search results, 13

value of data, 274

value of learning algorithms, 10, 12

Google Brain network, 117

Google Translate, 154, 304

Gould, Stephen Jay, 127

GPS, 212-214, 216, 277

Gradient descent, 109-110, 171, 189, 193, 241, 243, 249, 252, 257-258

Grammars, formal, 36-37

Grandmother cell, perceptron and, 99-100

Graphical models, 240, 245-250

Graphical user interfaces, 236

The Guns of August (Tuchman), 178

Handwritten digit recognition, 189, 195

Hart, Peter, 185

Hawking, Stephen, 47, 283

Hawkins, Jeff, 28, 118

Hebb, Donald, 93, 94

Hebb’s rule, 93, 94, 95

Heckerman, David, 151-152, 159-160

Held-out data, accuracy of, 75-76

Help desks, 198

Hemingway, Ernest, 106

Heraclitus, 48

Hidden Markov model (HMM), 154-155, 159, 210, 305

Hierarchical structure, Markov logic network with, 256-257

Hill climbing, 135, 136, 169, 189, 252

Hillis, Danny, 135

Hinton, Geoff, 103, 104, 112, 115, 137, 139

The Hitchhiker’s Guide to the Galaxy (Adams), 130

HIV testing, Bayes’ theorem and, 147-148

HMM. See Hidden Markov model (HMM)

Ho, Yu-Chi, 113

Hodges, Joe, 178-179, 186

Hofstadter, Douglas, 200

Holland, John, 122-128, 129, 130, 131, 134

Homo technicus, 288-289

Hopfield, John, 102-103, 170

Hopfield networks, 103, 116, 302

Horning, J. J., 36-37

Howbert, Jeff, 292

How to Create a Mind (Kurzweil), 28

H &R Block, 277

Hubble, Edwin, 14-15

Human complexity

as complexity monster, 5

machine learning and, 258-259

Human control of artificial intelligence, 282-286

Human-directed evolution, 286-289, 311

Human intuition, data and, 39

Humanities, machine learning and, 278

Human Rights Watch, 281

Hume, David, 58-59, 62, 63, 93, 178, 300-301

Hume’s problem of induction, 58-59, 145, 169, 197, 251

Humie Awards, 134

Hunt, Earl, 88

Hyperplanes, 98, 100, 195, 196

Hyperspace, 107-111, 187

Hypotheses

Bayesians and, 144, 167-168

machine learning and, 13-15

overfitting and, 73-75

preference for simpler, 77-78

Red Queen, 135

testing, 13-15, 49

IBM, 13, 37, 219

ICML. See International Conference on Machine Learning (ICML)

If… then… rules, 68-71, 84-85, 125-127, 132, 152, 155-156, 201-202, 244-245, 254

Ill-posed problem, 64

Immortality, genetic algorithms and, 126

Incognito mode, 266

Income, basic guaranteed, 279

Independent-component analysis, 215

Indexers, 8, 9

Indifference, principle of, 145

Induction

decision tree, 85-89

further readings, 300-302

as inverse of deduction, 80-83, 301

Master Algorithm and, 34

Newton’s rules of, 65-66

problem of, 59-62

Inductive logic programming. See Inverse deduction

Inductivist turkey, 61

Inference

Alchemy and, 256-257

Bayesian networks and, 161-166

Information, cyberwar and, 19-20

Information gain, 87, 188

Information processing systems, study of, 89

Information Revolution, 9

Instance-based learning, 201-202, 250

Institute of Control Sciences, 190

Intelligence

computers and, 35, 286, 287

symbolists and, 52, 89, 302

International Conference on Machine Learning (ICML), 136

Internet, 231, 236

Intuition, evidence and, 39

Inverse deduction, 52, 80-83, 90-91, 301

Alchemy and, 252

cell model and, 115

computational intensiveness of, 85

cure for cancer and, 83-85

Master Algorithm and, 90, 241, 242-243

Newton’s principle and, 82-83

shortcomings of, 91

IPsoft, 198

Irrelevant attributes, nearest neighbor algorithm and, 186-187, 188-189

Isomap, 217, 255, 308

Iterative search, 28

Jackel, Larry, 195

James, William, 93, 178, 205

Java, 4

Jelinek, Fred, 37

Jesus, 144

Jevons, William Stanley, 80

Johnson, Steven, 182-183

Jordan, Michael, 164, 170

Junction trees, 163

Kaggle.com, 292

Kahneman, Daniel, 141

Kalman filter, 155, 305

Kant, Immanuel, 178

Kekul?, August, 178

Kennedy, John F., 36, 177-178, 182

Kepler, Johannes, 65, 131

laws of, 40, 65, 131

Kepler phase of science, 39-40

Kernels, 192, 196, 243, 307

Keyword matching, 20

Kinect, 88, 237, 238

Kipling, Rudyard, 68

k-means algorithm, 208, 210, 308

k-nearest-neighbor algorithm, 183

Knowledge, 8, 52, 64

unity of, 31

Knowledge acquisition bottleneck, 89-90

Knowledge-based system, 89-90

Knowledge discovery, 8. See also Machine learning

Knowledge engineering

machine learning and, 102

symbolist learning and, 90

Knowledge engineers, 251

machiner learners vs., 34-38

Knowledge graph, 255

Koza, John, 131, 132, 134, 136

Krugman, Paul, 232

Kurzweil, Ray, 28, 83, 186, 286-289

Laird, John, 226

Laird, Nan, 209

Landmine example, support vector machines and, 192-193

Lang, Kevin, 136

Language learning, 36-37

Laplace, Pierre-Simon de, 144-145

Latent semantic analysis, 215, 308

Law of effect, 218

Law of similarity, 178

Lazy learning, 180-182

Learning

across problem domains, 199

by association, 93-94

Bayesian, 144, 166-170, 174-175

children’s, 203-204, 308

deep, 104, 115-118, 172, 195, 241

instance-based, 201-202

knowledge and, 64

lazy, 180-182

as problem solving, 226

reinforcement, 218-223, 308

rule-based, 69-70, 201-202

statistical, 37, 228, 297, 300, 307

statistical relational, 309

supervised, 57-202, 209, 214, 220, 222, 226

unsupervised, 203-233

Learning algorithms (learners)

Alchemy, 250-259

children’s learning and, 203-204

control of, 45

curse of dimensionality and, 188

empirical evaluation of, 76

evaluation and, 283

interaction with, 264-267

machine learning and, 6-10

optimization and, 283

prediction and, 39

representation and, 283

speed and, 139-142

as superpredators, 8-9

variety of tasks undertaken by, 23-25

See also individual algorithms

LeCun, Yann, 113, 195

Lehman Brothers, 106

Leibniz, Gottfried, 58, 64, 175, 198

Lenat, Doug, 35

Lewis, Michael, 39

Linear regression, 15, 50, 113, 182, 214, 306

LinkedIn, 269, 271

Lipson, Hod, 121

Local minimum, 102, 103, 110-111

Local optima, 111, 128

Locally weighted regression, 182, 306

Locke, John, 58, 93, 178

Logic, 50, 33, 49, 80-81

Bayesians and, 173

computers and, 2

Master Algorithm and, 240, 244, 245-246

probability and, 173-175, 245-246, 306, 309

unified with graphical models, 245-250

Logical inference, Alchemy and, 256

Logic gates, 96

Logistic curve. See S curves

Long-tail phenomenon, 12, 299

Long-term potentiation, 27

Loopy belief propagation, 163-164, 231

Lorenz, Konrad, 138

Low-pass filter, 133

Machine learners, knowledge engineers vs., 34-38

Machine learning, 6-10

analogy and, 178-179

bias and variance and, 78-79

big data and, 15-16

business and, 10-13

chunking, 223-227

clustering, 205-210

dimensionality reduction, 211-217

effect on employment, 276-279

exponential function and, 73-74

fitness function and, 123

further readings, 297-298

future of, 21-22

impact on daily life, 298

effect on employment, 276-279

meta-learning, 237-239

nature vs. nurture debate and, 29, 137-139

Newton’s principle and, 65-66

planetary-scale, 256-259

politics and, 16-19

principal-component analysis, 211-217

problem of unpredictability and, 38-40

reinforcement learning, 218-223, 226-227

relational learning, 227-233

relationship to artificial intelligence, 8

science and, 13-16, 235-236

significance tests and, 76-77

as technology, 236-237

Turing point and, 286, 288

war and, 19-21, 279-282

See also Algorithms

Machine-learning problem, 61-62, 109-110

Machine-translation systems, 154

MacKay, David, 170

Madrigal, Alexis, 273-274

Malthus, Thomas, 178, 235

Manchester Institute of Biotechnology, 16

Mandelbrot set, 30, 300

Margins, 192-194, 196, 241, 242, 243, 307

Markov, Andrei, 153

Markov chain Monte Carlo (MCMC), 164-165, 167, 170, 231, 241, 242, 253, 256

Markov chains, 153-155, 159, 304-305

Markov logic. See Markov logic networks (MLNs)

Markov logic networks (MLNs), 246-259, 309-310

classes and, 257

complexity and, 258-259

parts and, 256-257

with hierarchical structure, 256-257

See also Alchemy

Markov networks, 171-172, 229, 240, 245, 253, 306

Marr, David, 89

Marr’s three levels, 89

Master Algorithm, 239-246

Alchemy and, 250-259

Bayes’ theorem and, 148

brain as, 26-28

CanceRx, 259-261

candidates that fail as, 48-50

chunking and, 226

complexity of, 40-41

as composite picture of current and future learners, 263-264

computer science and, 32-34

equation, 50

evolution and, 28-29

five tribes and, 51-55

future and, 292

goal of, 39

Google and, 282

invention of, 25-26

Markov logic networks and, 236-250

meta-learning and, 237-239

physics and, 29-31

practical applications of, 41-45

statistics and, 31-32

symbolism and, 90-91

theory of everything and, 46-48

Turing point and, 286, 288

as unifier of machine learning, 237

unity of knowledge and, 31

Match.com, 12, 265

Matrix factorization for recommendation systems, 215

Maximum likelihood principle, 166-167, 168

Maxwell, James Clerk, 235

McCulloch, Warren, 96

McKinsey Global Institute, 9

MCMC. See Markov chain Monte Carlo (MCMC)

Means-ends analysis, 225

Mechanical Turk, 14

Medical data, sharing of, 272-273

Medical diagnosis, 23, 147, 149-150, 160, 169, 228-229, 248-249

Memorization, 48

Memory, time as principal component of, 217

Mencken, H. L., 230

Mendeleev, Dmitri, 235

Meta-learning, 237-239, 255, 309

Methane/methanol, 197-198

Michalski, Ryszard, 69, 70, 90

Michelangelo, 2

Microprocessor, 48-49, 236

Microsoft, 9, 22

Kinect, 88, 237, 238

Windows, 12, 133, 224

Xbox Live, 160-161

Microsoft Research, 152

Military robots, 21, 279-282, 299, 310

Mill, John Stuart, 93

Miller, George, 224

Minsky, Marvin, 35, 38, 100-101, 102, 110, 112, 113

Mitchell, Tom, 64, 69, 90

Mixability, 135

MLNs. See Markov logic networks (MLNs)

Moby Dick (Melville), 72

Molecular biology, data and, 14

Moneyball (Lewis), 39

Mooney, Ray, 76

Moore’s law, 287

Moravec, Hans, 288

Muggleton, Steve, 80

Multilayer perceptron, 108-111

autoencoder, 116-118

Bayesian, 170

driving a car and, 113

Master Algorithm and, 244

NETtalk system, 112

reinforcement learning and, 222

support vector machines and, 195

Music composition, case-based reasoning and, 199

Music Genome Project, 171

Mutation, 124, 134-135, 241, 252

Na?ve Bayes classifier, 151-153, 171, 304

Bayesian networks and, 158-159

clustering and, 209

Master Algorithm and, 245

medical diagnosis and, 23

relational learning and, 228-229

spam filters and, 23-24

text classification and, 195-196

Narrative Science, 276

National Security Agency (NSA), 19-20, 232

Natural selection, 28-29, 30, 52

as algorithm, 123-128

Nature

Bayesians and, 141

evolutionaries and, 137-142

symbolists and, 141

Nature (journal), 26

Nature vs. nurture debate, machine learning and, 29, 137-139

Neal, Radford, 170

Nearest-neighbor algorithms, 24, 178-186, 202, 306-307

dimensionality and, 186-190

Negative examples, 67

Netflix, 12-13, 183-184, 237, 266

Netflix Prize, 238, 292

Netscape, 9

NETtalk system, 112

Network effect, 12, 299

Neumann, John von, 72, 123

Neural learning, fitness and, 138-139

Neural networks, 99, 100, 112-114, 122, 204

convolutional, 117-118, 302-303

Master Algorithm and, 240, 244, 245

reinforcement learning and, 222

spin glasses and, 102-103

Neural network structure, Baldwin effect and, 139

Neurons

action potentials and, 95-96, 104-105

Hebb’s rule and, 93-94

McCulloch-Pitts model of, 96-97

processing in brain and, 94-95

See also Perceptron

Neuroscience, Master Algorithm and, 26-28

Newell, Allen, 224-226, 302

Newhouse, Neil, 17

Newman, Mark, 160

Newton, Isaac, 293

attribute selection, 189

laws of, 4, 14, 15, 46, 235

rules of induction, 65-66, 81, 82

Newtonian determinism, Laplace and, 145

Newton phase of science, 39-400

New York Times (newspaper), 115, 117

Ng, Andrew, 117, 297

Nietzche, Friedrich, 178

NIPS. See Conference on Neural Information Processing Systems ((NIPS)

“No free lunch” theorem, 59, 62-65, 70-71

“No Hands Across America,” 113

Noise, 73, 91, 155

Nonlinear dimensionality reduction, 215-217

Nonlinear models, 15, 112-114

Nonuniformity, 189-190

NOR gate, 49

Normal distributions, 187-188, 210

Normative theories, descriptive theories vs., 141-142, 304

Norvig, Peter, 152

NOT gate, 96

NOT operation, 2

Nowlan, Steven, 139

NP-completeness, 32-34, 102

NSA. See National Security Agency (NSA)

Nurture, nature vs., 29, 137-139

Obama, Barack, 17

Objective reality, Bayesians and, 167

Occam’s razor, 77-78, 196, 300-301

OkCupid, 265, 269, 310

O’Keefe, Kevin, 206

On Intelligence (Hawkins), 28, 118

Online analytical processing, 8

Online dating, 265-266, 269, 310

Open-source movement, 45, 279, 292

Optimization, 30-31, 33, 109, 135, 239, 241, 283

constrained, 193-195

O’Reilly, Tim, 9

The Organization of Behavior (Hebb), 93

OR gate, 96

The Origin of Species (Darwin), 28, 123

OR operation, 2

Overfitting, 59, 70-75, 126, 169, 301

avoiding, 76-77

hypotheses and, 73-75

Master Algorithm and, 243

nearest-neighbor algorithm and, 183

noise and, 73

singularity and, 287

support vector machines and, 196

P = NP question, 33-34

PAC learning, 74-75

Page, Larry, 55, 154, 227

PageRank algorithm, 154, 305

PAL (Personalized Assistant that Learns) project, 255

Pandora, 171

Papadimitriou, Christos, 135

Papert, Seymour, 100-101, 102, 110, 112, 113

Parallax effect, 287

Parallel processing, 257-258

Parasites, 135

Pascal, Blaise, 63

Pattern recognition, 8. See also Machine learning

Patterns in data, 70-75

PCA. See Principal-component analysis (PCA)

Pearl, Judea, 156-157, 163, 305

Pens?es (Pascal), 63

Pentagon, 19, 37

Perceptron, 96-101, 108-111, 152, 265. See also Multilayer perceptron

Perceptrons (Minsky & Papert), 100-101, 113

Personal data

ethical responsibility to share some types of, 272-273

as model, 267-270

professional management of, 273-276

sharing or not, 270-276

types of, 271-273

value of, 274

Phase transitions, 105-107, 288

Physical symbol system hypothesis, 89

Physics, 29-31, 46-47, 50

Pitts, Walter, 96

Planetary-scale machine learning, 256-259

Planets, computing duration of year of, 131-133

Plato, 58

Point mutation, 124

Poisson’s equation, 30

Policing, predictive, 20

Politics, machine learning and, 16-19, 299

Positive examples, 67, 69

Posterior probability, 146-147, 241, 242, 243, 249

Poverty of the stimulus argument, 36-37

Power law of practice, 224-225

The Power of Habit (Duhigg), 223

Practice

learning and, 223

power law of, 224-225

Predictive analytics, 8. See also Machine learning

Predictive policing, 20

Presidential election, machine learning and 2012, 16-19

Principal-component analysis (PCA), 211-217, 255, 308

Principia (Newton), 65

Principal components of the data, 214

Principle of association, 93

Principle of indifference, 145

Principle of insufficient reason, 145

Principles of Psychology (James), 93

Prior probability, 146-147

Privacy, personal data and, 275

Probabilistic inference, 52, 53, 161-166, 242, 256, 305

Probability

applied to poetry, 153-154

Bayesian networks and, 156-158

Bayesians and meaning of, 149, 169-170

Bayes’ theorem and, 145-149

estimating, 148-149

frequentist interpretation of, 149

logic and, 173-175, 245-246, 306, 309

Master Algorithm and, 245-246

posterior, 146-147

prior, 146-147

Probability theory, Laplace and, 145

Probably Approximately Correct (Valiant), 75

Problem solving

learning as, 226

theory of, 225

Procedures, learners and, 8

Programming by example, 298

Programming, machine learning vs., 7-8

Programs, 4

computers writing own, 6

survival of the fittest, 131-134

Program trees, 131-133

Prolog programming language, 252-253

Punctuated equilibria, 127, 303

Pushkin, Alexander, 153

Python, 4

Quinlan, J. Ross, 88, 90

Random forest, 238

Rationalists, 57-58

Reasoning, 57-58

analogical, 179, 197

case-based, 197-200, 307

transistors and, 2

Recommendation systems, 12-13, 42, 183-185, 268, 286

Redistribution of income, 278-279

Red Queen hypothesis, 135

Reinforcement learning, 218-223, 226-227, 254, 308

Relational databases, 236

Relational learning, 227-233, 254

Representation

learning algorithms and, 283

Markov logic networks and, 249

Master Algorithm and, 239-240, 241, 243

Retailers, sets of rules and stocking, 69-70

Rewards of states, 218-222

Richardson, Matt, 231, 246

Ridiculograms, 160

Ridley, Matt, 135

RISE algorithm, 201-202, 308

Robotic Park, 121

Robot rights, 285

Robots

empathy-eliciting, 285

evolution of, 121-22, 137, 303

genetic programming and, 133

housebots, 42-43, 218, 255

military, 19-21, 279-282, 299, 310

probabilistic inference and, 166

Romney, Mitt, 17

Rosenberg, Charles, 112

Rosenblatt, Frank, 97, 99, 100, 113

Rosenbloom, Paul, 224-226

Rove, Karl, 17

Rubin, Donald, 209

Rule-based learning, 69-70, 201-202

Rule mining, 301

Rule of succession, 145-146

Rules

filtering spam, 125-127

induction of, 81-82

instances and, 201

Master Algorithm and, 244

sets of, 68-71, 90, 91

See alsoIf… then… rules

Rumelhart, David, 112

Russell, Bertrand, 61

Rutherford, Ernest, 236

Safeway, 272

Saffo, Paul, 106

Sahami, Mehran, 151-152

Saint Paul, 144

Sampling principle, 258

Samuel, Arthur, 219

Sander, Emmanuel, 200

Satisfiability of a logical formula, 33-34, 106

Schapire, Rob, 238

Schemas, 129

Science

analogy and, 178

effect of machine learning on jobs in, 278

frequentism and, 167

machine learning and, 13-16, 235-236, 299

phases of, 39-40

The Sciences of the Artificial (Simon), 41

S curves, 104-107, 111, 249, 252, 287

Search engines, 9, 152, 227-228

Sejnowski, Terry, 103, 112

Selective breeding, genetic algorithms and, 123-124

Self-driving cars. See Driverless cars

Self-organizing systems, 8. See also Machine learning

Semantic network, 255, 309

Sets of classes, 86-87

Sets of concepts, 86-87

Sets of rules, 68-70, 90, 91

power of, 70-71

Sex, 124-126, 134-137

Shannon, Claude, 1-2

Shavlik, Jude, 76

Sigmoid curve. See S curves

Significance tests, 87

Silver, Nate, 17, 238

Similarity, 178, 179

Similarity measures, 192, 197-200, 207

Simon, Herbert, 41, 225-226, 302

Simultaneous localization and mapping (SLAM), 166

Singularity, 28, 186, 286-289, 311

The Singularity Is Near (Kurzweil), 286

Siri, 37, 155, 161-162, 165, 172, 255

SKICAT (sky image cataloging and analysis tool), 15, 299

Skills, learners and, 8, 217-227

Skynet, 282-286

Sloan Digital Sky Survey, 15

Smith, Adam, 58

Snow, John, 183

Soar, chunking in, 226

Social networks, information propagation in, 231

The Society of Mind (Minsky), 35

Space complexity, 5

Spam filters, 23-24, 151-152, 168-169, 171

Sparse autoencoder, 117

Speech recognition, 155, 170-172, 276, 306

Speed, learning algorithms and, 139-142

Spin glasses, brain and, 102-103

Spinoza, Baruch, 58

Squared error, 241, 243

Stacked autoencoder, 117

Stacking, 238, 255, 309

States, value of, 219-221

Statistical algorithms, 8

Statistical learning, 37, 228, 297, 300, 307

Statistical modeling, 8. See also Machine learning

Statistical relational learning, 227-233, 254, 309

Statistical significance tests, 76-77

Statistics, Master Algorithm and, 31-32

Stock market predictions, neural networks and, 112, 302

Stream mining, 258

String theory, 46-47

Structure mapping, 199-200, 254, 307

Succession, rule of, 145-146

The Sun Also Rises (Hemingway), 106

Supervised learning, 209, 214, 220, 222, 226

Support vector machines (SVMs), 53, 179, 190-196, 240, 242, 244, 245, 254, 307

Support vectors, 191-193, 196, 243-244

Surfaces and Essences (Hofstadter & Sander), 200

Survival of the fittest programs, 131-134

Sutton, Rich, 221, 223

SVMs. See Support vector machines (SVMs)

Symbolists/symbolism, 51, 52, 54, 57-91

accuracy and, 75-79

Alchemy and, 251-252

analogizers vs., 200-202

assumptions and, 64

conjunctive concepts, 65-68

connectionists vs., 91, 94-95

decision tree induction, 85-89

further reading, 300-302

hill climbing and, 135

Hume and, 58-59

induction and, 80-83

intelligence and, 52, 89

inverse deduction and, 52, 82-85, 91

Master Algorithm and, 240-241, 242-243

nature and, 141

“no free lunch” theorem, 62-65

overfitting, 70-75

probability and, 173

problem of induction, 59-62

sets of rules, 68-70

Taleb, Nassim, 38, 158

Tamagotchi, 285

Technology

machine learning as, 236-237

sex and evolution of, 136-137

trends in, 21-22

Terrorists, data mining to catch, 232-233

Test set accuracy, 75-76, 78-79

Tetris, 32-33

Text classification, support vector machines and, 195-196

Thalamus, 27

Theory, defined, 46

Theory of cognition, 226

Theory of everything, Master Algorithm and, 46-48

Theory of intelligence, 35

Theory of problem solving, 225

Thinking, Fast and Slow (Kahneman), 141

Thorndike, Edward, 218

Through the Looking Glass (Carroll), 135

Tic-tac-toe, algorithm for, 3-4

Time, as principal component of memory, 217

Time complexity, 5

The Tipping Point (Gladwell), 105-106

Tolstoy, Leo, 66

Training set accuracy, 75-76, 79

Transistors, 1-2

Treaty banning robot warfare, 281

Truth, Bayesians and, 167

Turing, Alan, 34, 35, 286

Turing Award, 75, 156

Turing machine, 34, 250

Turing point, Singularity and, 286, 288

Turing test, 133-134

“Turning the Bayesian crank,” 149

UCI repository of data sets, 76

Uncertainty, 52, 90, 143-175

Unconstrained optimization, 193-194. See also Gradient descent

Underwood, Ben, 26, 299

Unemployment, machine learning and, 278-279

Unified inference algorithm, 256

United Nations, 281

US Patent and Trademark Office, 133

Universal learning algorithm. See Master Algorithm

Universal Turing machine, 34

Uplift modeling, 309

Valiant, Leslie, 75

Value of states, 219-221

Vapnik, Vladimir, 190, 192, 193, 195

Variance, 78-79

Variational inference, 164, 170

Venter, Craig, 289

Vinge, Vernor, 286

Virtual machines, 236

Visual cortex, 26

Viterbi algorithm, 165, 305

Voronoi diagrams, 181, 183

Wake-sleep algorithm, 103-104

Walmart, 11, 69-70

War, cyber-, 19-21, 279-282, 299, 310

War of the Worlds (radio program), 156

Watkins, Chris, 221, 223

Watson, James, 122, 236

Watson, Thomas J., Sr., 219

Watson (computer), 37, 42-43, 219, 237, 238

Wave equation, 30

Web 2.0, 21

Web advertising, 10-11, 160, 305

Weighted k-nearest-neighbor algorithm, 183-185, 190

Weights

attribute, 189

backpropagation and, 111

Master Algorithm and, 242

meta-learning and, 237-238

perceptron’s, 97-99

relational learning and, 229

of support vectors, 192-193

Welles, Orson, 156

Werbos, Paul, 113

Wigner, Eugene, 29

Will, George F., 276

Williams, Ronald, 112

Wilson, E. O., 31

Windows, 12, 133, 224

Wired (magazine), 265

Wizard of Oz problem, 285

Wolpert, David, 62, 238

Word of mouth, 231

Xbox Live, 160-161

XOR. See Exclusive-OR function (XOR)

Yahoo, 10

Yelp, 271, 277

YouTube, 266

Zuckerberg, Mark, 55

Оглавление книги