How Do Machines Learn?

Machines are not really intelligent, but they can learn

Machines can mimic intelligence by watching repeated lessons, much like a child learns behaviors from their parents. For this reason, the term Machine Learning (ML) is in many ways more specific and meaningful than “Artificial Intelligence.”

Machine Learning is only a subset of AI, though it is the most exciting part of the field, and the part which has been making the biggest break-throughs recently. However, Machine Learning is not magic. It involves repeating relatively generic math computations on a vast scale that yields uncanny and sometimes unexpected results.

If you already know how to get the right answer, you don’t need machine learning to instruct the computer

As discussed in the previous post, ML is very different from traditional programming.

In traditional programming, you start with some explicit known formulas and logic. The relationship between inputs and outputs (i.e., between what is known and what is to be predicted) has already been derived by someone based on their own analysis and understanding of the world.

The task for the programmer is to translate this relationship into a language the computer can understand, breaking down the problem into very small steps.

For example, if you know that people who apply for a mortgage in which their payments are higher than 50% of their income, are likely to default, then you can simply encode that rule into your loan approval process. When you start adding many rules like this together we call it an expert system. But the learning, or in this case rule generation, is done by the people, not by the computer.

Programmers and designers are the main actors in traditional computer science.

Machine Learning is useful when you can’t explain the specific relationship between inputs and outputs

With Machine Learning, the relationship between inputs and outputs is not explainable or even understood.

In the case of handwriting recognition, a person can identify which number each of the handwritten symbols represent, but they can’t really tell how they know that in a way that a computer can be instructed. With ML, you give the computer pairs of data, inputs and outputs, and it uses mathematical models to derive a connection between them. Note, the computer does not understand the connection (e.g., why this looks like a three), it simply makes the connection. These relationships are inferred from the data.

With ML, mathematicians and data scientists, and in most cases domain experts, are critical actors.

Machine Learning is simply the process of fitting a model that explains the training data but is general enough for the real world

Of course, getting the data is the first step in ML. In subsequent posts we’ll discuss the importance of how to prepare data for the process of machine learning, since this is a big part of the field of data science.

Selecting the correct model type is also essential. The choice of model depends on the needs of the application, since some models are better at solving some sorts of problems. Examples include decision trees and regressions, but one of the most interesting types of models is called an Artificial Neural Network (ANN). ANN are loosely modeled on our brains which comprises a network of billions of neurons connected by synapses.

The training process is where the model gets created. It starts with an “empty” model and populated it with the numbers (aka parameters) perhaps at random. Through an iterative process of applying relatively generic algorithms, the model parameters keep getting adjusted so that the inputs of the training data better fit the predicted output. In the case of an ANN, the training process may need to derive the values for millions or even billions of parameters.

Once training is completed, then the model needs to be evaluated against fresh data (aka test data) to ensure it is working as expected and with an acceptable error rate. Once it has passed this step, then the model is ready to be used in the world, using new data to predict.

An AI is born!

Note: this article is based on a lecture given by me at UC Berkeley, Haas School of Business, Global Access Program, “What Real People Need To Know About Artificial Intelligence” February 15, 2019