Although Substack provides a transcription, here’s the original with my formatting.
Hello and welcome to another episode of Practicum AI. I’m Dan Maxwell. In this short presentation, I’m going to talk briefly about the difference between algorithms and models.
Writers are often imprecise and fuzzy when using technical vocabulary. Consider this sentence from a recent article in a peer-reviewed journal. “According to machine learning, the right algorithms can help a computer self-improve as it acquires more experience.” In this example, the author links the word “algorithm” to the phrase “machine learning” – the thought being that algorithms power machine learning. Unfortunately, this is technically unclear.
In computer science, an algorithm is different from a model. Models lie at the heart of machine learning, not algorithms.
Here’s how I might rewrite this sentence in a more precise way. “In machine learning, a good model quickly learns the patterns found in a specific dataset.” Note that I replaced “algorithms” with “model” and “self-improve” with “learn.” But why did I do that? Let’s begin with a closer look at algorithms.
Algorithms have a long and distinguished history in mathematics and computer science. An algorithm is like a recipe. It specifies the steps to achieve a task (making muffins in this case) as well as the required ingredients.
Programmers often write pseudocode to help them understand the logic of a new algorithm. Here, we see the steps for making a cup of tea. In pseudocode, a programmer concisely states what each program step will do. They then specify the how of each step by writing code in C++, Python, or some other language.
Algorithms lie at the heart of traditional programming. As noted earlier, an algorithm is like a recipe. At each step, it needs ingredients (data) and rules (underlying process logic) to arrive at an answer.
The image shown here has been the backbone of development since the beginning of computer science. But it’s limited. This approach works well when we know all the rules and can implement them in code. But what happens when cannot know all the rules? That’s when complexity quickly overwhelms us.
Let’s consider a concrete example…
Consider a case where we want to create a program to predict a person’s activity, given their speed as calculated by a fitness monitor. In this example, a programmer has already implemented an algorithm in code.
Our first program is simple. If the measured speed is less than 4 miles per hour, the program predicts ‘WALKING’ as the activity.
Our second program, however, is a bit more complex. In this case – if the person’s speed is at least 4 miles per hour or greater – the program predicts ‘RUNNING’ as the activity.
Our third program is even more complex. If a person’s speed is 12 miles per hour or greater, the program predicts ‘BIKING’. If the speed is at least 4 miles per hour but less than 12, it predicts ‘RUNNING’. And finally, if the speed is less than 4 miles per hour, it predicts ‘WALKING’.
But what if we want to extend this even further to predict ‘GOLFING’ as the activity. Now, we’re stuck. How do we model that in code? While golfing, a person might walk a bit, stop, do some activity, walk a bit more, stop, and so on. Our algorithm quickly turns into a nightmare. Clearly, our ability to detect this activity using traditional rules has hit a wall. Is there a better way? Well, yes there is. Enter machine learning…
Let’s take another look at our traditional programming diagram. Here, the rules implemented in the code act on the data to give us answers or predictions. In our activity detection example, the data is the speed at which a person is moving. Using that speed, we then designed an algorithm or set of rules to detect their activity, whether walking, biking, or running. However, we hit a wall with golfing because we couldn’t figure out the rules for that activity.
But what would happen if we flipped the axes on this diagram? Rather than figuring out the rules, what if we fed the answers and the data to a model that then discovered those rules? That, in a nutshell, is machine learning.
So, what are the implications of all this? Well – with machine learning – we don’t try to figure out the algorithm or rules. Instead, we first collect a lot of data and label it. We then let the model figure out the rules that make one piece of data match a specific label and another match a different one.
So, how does machine learning solve the complex activity detection problem? Well, the machine learning strategy goes like this. We first collect data from various sensors worn by our research participants. Those sensors might collect heart rate, location, speed, and perspiration. And if we collect this data while our participants are doing various activities, we end up with a dataset that allows the model to “see” what walking looks like, what running looks like, and so on. Now, our job has changed from designing algorithms to creating machine learning models that can match the data to the labels.
Well, what is a model? From a computational perspective, a model is simply a mathematical/statistical representation of reality. Note: a model is not reality. The adage that the “map is not the territory” also holds true for machine learning models. E.P. Box once said, “All models are wrong, but some are useful.” Etch that thought into your mind!
Whereas algorithms execute a predefined sequence of steps to complete a task, machine learning models make predictions. Note: these are two very different things. Neural networks, for example, do not take an algorithmic approach to learning. Instead, they are made of artificial neurons stacked in layers. Imagine a layered cake. During training, the model adjusts the weights between the layers – the frosting in-between – in response to the feedback it receives from its predictions. This is a messy and largely unstructured process, a far cry from the methodical step-by-step approach of the algorithm.
Now you know why I rewrote the sentence from our first slide. In future podcasts, I will talk about neural network anatomy.
Thank you for watching this presentation! If you have any questions or if anything was unclear, please leave me a comment.
Share this post