 Navigation

Click to Take the FREE Crash-Course

Search... 

How to Think About Machine Learning

by Jason Brownlee on August 15, 2019 in Start Machine Learning  38

Share Tweet Share

Machine learning is a large and interdisciplinary field of study.

You can achieve impressive results with machine learning and find solutions to very challenging problems. But this
is only a small corner of the broader field of machine learning often called predictive modeling or predictive
analytics.

In this post, you will discover how to change the way you think about machine learning in order to best serve you as
a machine learning practitioner.

After reading this post, you will know:

What machine learning is and how it relates to artificial intelligence and statistics.
The corner of machine learning that you should focus on.
How to think about your problem and the machine learning solution to your problem.

Let’s get started.

How to Think About Machine Learning

Photo by Rajarshi MITRA, some rights reserved.
Overview
This post is divided into 3 parts; they are:

1. You’re Confused
2. What is Machine Learning?
3. Your Machine Learning

You’re Confused
You have a machine learning problem to solve, but you’re confused about what exactly machine learning is.

There’s good reason to be confused. It is confusing to beginners.

Machine learning is a large field of study, and not all much of it is going to be relevant to you if you’re focused on
solving a problem.

In this post, I hope to clear things up for you.

We will start off by describing machine learning in the broadest terms and how it relates to other fields of study like
statistics and artificial intelligence.

After that, we will zoom in on the aspects of machine learning that you really need to know about for practical
engineering and problem solving.

What is Machine Learning?

Machine learning is a field of computer science concerned with programs that learn.

The field of machine learning is concerned with the question of how to construct computer programs that
 automatically improve with experience.

— Machine Learning, 1997.

That is super broad.

There are many types of learning, many types of feedback to learn from, and many things that can be learned.

This could encompass diverse types of learning, such as:

Developing code to investigate how populations of organisms “learn” to adapt to their environment over
evolutionary time.
Developing code to investigate how one neuron in the brain “learns” in response to stimulus from other
neurons.
Developing code to investigate how ants “learn” the optimal path from their home to their food source.

I give these esoteric examples on purpose to help you really nail down that machine learning is a broad and far
reaching program of research.

Another case that you may be more familiar with is:

Developing code to investigate how to “learn” patterns in historical data.

This is less glamorous, but is the basis of the small corner of machine learning in which we as practitioners are
deeply interested.

This corner is not distinct from the other examples; there can be a lot of overlap in methods for learning,
fundamental tasks, ways of evaluating learning, and so on.

What About Artificial Intelligence?

Machine learning is a subfield of artificial intelligence.

It overlaps with machine learning.

Artificial intelligence is also an area of computer science, but it is concerned with developing programs that are
intelligent, or can do intelligent things.

Intelligence involves learning, e.g. machine learning, but may involve other concerns such as reasoning, planning,
memory, and much more.

This could encompass diverse types of learning such as:

Developing code to investigate how to optimally plan logistics.

Developing code to investigate how to reason about a paragraph of text.
Developing code to investigate how to perceive the contents of a photograph.

Artificial intelligence is often framed in the context of an agent in an environment with the intent to address some
problem, but this does not have to be the case.

Machine learning could just as easily be named artificial learning to remain consistent with artificial intelligence and
help out beginners.

The lines are blurry. Machine learning problems are also artificial intelligence problems.

What About Statistics?

Statistics, or applied statistics with computers, is a sub-field of mathematics that is concerned with describing and
understanding the relationships in data.

This could encompass diverse types of learning such as:

Developing models to summarize the distribution of a variable.

Developing models to best characterize the relationship between two variables.
Developing models to test the similarity between two populations of observations.

It also overlaps with the corner of machine learning interested in learning patterns in data.

Many methods used for understanding data in statistics can be used in machine learning to learn patterns in data.
These tasks could be called machine learning or applied statistics.

Your Machine Learning

Machine learning is a large field of study, and it can help you solve specific problems.

But you don’t need to know about all of it.

 You’re not an academic investigating an esoteric type of learning as in machine learning.
You’re not trying to make an intelligent agent as in artificial intelligence.
You’re not interested in learning more about why variables relate to each other in data as in statistics.

In fact, when it comes to learning relationships in data:

You’re not investigating the capabilities of an algorithm.

You’re not developing an entirely new theory or algorithm.
You’re not extending an existing machine learning algorithm to new cases.

These may be activities in the corner of machine learning that we may be interested in, but activities for academics,
not practitioners like you.

So what parts of machine learning do you need to focus on?

I think there are two ways to think about machine learning:

1. In terms of the problem you are trying to solve.

2. In terms of the solution you require.

Your Machine Learning Problem

Your problem can best be described as the following:

Find a model or procedure that makes best use of historical data comprised of inputs and outputs in
 order to skillfully predict outputs given new and unseen inputs in the future.

This is super specific.

First of all, it discards entire sub-fields of machine learning, such as unsupervised learning, to focus on one type of
learning called supervised learning and all the algorithms that fit into that bucket.

That does not mean that you cannot leverage unsupervised methods; it just means that you do not focus your
attention there, at least not to begin with.

Second of all, it gives you a clear objective that dominates all others: that is model skill at the expense of other
concerns such as model complexity, model interpretability, and so on.

Again, this does not mean that these are not important, just that they are considered after or in conjunction with
model skill.

Thirdly, the framing of your problem this way fits neatly into another field of study called predictive modeling. That is
a field of study that borrows methods from machine learning with the objective of developing models that make
skillful predictions.

In some areas of business, this area may also be called predictive analytics and encompasses more than just the
modeling component to include related activities of gathering and preparing data and deploying and maintaining the
model.

More recently, this activity can also be called data science, although that phrase also has connotations of inventing
or discovering the problem in addition to working it through to a solution.
I don’t think it matters what you call this activity. But I do think it is important to deeply understand that your interest
in and use of machine learning is highly specific and different from some other uses by academics.

It allows you to filter the material you read and the tools you choose in order to stay focused on the problem you’re
trying to solve.

Your Machine Learning Solution

The solution you require is best described as the following:

A model or procedure that automatically creates the most likely approximation of the unknown underlying
 relationship between inputs and associated outputs in historical data.

Again, this is super specific.

You need an automatic method that produces a program or model that you can use to make predictions.

You cannot sit down and write code to solve your problem. It is entirely data-specific and you have a lot of data.

In fact, problems of this type resist top-down hand-coded solutions. If you could sit down and write some if-
statements to solve your problem, you would not need a machine learning solution. It would be a programming
problem.

The type of machine learning methods that you need will learn the relationship between the inputs and outputs in
your historical data.

This framing allows you to think about what that real underlying yet unknown mapping function might look like and
how noise, corruption, and sampling of your historical data may impact approximations of this mapping made by
different modeling methods.

Without this framing, you will wonder things like:

Why there isn’t just one super algorithm or set of parameters.

Why the experts can’t just tell you what algorithm to use.
Why you can’t achieve a zero error rate with predictions from your model.

It helps you see the ill-defined nature of the predictive modeling problem you’re trying to solve and sets reasonable
expectations.

Next Step
Now that you know how to think about machine learning, the next step is to change the way you think about the
process of solving a problem with a machine learning solution.

For a hint, see the post:

A Gentle Introduction to Applied Machine Learning as a Search Problem

Further Reading
This section provides more resources on the topic if you are looking to go deeper.
Posts
Gentle Introduction to Predictive Modeling
How Machine Learning Algorithms Work
What is Machine Learning?
A Gentle Introduction to Applied Machine Learning as a Search Problem
Where Does Machine Learning Fit In?

Articles
Machine learning on Wikipedia
Predictive modelling on Wikipedia
Predictive analytics on Wikipedia

Summary
In this post, you discovered how to change the way you think about machine learning in order to best serve you as
a machine learning practitioner.

Specifically, you learned:

What machine learning is and how it relates to artificial intelligence and statistics.
The corner of machine learning that you should focus on.
How to think about your problem and the machine learning solution to your problem.

Do you have any questions?

Ask your questions in the comments below and I will do my best to answer.

Share Tweet Share

More On This Topic

Practice Machine Learning with How to Create a Linux Virtual Machine Learning Q&A: Concept
Datasets from the UCI… Machine For Machine… Drift, Better Results…
 About Jason Brownlee
Jason Brownlee, PhD is a machine learning specialist who teaches developers how to get results with modern
machine learning methods via hands-on tutorials.
View all posts by Jason Brownlee →

 Comparing 13 Algorithms on 165 Datasets (hint: use Gradient Boosting) So, You are Working on a Machine Learning Problem… 

38 Responses to How to Think About Machine Learning

REPLY 
Bart April 2, 2018 at 6:51 am #

Hello Jason,

Actually, I have somewhat opposite experience to what you wrote:

“You cannot sit down and write code to solve your problem. It is entirely data-specific and you have a lot of data.”

In fact, the access to data is an issue from my point of view. Do we need another algorithm or another proverbial
neural network layer?

In fact, my feeling is that we are armed with a lot on a programming side. However difficult / scarce access to data is
a bottleneck (as opposed to facebooks / googles / amazons of this world)

I wanted to ask if you have a hint/advice regarding the data access/collections for machine learning problems. What
advice/tricks can you share?

It seems like the dice are loaded, and small organisations (not to mention individuals) cannot compete with the
evilcorps stated above

REPLY 
Jason Brownlee April 2, 2018 at 2:46 pm #

Interesting perspective Bart.

Perhaps get creative? Some ideas:

– Perhaps offer free services to collect your own dataset?

– Perhaps use data augmentation?
– Perhaps use processes/simulations to generate synthetic data?

I hope that stimulates some thoughts.

REPLY 
David Lowe April 5, 2018 at 5:31 am #
 Bart, For practicing machine learning, there are a number of data sources around just waiting to be
tapped. Kaggle is one immediate example that comes to mind. There are many other sources that may
be worthy of your consideration. I keep a list handy on my own website (https://daines-
analytics.com/data-sources/) so I can look them up from time to time.

REPLY 
Jason Brownlee April 5, 2018 at 6:16 am #

Great list.

REPLY 
Randall Gyebi December 28, 2018 at 11:20 pm #

Completely agree. I’m trying to implement machine learning to improve my plant but there so much
ambiguous/lost data it’s hard to start.
’ve now decided to use machine learning itself to help with the data collection with OCR type recognition that
count and register times so I can then work on that collected data.
I can’t comment on how effective this is but I’ll give an update in a few months

REPLY 
Mahesh April 3, 2018 at 5:37 pm #

Thanks for sharing, nice post for the practicle experience in the machine Learning visit Aretove

REPLY 
Jason Brownlee April 4, 2018 at 6:09 am #

Thanks.

REPLY 
babita April 4, 2018 at 9:39 pm #

Thanks for sharing this amazing post on machine learning

REPLY 
Jason Brownlee April 5, 2018 at 5:58 am #

You’re welcome, I hope it helps.

REPLY 
Shravan April 13, 2018 at 1:45 pm #

Being focused is half problem solved!

REPLY 
Jason Brownlee April 13, 2018 at 3:32 pm #
 Indeed, with most things!

REPLY 
Kotrappa Sirbi April 15, 2018 at 1:03 am #

Hello Sir,

Nice article and explained in simple words to make understand what is machine learning, AI ,stats and Data Science.
But sir, no where you mentioned importance of mathematics as basis for Machine Learning . Thanks

REPLY 
Jason Brownlee April 15, 2018 at 6:26 am #

The math can come later. I recommend start by learning how to work through problems end to end, then
circle back to the math later in order to better understand what you’re doing and how to get better results.

REPLY 
Sean April 3, 2019 at 12:43 am #

Thank you Dr. Brownlee,

For me this approach is much more engaging, and it motivates me to want to learn more rather than making
it so daunting as to seem impossible to approach.

REPLY 
Jason Brownlee April 3, 2019 at 6:44 am #

Thanks Sean.

REPLY 
JonS April 26, 2018 at 8:43 am #

Hi, nice post. Do you have any recommended reading for if your focus IS designing an intelligent agent? My
industry is mostly focused on ML algorithms for automated bidding. I’d love to read about some general frameworks
for thinking about univariate TS models on sparse/volatile data

REPLY 
Jason Brownlee April 26, 2018 at 3:02 pm #

I used to work in an area called intelligent agents or multi-agent systems.

It is great technology, you can learn more about it here:

https://en.wikipedia.org/wiki/Multi-agent_system

REPLY 
Arshdeep Singh May 14, 2018 at 3:47 am #

After learning machine learning concepts, algorithms related to it (SVM, logistic regression, ANN, naive
Bayes, random forests, k means, etc) and a basic statistics(descriptive and inferential), what method or approach will
you suggest a student to learn in order to solve the problems in society. What should I learn next? I would love to
master this. Please advise. Thank you!

REPLY 
Jason Brownlee May 14, 2018 at 6:40 am #

I outline my recommended approach to self study here:

https://machinelearningmastery.com/faq/single-faq/how-do-i-self-study-machine-learning

REPLY 
Sam July 16, 2018 at 4:35 pm #

Hi Jason Brownlee, This article is really helpful. I am looking for these type of articles. It cleared my doubt
about Machine Learning.
Good Article.
Thanks, Jason Brownlee.

REPLY 
Jason Brownlee July 17, 2018 at 6:10 am #

Thanks Sam, I’m glad it helped.

REPLY 
shri September 24, 2018 at 11:02 am #

Hi Jason,

Thank you for developing and maintaining this website.

I loved these two definitions:

“Find a model or procedure that makes the best use of historical data comprised of inputs and outputs in order to
skillfully predict outputs given new and unseen inputs in the future.”

“A model or procedure that automatically creates the most likely approximation of the unknown underlying
relationship between inputs and associated outputs in historical data.”

I always thought that predictive analytics or developing a model should not require an in-depth programming
knowledge nor should it require one to be a statistics guru, especially when it comes to solving people-related
problems. (I may be wrong)

I am a total newbie and always been confused about Statistics & ML and where does programming knowledge fit in.

I am in the People Analytics field, understand a bit of statistics and want to use predictive analytics / modeling / ML to
predict employee behavior.

I thought of learning R/ Python and Statistics, but this bottom-up approach seems a bit daunting for me.

Your website has given me some hope.

What do you think is the best way forward for me?

Thanks.

Shri
I have no programming background, do you think I can still What do you think is the best way forward for me?

REPLY 
Jason Brownlee September 24, 2018 at 2:09 pm #

Thanks.

I think your best way forward is with Weka:

https://machinelearningmastery.com/start-here/#weka

REPLY 
JG November 7, 2018 at 4:16 am #

Hi Jason,

Interesting reflections !

In my opinion the big question in ML/DL, also it is the question that a company need to be addressed to the expert
ML software developers is, what kinds of predictive questions can we ask to a (big) DATASET?.
Latter on, technical issued like framing o setting up this (historical dataset) in terms of features, time, pixels, time-
steps, signals, etc. beside the labels associated to each case of subset of the datasets, is coming next. In addition to
define the right ANN models in terms of units, layers, loss, metrics, learning rate, epochs, batch, plus compile it… It is
not easy to see if the question we can make can be managed (or not) by the ML/DL algorithms and the dataset build-
up for each occasion.

This the big challenge for me. Learn to ask questions to a ML process or defining the ML problem !

REPLY 
Jason Brownlee November 7, 2018 at 6:12 am #

Yes, before the model we must have a question.

REPLY 
D_Squared August 14, 2019 at 8:27 pm #

Hey Jason,

Just a heads up… The Predictive Modeling link actually leads to the Predictive Analytics Wiki…

Below is the model link if you want to update. 🙂

https://en.wikipedia.org/wiki/Predictive_modelling

REPLY 
Jason Brownlee August 15, 2019 at 8:06 am #

Thanks. Fixed.

REPLY 
Shruti pandey January 8, 2020 at 3:05 am #
Thank you so much for the post. It is really helpful for people like me who are starting on their own.

REPLY 
Jason Brownlee January 8, 2020 at 8:31 am #

You’re welcome.

REPLY 
Neha Boddolla May 14, 2020 at 4:47 pm #

Yeah, somewhat relatable…I love your examples

REPLY 
Jason Brownlee May 15, 2020 at 5:56 am #

Thanks.

REPLY 
Gemma Almendros June 13, 2020 at 12:33 am #

Hi Jason, I know that maybe my question has nothing to do with the ones that this forum frequents, but the
thing is that I have a job from University on terminology and by the terms they have given us, we are pretty sure that
it’s about Machine Learning.

However, my colleagues and I lack so much knowledge on the subject and we find it difficult to relate the terms, with
which we must develop an outline, these are the terms:

-Risk function *
-Optimal error *
-Composite outlier*
-Detection
-Nonparametric
-Decentralized detection *
-Multitask linear *
-Regression

Some of these terms we have been able to understand, sadly the ones with the asteriscs are the one that we have
more trouble with.

We have been documenting ourselves for a week but we still do not find articles or texts that clearly define these
concepts, since most of what we find are graduate works, where they only focus on some parts of Machine Learning.

We would appreciate it if you could give us some details about these terms or how they can be related to each other.

Thank you very much,

Gemma.

P.D: We also want to tell you that this blog has helped us a lot to understand certain aspects of Machine Learning in
a more pleasant way.

REPLY 
Jason Brownlee June 13, 2020 at 6:07 am #
 I’m eager to help, but I don’t have the capacity to research these terms for you. Perhaps you can
find/hire an academic to complete the task?

REPLY 
Gemma Almendros June 13, 2020 at 7:40 pm #

I don’t think that’s possible for us as we don’t know anyone who is specialized in these areas of
study, we will keep doing more research on these terms, but the problem is that we are spanish and studying
translation so we dont really know anything about this topic.

Thank you for replying Jason, have a good day.

REPLY 
maneesha p July 4, 2020 at 12:16 am #

This platform has been us to know about the machine learning…

Jason Brownlee July 4, 2020 at 6:01 am #

Thanks!

REPLY 
Jean-Christophe Chouinard February 23, 2021 at 10:13 am #

Thanks Jason, I like that you create that frame around what to start learning with. Unsupervised learning for
predictive analytics seems like a field that can reach everyone starting in machine learning. Thanks for the good
work!

REPLY 
Jason Brownlee February 23, 2021 at 10:15 am #

I’m happy it helped.

Leave a Reply

Name (required)
 Email (will not be published) (required)

SUBMIT COMMENT

Welcome!
I'm Jason Brownlee PhD
and I help developers get results with machine learning.
Read more

Never miss a tutorial:

Picked for you:

Find Your Machine Learning Tribe

What Is Holding You Back From Your Machine Learning Goals?

Difference Between Classification and Regression in Machine Learning

Machine Learning for Developers

Why Machine Learning Does Not Have to Be So Hard

 Loving the Tutorials?

The EBook Catalog is where

you'll find the Really Good stuff.

>> SEE WHAT'S INSIDE

© 2024 Guiding Tech Media. All Rights Reserved.

LinkedIn | Twitter | Facebook | Newsletter | RSS

Privacy | Disclaimer | Terms | Contact | Sitemap | Search