Introduction to Machine Learning

HANDS-ON ONLINE TRAINING
ON DATA SCIENCE
KNOWLEDGE AND SKILLS FORUM
INTRODUCTION TO
MACHINE LEARNING
FAITHFUL ONWUEGBUCHE
OLUWASEUN ODEYEMI
AUGUSTINE OKOLIE

WHAT IS MACHINE
LEARNING
• Machine learning is a type of artificial
intelligence (AI) that enables computers to
learn and make decisions without being
explicitly programmed.
• Using algorithms that iteratively learn
from data, machine learning allows
computers to find hidden insights without
being explicitly programmed where to
look.

MACHINE
LEARNING VS.
STATISTICS
VS.
COMPUTER
SCIENCE
Machine Learning Statistics Computer Science
Objective
Focuses on learning from
data to make predictions or
decisions without being
explicitly programmed. It
prioritizes prediction
accuracy and
generalizability.
Aims to infer properties of
an underlying distribution
from a data sample. It
emphasizes understanding
and interpreting data and
probabilistic models.
Focuses on the
creation and
application of
algorithms to
manipulate, store, and
communicate digital
information.
Methodologie
s
Typically uses complex
models (like neural
networks) and large
amounts of data to train
models for prediction.
Utilizes both supervised
and unsupervised learning
methods.
Often employs simpler,
more interpretable
models. Focuses on
hypothesis testing,
experimental design,
estimation, and
mathematical analysis.
Involves algorithm
design, data
structures,
computation theory,
computer architecture,
software development,
and more.
Validation
Measures model
performance through
methods like cross-
validation and seeks to
improve generalization to
unseen data.
Validates models using
methods such as
confidence intervals, p-
values, and hypothesis
tests to quantify
uncertainty.
Uses formal methods
for verifying
correctness, analyzing
computational
complexity, and
proving algorithmic
bounds.
Primary
Concern
Creating models that can
learn from and make
decisions or predictions
based on data.
Drawing valid conclusions
and quantifying
uncertainty about
observed data and
underlying distributions.
Creating efficient
algorithms and data
structures to solve
computational
problems.

TYPES OF MACHINE LEARNING
Three Types of Problems
• Supervised
• Unsupervised
• Reinforcement

SUPERVISE
D
• Trained using labeled examples
• Desired output is known
• Methods include classification, regression, etc.
• Uses patterns to predict the values of the label on
additional unlabeled data
• Algorithms:
• Linear regression
• Logistic regression
• K-Nearest Neighbors (KNN)
• Decision Trees and Random Forests
• Support Vector Machines
• Naive Bayes
• Neural Networks

UNSUPERVIS
ED
• Used against data that has no historical labels
• Desired output is unknown
• Goal is to explore the data and find some
structure within the data
• Algorithms
• Anomaly detection
• K-means clustering
• Hierarchical clustering
• DBSCAN
• Principal Component Analysis (PCA)
• Neural Networks

REINFORCEMEN
T
• Algorithm discovers through trial and error
which actions yield the greatest rewards.
• Three primary components:
• the agent (the learner or decision maker),
• the environment (everything the agent
interacts with)
• actions (what the agent can do).
• Objective: the agent chooses actions that
maximize the expected reward over a given
amount of time.
• Algorithms
• Markov Decision Process
• Q-Learning
• Deep Q Network (DQN)

WHY USE IT?
• Machine learning based models can extract patterns from massive
amounts of data which humans cannot do because
• We cannot retain everything in memory or we cannot perform
obvious/redundant computations for hours and days to come up
with interesting patterns.
• “Humans can typically create one or two good models in a week;
machine learning can create thousands of models in a week” (Thomas
H. Davenport)
• Solve problems we simply could not before

USE CASES
• Email spam filter
• Recommendation systems
• Self driving car
• Finance
• Image Recognition
• Competitive machines

TYPICAL MACHINE LEARNING PROCESS
Source: INTRODUCING AZURE MACHINE
LEARNING, pg. 5

TO GIVE CREDIT, OR NOT TO GIVE CREDIT
• You are asked by your boss while working at Big Bank Inc. to
develop an automated decision maker on whether to give a
potential client credit or not.

WHAT IS THE QUESTION?
• What question are we trying to answer here? What problem are
we looking to solve?

SELECTING DATA
Feature
• An individual measurable property of a phenomenon being
observed
• Best found through industry experts

SELECTING DATA
Feature Extraction
• Feature extraction is a general term for methods of
constructing combinations of the variables to get around
certain problems while still describing the data with sufficient
accuracy.
• Analysis with a large number of variables generally requires a
large amount of memory and computation.
• Reducing the amount of resources required to describe a
large set of data

SELECTING DATA
PCA (Principal ComponentAnalysis)
• We have a huge list of different features
• Many of them will measure related properties and so will be
redundant
• Summarize with less features

PREPARING DATA
• Cleaning
• Units
• Missing Values
• Metadata

DEVELOPING MODEL
• What is the problem being solved?
• What is the goal of the model?
• Minimize error on the “training” data
• Training data is the data used to train the model (all of it but the part we
removed)

DEVELOPING MODEL
Linear Model
• Relationships are modeled using linear predictor functions
whose unknown model parameters are estimated from the data
• Complex way of saying the model draws a line between two
categories (classification) or to estimate a value (regression)
• Linear regression the most common form of linear model

DEVELOPING MODEL
Non-linear Model
• A nonlinear model describes nonlinear relationships in
experimental data
• The parameters can take the form of an exponential,
trigonometric, power, or any other nonlinear function

DEVELOPING
MODEL
Overfitting vs. Bias in Machine
Learning
Overfitting Bias
Definition
Overfitting occurs when a
model fits the data more
than is warranted. It
captures the noise along
with the underlying
pattern in the data.
Bias is error from
erroneous assumptions in
the learning algorithm.
High bias can cause
an algorithm to miss
relevant relations between
features and
target outputs.
Consequence
Overfitting leads to a
smaller error on the
training data set but
a larger one on unseen
data, reducing the model's
ability to generalize.
High bias often leads
to underfitting, where the
model oversimplifies the
data and doesn't capture
its complexity.
Example
Creating a very complex
decision tree that classifies
each
training instance perfectly,
but performs poorly on
unseen data.
Fitting a quadratic dataset
using a linear model – the
model will consistently fail
to capture the
true relationship and make
errors.

DEVELOPING MODEL
Overfitting vs. Bias in Machine
Learning

DEVELOPING MODEL
Rule Addition
• Minimize error on the “training” data
• AND make sure that error in the “unseen” data is close to error in
the “training” data

DEVELOPING A MODEL
• Keep it Simple
• Go for simpler models over more complicated models
• Generally, the fewer parameters that you have to tune the better
• Cross-Validation
• K-fold cross validation is a great way to estimate error on training data
• Regularization
• Can sometimes help penalize certain sources of overfitting.
• LASSO
• Forces the sum of the absolute value of coefficients to be less than a
fixed value
• Effectively choosing a simpler model

DEVELOPIN
G A MODEL
Data Snooping or Data Dredging
• It's a form of bias that arises when you make decisions based on
the same data you've used to train and test your model.
• “If a data set has affected any step in the learning process, its
ability to access the outcome has been compromised”
• Experimenting
• Reuse of the same data set to determine quality of model
• Once a data set has been used to test the performance of a
data set, it should be considered contaminated
Source: Learning From Data, pg. 1

INTERPRETING RESULTS
• Validation
• Cross validation
• Test set
• Once the test set has been used, you must find new data!

Introduction to Machine Learning

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Introduction to Machine Learning

Similar to Introduction to Machine Learning (20)

Recently uploaded

Recently uploaded (20)

Introduction to Machine Learning

Editor's Notes