Statistics-Regression analysis

1Presented by: Group-E (The Anonymous)
Rabin B.K
Bimal Pradhan
Bikash Dhakal
Gagan Puri
Bikram Bhurtel

What is regression ?
Simple linear regression
Properties of regression coefficients
Difference between correlation and regression
Measures of variation
Standard error of the estimate
Coefficient of determination
Test of significance of regression coefficients
Example
2

Dictionary meaning: Returning back to previous state
Statistically, it means stepping back towards average
It is statistical tool
Used to determine the relationship among two or more variables
for further estimation
3

Dependent variable
The unknown variable or explained or regressed variable
Independent variable
The known variable or explanatory variable
Linear and non linear regression
If graph between independent and dependent variable is linear
trend then it is linear regression
If graph between independent and dependent variable is not
linear trend then it is non linear regression 4

Let us consider bi-variate distribution (𝑥𝑖 , 𝑦𝑖) , i=1,2,3,….,n , Y
is dependent variable and X is independent variable , then
regression equation yon x .
which is given by : y = a + bx
a and b are constants
a=Y-intercept
b=slope or regression coefficient
Simple Linear Regression
5

Consider the regression model:
y = a + bx + 𝜀 (error)
Major assumption on the random error are:
Regression model is linear in parameter.
E is random real variable.
The random errors E has constant variance.
The random error R have zero mean.
The random variable E is normally distributed.
The explanatory variable x measures without error.
Note: It is considered a serious problem in modeling if any of the above
is violated by the error term.
Assumption of linear regression
6

Correlation is geometric mean of regression coefficient.
If one regression coefficient is greater than unity then the other must be
less than unity.
Regression coefficient are independent of change in origin but not of scale.
Arithmetic mean of regression coefficient is greater than the correlation
coefficient.
The product of two regression coefficient of always equal to 1.
Properties of regression coefficient
r = ∓ 𝑏 𝑥𝑦 . 𝑏 𝑦𝑥
𝑏 𝑥𝑦 > 1 , 𝑏 𝑦𝑥 < 1
𝐴. 𝑀 − 𝑟 ≥ 0
𝑏 𝑥𝑦 . 𝑏 𝑦𝑥 ≤ 1
7
𝑏 𝑥𝑦 = r ×
𝜎 𝑥
𝜎 𝑦
𝑏 𝑦𝑥 = r ×
𝜎 𝑦
𝜎 𝑥

Correlation
a) It is the relationship between two variables.
b) It is not cause an effect of relationship
between variable.
c) It’s coefficient is symmetric,
i.e.
d) Correlation coefficient is a pure number
independent of unit of measurement.
e) It is measure of direction & degree of linear
relationship between variables.
f) It can’t be used in estimating values.
g) It studies only linear relationship between
variables.
Regression
a) It is the average relationship between two
variables.
b) It is cause an effect of relationship between
variable.
c) It’s coefficient isn’t symmetric,
i.e.
d) Regression coefficient are not pure of
measurement.
e) It is functional relationship between variables.
f) It is used to estimate value of depending
variables using value of independent
variables.
g) It studies linear & non linear relationship
between variables.
yxxy rr 
xyyx bb 
9

In regression model value of dependent variable are estimated on
the basis of independent variables
In regression analysis,
Total sum of square (TSS) = sum of square due to regression
(SSR) + sum of square due to error (SSE)
i.e. TSS = SSR + SSE
10
Measure of Variation

For the regression model y = a + bx , where y is dependent
variables & x is independent variables.
Also,
TSS = ∑(Y-Ῡ)²
SSR = ∑(Ŷ-Ῡ)²
SSE = (Υ-Ŷ) (measure of unexplained variation)
(measure of explained variation)
SSE = TSS - SSR
11

Source
of
Variation(SV)
Sum of
Squares(SS) Degree of
freedom (df)
Mean
Square(MS)
[Regression]
Model SSR K(no of
independent
variables)
MSR=SSR/
k
[Residual]
Error SSE n – k-1 MSE=SSE/
n-k-1
Total SST n - 1
test:
p  0.05
SST
12
ANOVA table of regression analysis

 It is a measure of the average variation in data set around
regression line
 The square root of the variance computed from the data set is
standard error
 It is used to measure the reliability of the regression equation
 Regression line is more reliable if standard error of the estimate
is less
 It is given by: Se =
𝑆𝑆𝐸
𝑛−𝑘−1
Where, n = number of observations in the sample and
k = total number of variables in the model
SSE= sum of square due to error
When se= 0, there is no variation in data set around regression line 13
Standard error of the estimate

 It is based on measure of variation
 Measures the proportion of variation in dependent variable that is
explained by the set of independent variables
 Denoted by R2
 It is used to determine the fitness of the data to the regression model
 It is given by: R² =
SSR
TSS
14
Coefficient of determination

For regression equation of y on x
 It’s value lies between 0-1
 R2 = r2 (square of correlation coefficient is R2)
 Higher the value of R2 more reliable is the fitted equation
𝑇𝑆𝑆 = 𝑦 − 𝑦 2 = 𝑦2 − 𝑛 𝑦 2
𝑆𝑆𝐸 = 𝑦2
− 𝑎 𝑦 − 𝑏 𝑥𝑦
𝑆𝑆𝑅 = 𝑇𝑆𝑆 − 𝑆𝑆𝐸
15

x 50 55 55 60 65 70 65 60
y 11 13 14 16 16 15 15 20
x y x2 y2 xy
50 11 2500 121 550
55 13 3025 169 715
55 14 3025 196 770
60 16 3600 256 960
65 16 3600 256 1040
70 15 4900 225 1050
65 15 4225 225 975
60 20 3600 400 1200
Soln.
𝑥 = 480 𝑦 = 120 𝑥2
= 29100 𝑦2
= 1848 𝑥𝑦 = 7260
Example:
16

r =
n xy − x y
n x2 − x 2 n y2 − y 2
𝑟 =
8 × 7260 − 480 × 120
8 × 29100 − 480 2 8 × 1848 − 120 2
𝑟 =
480
48.989 × 19.596
𝑟2
= 0.5 2
= 0.25
The coefficient of determination R2= r2= 0.25
17

Determine whether there is significant linear relationship
between dependent variable and independent variable
Also called as t test
For regression equation:
y= dependent variable
x= independent variable
a= slope intercept
b= regression coefficient of y on x
18
Test of significance of regression coefficients
y = a + bx

Problem to test
H0 : β=0 β = population regression co efficient.
H1 : β≠0
Test Statistics
t=
𝑏
Sb
~ t(n-k-1) df n=no. of observation
k=no. of Independent variable
Sb =
(Y−Y)2
(𝑛−𝑘−1)( (X−X)2 =
MSE
( (X−X)2
19
Different steps in the test

Level of significance
usually take α=0.05 unless we are given
Critical value
obtained from the table according to the level of significance,
degree of freedom and alternative hypothesis
Decision
Reject Hₒ if |t|> ttabulated, accept otherwise
Confidence interval for regression coefficient:
(100-α%) confidence or fiducial limits for regression coefficient
β is given by b ± tₐ(n-k-1)Sb
20

x 1 2 3 4 5
y 5 7 9 10 11
Solution:
x y x² xy y² (x- 𝐱)²
1 5 1 5 25 4
2 7 4 14 49 1
3 9 9 27 81 0
4 10 16 40 100 1
5 11 25 55 121 4
∑x=15 ∑y=42 ∑x²=55 ∑xy=141 ∑y²=376 ∑(x- 𝐱)²=10
Example:
21

To fit y = a + bx
∑y = na + b∑x
42= 5a + 15b ------------(i)
∑xy = a∑x + b∑x²
141 = 15a + 55b --------(ii)
Solving equations (i) and (ii) we get,
a= 3.9 , b= 1.5
Hence, regression equation is y = 3.9 + 1.5x
now,
Problem to test : Hₒ : β=0
Hˌ : β≠0
22

SSE= ∑y² – a∑y – b∑xy = 376-3.9×42-1.5×141 = 0.7
MSE=
SSE
(n−k−1)
= 0.7/(5-1-1) = 0.233
Sb=
MSE
∑(x−x)²
=
0.223
10
= 1.49
Test statistics , t =
b
Sb
=
1.5
0.149
=10.067
Critical value: Let 5% be the level of significance, then t0.05(3) = 3.18
Decision: t=10.067 > t0.05(3)=3.18, reject Hₒ at 5% level of
significance
Conclusion: There is linear relationship between dependent variable y and
independent variable x
23

24
Value of
tabulated t at
5% level of
significance

References:
Probability and Statistics (Vikash Raj Satyal)
Teachers note
google.com
wikipedia.com
25

Statistics-Regression analysis

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Statistics-Regression analysis

Similar to Statistics-Regression analysis (20)

More from Rabin BK

More from Rabin BK (20)

Recently uploaded

Recently uploaded (20)

Statistics-Regression analysis