Machine留学生辅导、讲解AIC、BIC、讲解R编程语言、辅导R
- 首页 >> 其他 Machine Learning Homework 4
Rob McCulloch
2/13/2019
Contents
1. Expectation, Covariance, and Correlation 2
2. Poisson MLE 3
3. Minimum Variance Portfolio 3
4. AIC and BIC in SLR 4
11. Expectation, Covariance, and Correlation
Note: this problem introduces a simple form of factor analysis, which is a very important model.
Let Suppose a person is about to take three tests.
Let Yi be the score on the i
th test, i = 1, 2, 3.
Let Yi = A + Ti
The idea is the A is a general ability and the Ti are test specific abilities.
Let A and all the Ti be independent with, E(A) = μ, E(Ti) = 0
(a)What is E(Yi), i = 1, 2, 3
(b)What is the 3 × 3 variance matrix Σ for Y = (Y1, Y2, Y3)0.
(c)Supppse μ = 100, σ2
A = 100, and σ2
i = 25, i = 1, 2, 3.
What is the correlation matrix of Y ?
(d)Suppose we have the same setup as in (c).
What is the expected value and variance of S = a
0Y where a 0 = (1, 1, 1), so that S is the total score?
(e)Suppose we have the same setup as in (c).
and D = b
0Y where b
0 = (1, .5,.5) so that D is the difference between the score on the first test and the
average score on the other two tests.
What are the mean and variance of D?
(f)Suppose we have the same setup as in (c).
What is σS,D = cov(S, D)?
What is ρS,D?
(g)2Suppose we have the same setup as in (c).
Note that in general
V ar(AY ) = A V ar(Y ) A0
Let A0 = (a, b) and check your previous results by computing A V ar(Y ) A0.
(h) Suppose
Y = μ + β A + T
where Y = (Y1, Y2, . . . , Yp)0, β = (β1, β2, . . . , βp), F is a single random variable with E(F) = 0, V ar(F) = σ2F,
and T = (T1, T2, . . . , Tp) with the Ti
iid, E(Ti) = 0, σ2i = V ar(Ti) = σ2.
Hence V ar(T) = σ2
Ip where Ip is the p × p identity matrix.
What is V ar(Y ) in terms of β and σ. Poisson MLE
Suppose yi
is a count then a very common model is to assume the Poisson disttribuion:
P(Y = y | λ) = eλ λyy!, y = 0, 1, 2, . . .Given Yi ～ P oisson(λ) iid, (that is, Yi = yi), what is the MLE of λ?
3. Minimum Variance Portfolio
Suppose we are considering investing in p stocks where the uncertain return on the i
th stock is denoted by Ri
, i = 1, 2, . . . , p.
Let R = (R1, R2, . . . , Rp)0.
A portfolio is a given by w = (w1, w2, . . . , wp)
0 where wi
is the fraction of wealth invested in asset i.
The {wi} must satisfy Pwi = 1.
The return on the portfolio is then
P = w 0R =XwiRi.
We want to find the global minimum variance portfolio:
min wV ar(P), subject toXwi = 1.
If we let ι = (1, 1, . . . , 1)0
, the vector of ones, and V ar(R) = Σ then our problem is
min 0Σw subject to wι = 1.
Find the global minimum variance portfolio in terms of Σ and ι.
34. AIC and BIC in SLR
Get the midcity housing data from the Rob data page.
md = read.csv("http://www.rob-mcculloch.org/data/midcity.csv")
head(md)
## Home Nbhd Offers SqFt Brick Bedrooms Bathrooms Price
## 1 1 2 2 1790 No 2 2 114300
## 2 2 2 3 2030 No 4 2 114200
## 3 3 2 1 1740 No 3 2 114800
## 4 4 2 3 1980 No 3 2 94700
## 5 5 2 3 2130 No 3 3 119800
## 6 6 1 2 1780 No 3 2 114600
dim(md)
## [1] 128 8
We will use SLR (simple linear regression) to relate house price to house size.
ddf = data.frame(size = md$SqFt/1000,price=md$Price/1000)
plot(ddf$size,ddf$price)
1.6 1.8 2.0 2.2 2.4 2.6
80 120 160 200
ddf$size
ddf$price
As in the notes let
θ2 = (β0, β1, σ).
According the the usual regression ouput:
4lmf = lm(price~size,ddf)
summary(lmf)
##
## Call:
## lm(formula = price ~ size, data = ddf)
##
## Residuals:
## Min 1Q Median 3Q Max
## -46.59 -16.64 -1.61 15.12 54.83
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) -10.091 18.966 -0.532 0.596
## size 70.226 9.426 7.450 1.3e-11 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 22.48 on 126 degrees of freedom
## Multiple R-squared: 0.3058, Adjusted R-squared: 0.3003
## F-statistic: 55.5 on 1 and 126 DF, p-value: 1.302e-11
we might prefer M2.
(a)
According to BIC, which model is preffered.
(b)
According to AIC, which model is preffered.
(c)
Using BIC what is the approximate value of P(M2 | the data).
Rob McCulloch
2/13/2019
Contents
1. Expectation, Covariance, and Correlation 2
2. Poisson MLE 3
3. Minimum Variance Portfolio 3
4. AIC and BIC in SLR 4
11. Expectation, Covariance, and Correlation
Note: this problem introduces a simple form of factor analysis, which is a very important model.
Let Suppose a person is about to take three tests.
Let Yi be the score on the i
th test, i = 1, 2, 3.
Let Yi = A + Ti
The idea is the A is a general ability and the Ti are test specific abilities.
Let A and all the Ti be independent with, E(A) = μ, E(Ti) = 0
(a)What is E(Yi), i = 1, 2, 3
(b)What is the 3 × 3 variance matrix Σ for Y = (Y1, Y2, Y3)0.
(c)Supppse μ = 100, σ2
A = 100, and σ2
i = 25, i = 1, 2, 3.
What is the correlation matrix of Y ?
(d)Suppose we have the same setup as in (c).
What is the expected value and variance of S = a
0Y where a 0 = (1, 1, 1), so that S is the total score?
(e)Suppose we have the same setup as in (c).
and D = b
0Y where b
0 = (1, .5,.5) so that D is the difference between the score on the first test and the
average score on the other two tests.
What are the mean and variance of D?
(f)Suppose we have the same setup as in (c).
What is σS,D = cov(S, D)?
What is ρS,D?
(g)2Suppose we have the same setup as in (c).
Note that in general
V ar(AY ) = A V ar(Y ) A0
Let A0 = (a, b) and check your previous results by computing A V ar(Y ) A0.
(h) Suppose
Y = μ + β A + T
where Y = (Y1, Y2, . . . , Yp)0, β = (β1, β2, . . . , βp), F is a single random variable with E(F) = 0, V ar(F) = σ2F,
and T = (T1, T2, . . . , Tp) with the Ti
iid, E(Ti) = 0, σ2i = V ar(Ti) = σ2.
Hence V ar(T) = σ2
Ip where Ip is the p × p identity matrix.
What is V ar(Y ) in terms of β and σ. Poisson MLE
Suppose yi
is a count then a very common model is to assume the Poisson disttribuion:
P(Y = y | λ) = eλ λyy!, y = 0, 1, 2, . . .Given Yi ～ P oisson(λ) iid, (that is, Yi = yi), what is the MLE of λ?
3. Minimum Variance Portfolio
Suppose we are considering investing in p stocks where the uncertain return on the i
th stock is denoted by Ri
, i = 1, 2, . . . , p.
Let R = (R1, R2, . . . , Rp)0.
A portfolio is a given by w = (w1, w2, . . . , wp)
0 where wi
is the fraction of wealth invested in asset i.
The {wi} must satisfy Pwi = 1.
The return on the portfolio is then
P = w 0R =XwiRi.
We want to find the global minimum variance portfolio:
min wV ar(P), subject toXwi = 1.
If we let ι = (1, 1, . . . , 1)0
, the vector of ones, and V ar(R) = Σ then our problem is
min 0Σw subject to wι = 1.
Find the global minimum variance portfolio in terms of Σ and ι.
34. AIC and BIC in SLR
Get the midcity housing data from the Rob data page.
md = read.csv("http://www.rob-mcculloch.org/data/midcity.csv")
head(md)
## Home Nbhd Offers SqFt Brick Bedrooms Bathrooms Price
## 1 1 2 2 1790 No 2 2 114300
## 2 2 2 3 2030 No 4 2 114200
## 3 3 2 1 1740 No 3 2 114800
## 4 4 2 3 1980 No 3 2 94700
## 5 5 2 3 2130 No 3 3 119800
## 6 6 1 2 1780 No 3 2 114600
dim(md)
## [1] 128 8
We will use SLR (simple linear regression) to relate house price to house size.
ddf = data.frame(size = md$SqFt/1000,price=md$Price/1000)
plot(ddf$size,ddf$price)
1.6 1.8 2.0 2.2 2.4 2.6
80 120 160 200
ddf$size
ddf$price
As in the notes let
θ2 = (β0, β1, σ).
According the the usual regression ouput:
4lmf = lm(price~size,ddf)
summary(lmf)
##
## Call:
## lm(formula = price ~ size, data = ddf)
##
## Residuals:
## Min 1Q Median 3Q Max
## -46.59 -16.64 -1.61 15.12 54.83
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) -10.091 18.966 -0.532 0.596
## size 70.226 9.426 7.450 1.3e-11 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 22.48 on 126 degrees of freedom
## Multiple R-squared: 0.3058, Adjusted R-squared: 0.3003
## F-statistic: 55.5 on 1 and 126 DF, p-value: 1.302e-11
we might prefer M2.
(a)
According to BIC, which model is preffered.
(b)
According to AIC, which model is preffered.
(c)
Using BIC what is the approximate value of P(M2 | the data).