最小二乘法估计boston房屋价格_用最小二乘法解决波士顿房价问题

最小二乘法估计boston房屋价格

线性回归的推导

$$f(\theta ,b)=||A\theta +b-y|{|}_2^2$$
其 中 A θ + b − y = [ x 11 x 12 ⋯ x 1 n x 21 x 22 ⋯ x 2 n ⋮ ⋮ ⋱ ⋮ x n 1 x n 2 ⋯ x n n ] [ θ 1 θ 2 ⋮ θ n ] + [ b 1 b 2 ⋮ b n ] − [ y 1 y 2 ⋮ y n ] 其中A\theta+b-y=

$$\begin{bmatrix}x_{11} & x_{12} & \cdots & x_{1n} \\ x_{21} & x_{22} & \cdots & x_{2n} \\ \vdots & \vdots & \ddots & \vdots\\ x_{n1} & x_{n2} & \cdots & x_{nn} \\ \end{bmatrix}$$ $$\begin{bmatrix}\theta_1\\ \theta_2 \\ \vdots \\\theta_n \\ \end{bmatrix}$$ + $$\begin{bmatrix}b_1\\ b_2\\ \vdots \\b_n \\ \end{bmatrix}$$ - $$\begin{bmatrix}y_1\\ y_2\\ \vdots \\y_n \\ \end{bmatrix}$$ 其中Aθ+b−y=⎣⎢⎢⎢⎡​x11​x21​⋮xn1​​x12​x22​⋮xn2​​⋯⋯⋱⋯​x1n​x2n​⋮xnn​​⎦⎥⎥⎥⎤​⎣⎢⎢⎢⎡​θ1​θ2​⋮θn​​⎦⎥⎥⎥⎤​+⎣⎢⎢⎢⎡​b1​b2​⋮bn​​⎦⎥⎥⎥⎤​−⎣⎢⎢⎢⎡​y1​y2​⋮yn​​⎦⎥⎥⎥⎤​
将上式转化为下式时，需要在A矩阵前加一列数值为1的列向量，
$$A\theta +b-y=\left[\begin{array}{cccc}1& x_{11}& \cdots & x_{1n}\\ 1& x_{21}& \cdots & x_{2n}\\ ⋮& ⋮& \ddots & ⋮\\ 1& x_{n1}& \cdots & x_{nn}\end{array}\right]\left[\begin{array}{c}{\theta }_1\\ {\theta }_2\\ ⋮\\ {\theta }_n\end{array}\right]-\left[\begin{array}{c}{y}_1\\ y_2\\ ⋮\\ y_n\end{array}\right]=\hat{A}\hat{\theta }-y$$
目标函数：
$$f(\hat{\theta })=||\hat{A}\hat{\theta }-y|{|}_2^2$$
目标函数对$\hat{\theta }$求导，求解$argminf(\theta )$:

简化书写，令$\hat{A}=A,\hat{\theta }=\theta$

$$argmin||A\theta -y|{|}_2^2=(A\theta -y{)}^T(A\theta -y)={\theta }^T{A}^{T}A\theta -{\theta }^{T}Ay-y^{T}A\theta +y^{T}y$$
上式对$\theta$求导后,并令其等于0，
$$2A^{T}A\theta -2A^{T}y=0$$
解得，
$$\theta =(A^{T}A{)}^{-1}{A}^{T}y,其中A^{T}A伪逆$$

代码部分：

#载入库
from sklearn import datasets
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_squared_error
import numpy as np
import pandas as pd
import matplotlib as mpl
import matplotlib.pyplot as plt

#提取数据并进行分割训练集和测试集
house = datasets.load_boston()
x = house.data
y = house.target
x_train, x_test, y_train, y_test = train_test_split(x, y, test_size = 0.3)

#调用sklearn中LinerRegression模型
model = LinearRegression()
model.fit(x_train, y_train)
y_sk_pred = model.predict(x_test)
error_1 = mean_squared_error(y_test, y_sk_pred)
print(y_sk_pred)
print(error_1)

#根据最小二乘法建立线性回归模型
class LR:
    def fit(self, X, Y):
        X = np.asmatrix(X.copy())
        Y = np.asmatrix(Y).reshape(-1,1) #列向量
        print(np.shape(X)[1])
        self.w = (X.T * X).I * X.T * Y  #调用最小二乘法求得的系数矩阵
    def predict(slef, X):
        X = np.asmatrix(X.copy())
        result = X * slef.w
        return np.asarray(result).ravel()


#改变输入矩阵，在最前边增加一列
b = np.ones(len(x_train))
c = np.ones(len(x_test))
x_train = np.insert(x_train, 0, values = b, axis = 1)
x_test = np.insert(x_test, 0, values = c, axis = 1)

#调用线性回归函数
lr = LR()
lr.fit(x_train, y_train)
y_lr_pred = lr.predict(x_test)
print(y_lr_pred)
print(lr.w) #系数矩阵theta
error_2 = mean_squared_error(y_test, y_lr_pred)
print(error_2)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52