大学生录取预测——逻辑回归_采用对数几率回归(逻辑回归)模型实现“研究生入学录取预测”

Dataset

每年高中生和大学生都会申请进入到各种各样的高校中去。每个学生都有一组唯一的考试分数，成绩和背景数据。录取委员会根据这个数据决定是否接受这些申请者。在这种情况下一个二元分类算法可用于接受或拒绝申请，逻辑回归是个不错的方法。

• 数据集admissions.csv包含了1000个申请者的信息，特征如下：

gre - Graduate Record Exam（研究生入学考试）, a generalized test for prospective graduate students（一个通用的测试未来的研究生）, continuous between 200 and 800.
gpa - Cumulative grade point average（累积平均绩点）, continuous between 0.0 and 4.0.
admit - Binary variable, 0 or 1, where 1 means the applicant was admitted to the program.

Use Linear Regression To Predict Admission

• 这是原本的数据，admit的值是0或者1。可以发现”gpa”和”admit”并没有线性关系，因为”admit”只取两个值。

import pandas
import matplotlib.pyplot as plt
admissions = pandas.read_csv("admissions.csv")
plt.scatter(admissions["gpa"], admissions["admit"])
plt.show()
1
2
3
4
5

• 这是通过线性回归模型预测的admit的值，发现admit_prediction 取值范围较大，有负值，不是我们想要的。

# The admissions DataFrame is in memory

# Import linear regression class
from sklearn.linear_model import LinearRegression

# Initialize a linear regression model
model = LinearRegression()

# Fit model
model.fit(admissions[['gre', 'gpa']], admissions["admit"])

# Prediction of admission
admit_prediction = model.predict(admissions[['gre', 'gpa']])

# Plot Estimated Function
plt.scatter(admissions["gpa"], admit_prediction)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

• 因此我们期望构造一个模型，能够给我们一个接受（admission）的概率，并且这个概率取值在[0~1]，然后我们根据银行信用卡批准——模型评估ROC&AUC这篇文章的方法来选择合适的阈值进行分类。

The Logit Function

逻辑回归是一个流行的分类方法,它将输出限制在0和1之间。这个输出可以被视为一个给定一组输入某个事件的概率,就像任何其他分类方法。

• logit function是逻辑回归的基础，这个函数的形式如下：

  

• 观察一下logit function的样子：

# Logistic Function
def logit(x):
    # np.exp(x) raises x to the exponential power, ie e^x. e ~= 2.71828
    return np.exp(x) / (1 + np.exp(x)) 

# Linspace is as numpy function to produced evenly spaced numbers over a specified interval.
# Create an array with 50 values between -6 and 6 as t
t = np.linspace(-6,6,50, dtype=float)

# Get logistic fits
ylogit = logit(t)

# plot the logistic function
plt.plot(t, ylogit, label="logistic")
plt.ylabel("Probability")
plt.xlabel("t")
plt.title("Logistic Function")
plt.show()
a = logit(-10)
b = logit(10)
'''
a:4.5397868702434395e-05
b:0.99995460213129761
'''
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

The Logistic Regression

• 逻辑回归就是将线性回归的输出当做Logit Function的输入然后产生一个输出当做最终的概率。其中β0是截距，其他的βi是斜率，也是特征的系数。
  
• 与线性模型一样，我们想要找到最优的βi的值使得预测值与真实值之间的误差最小。通常用来最小化误差的方法是最大似然法和梯度下降法。

Model Data

• 下面进行逻辑回归实验，每次进行训练测试集划分之前，需要将样本数据进行洗牌，这样抽样具有随机性。看到最后的gre和预测值的关系发现，当gre越大时，被接受的概率越大，这是符合实际情况的。

from sklearn.linear_model import LogisticRegression

# Randomly shuffle our data for the training and test set
admissions = admissions.loc[np.random.permutation(admissions.index)]

# train with 700 and test with the following 300, split dataset 
num_train = 700
data_train = admissions[:num_train]
data_test = admissions[num_train:]

# Fit Logistic regression to admit with gpa and gre as features using the training set
logistic_model = LogisticRegression()
logistic_model.fit(data_train[['gpa', 'gre']], data_train['admit'])

# Print the Models Coefficients
print(logistic_model.coef_)
'''
[[ 0.38004023  0.00791207]]
'''

# Predict the chance of admission from those in the training set
fitted_vals = logistic_model.predict_proba(data_train[['gpa', 'gre']])[:,1]
fitted_test = logistic_model.predict_proba(data_test[['gpa', 'gre']])[:,1]

plt.scatter(data_test["gre"], fitted_test)
plt.show()
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

Predictive Power

• 这里有个用法需要提一下，accuracy_train = (predicted == data_train[‘admit’]).mean()中predicted == data_train[‘admit’]得到是一个布尔型array,在计算mean()时，会将True记作1，False记作0，然后求均值。但是在list中是不行的，list对象的布尔型数据没有mean()这个函数。

# .predict() using a threshold of 0.50 by default
predicted = logistic_model.predict(data_train[['gpa','gre']])

# The average of the binary array will give us the accuracy
accuracy_train = (predicted == data_train['admit']).mean()

# Print the accuracy
print("Accuracy in Training Set = {s}".format(s=accuracy_train))
'''
# 这种输出方式也很好
Accuracy in Training Set = 0.7785714285714286
'''
# Percentage of those admitted
percent_admitted = data_test["admit"].mean() * 100

# Predicted to be admitted
predicted = logistic_model.predict(data_test[['gpa','gre']])

# What proportion of our predictions were true
accuracy_test = (predicted == data_test['admit']).mean()
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

• sklearn中的逻辑回归的阈值默认设置为0.5

Admissions ROC Curve

• 逻辑回归中的predict_proba这个函数返回的不是类标签，而是接受的概率，这可以允许我们自己修改阈值。首先我们需要作出它的ROC曲线来观察合适阈值：

from sklearn.metrics import roc_curve, roc_auc_score

# Compute the probabilities predicted by the training and test set
# predict_proba returns probabilies for each class.  We want the second column
train_probs = logistic_model.predict_proba(data_train[['gpa', 'gre']])[:,1]
test_probs = logistic_model.predict_proba(data_test[['gpa', 'gre']])[:,1]
# Compute auc for training set
auc_train = roc_auc_score(data_train["admit"], train_probs)

# Compute auc for test set
auc_test = roc_auc_score(data_test["admit"], test_probs)

# Difference in auc values
auc_diff = auc_train - auc_test

# Compute ROC Curves 
roc_train = roc_curve(data_train["admit"], train_probs)
roc_test = roc_curve(data_test["admit"], test_probs)

# Plot false positives by true positives
plt.plot(roc_train[0], roc_train[1])
plt.plot(roc_test[0], roc_test[1])
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

可以看到ROC曲线开始非常的陡峭，慢慢地变得平缓。测试集的AUC值是0.79小于训练集的AUC值0.82，没有过拟合.这些迹象表明我们的模型可以根据gre和gpa来预测是否录取了。
我们也可以通过银行信用卡批准——模型评估ROC&AUC这篇文章中提到的精确度，查准率，查全率等度量标准来衡量模型的好坏。