赞
踩
贝叶斯优化(Bayesian Optimization)可以用来调整机器学习模型的超参数,使其在给定的问题上表现更好。XGBoost是一个梯度提升树模型,其性能很大程度上取决于超参数的选择。使用贝叶斯优化可以帮助寻找最优的超参数组合,以提高XGBoost模型的性能。
下面是一个使用贝叶斯优化调整XGBoost超参数的示例:
首先,确保安装了必要的库,包括xgboost
, scikit-learn
, hyperopt
和 numpy
。您可以使用以下命令来安装它们:
pip install xgboost scikit-learn hyperopt numpy
- import numpy as np
- from sklearn.datasets import make_classification
- from sklearn.model_selection import train_test_split
- from xgboost import XGBClassifier
- from hyperopt import hp, tpe, fmin, Trials
-
- # 准备虚拟数据集
- X, y = make_classification(n_samples=1000, n_features=20, n_classes=2, random_state=42)
- X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
-
- # 定义超参数搜索空间
- space = {
- 'max_depth': hp.quniform('max_depth', 3, 10, 1),
- 'learning_rate': hp.loguniform('learning_rate', np.log(0.01), np.log(0.5)),
- 'n_estimators': hp.choice('n_estimators', [100, 200, 300, 400, 500]),
- 'gamma': hp.uniform('gamma', 0.0, 0.5),
- 'subsample': hp.uniform('subsample', 0.5, 1.0),
- 'colsample_bytree': hp.uniform('colsample_bytree', 0.5, 1.0),
- }
-
- # 定义优化目标函数(交叉验证)
- def objective(params):
- model = XGBClassifier(**params)
- score = cross_val_score(model, X_train, y_train, cv=5, scoring='accuracy').mean()
- return -score # 负的准确率,因为贝叶斯优化寻找最小值,我们想要最大化准确率
-
- # 运行贝叶斯优化
- trials = Trials()
- best = fmin(fn=objective,
- space=space,
- algo=tpe.suggest,
- max_evals=50, # 迭代次数
- trials=trials)
-
- print("最优超参数:", best)
Copyright © 2003-2013 www.wpsshop.cn 版权所有,并保留所有权利。