基于贝叶斯最优化的Xgboost算法的改进及应用

李叶紫; 王振友; 周怡璐; 韩晓卓

doi:10.12052/gdutxb.170124

欢迎来到能源电力期刊网！

English Version 期刊登录

您当前的位置：

首页 >

文章列表页 >

基于贝叶斯最优化的Xgboost算法的改进及应用

更新时间：2026-05-27

- 基于贝叶斯最优化的Xgboost算法的改进及应用
- The Improvement and Application of Xgboost Method Based on the Bayesian Optimization
- 广东工业大学学报 2018年35卷第1期页码：23-28
- 作者机构：
  
  广东工业大学　应用数学学院,广东,广州,510520
- 作者简介：
  
  李叶紫(1993–)，女，硕士研究生，主要研究方向为算法设计与分析、图像处理
  韩晓卓(1978–)，女，副教授，主要研究方向为生物数学，算法设计与分析. E-mail：hanxzh03@163.com
- 基金信息：
  
  国家自然科学基金资助项目(11401115);广州市科技计划项目(201707010435)
- DOI：10.12052/gdutxb.170124
  中图分类号： TP18
- 网络首发：2018-01-09，
  
  纸质出版：2018
- 稿件说明：
移动端阅览
李叶紫, 王振友, 周怡璐, 韩晓卓. 基于贝叶斯最优化的Xgboost算法的改进及应用[J]. 广东工业大学学报, 2018, 35(1): 23-28.

Li Ye-zi, Wang Zhen-you, Zhou Yi-lu, et al. The Improvement and Application of Xgboost Method Based on the Bayesian Optimization[J]. 2018, 35(1): 23-28.
李叶紫, 王振友, 周怡璐, 韩晓卓. 基于贝叶斯最优化的Xgboost算法的改进及应用[J]. 广东工业大学学报, 2018, 35(1): 23-28. DOI： 10.12052/gdutxb.170124.

Li Ye-zi, Wang Zhen-you, Zhou Yi-lu, et al. The Improvement and Application of Xgboost Method Based on the Bayesian Optimization[J]. 2018, 35(1): 23-28. DOI： 10.12052/gdutxb.170124.

摘要

在使用Xgboost框架时

经常涉及各种参数的调整

并且参数组合的选取对模型的分类性能影响较大. 传统的参数寻优方法

通常先导出一个惩罚函数

然后运用经验或者穷举法调整参数值来最大化或最小化这个惩罚函数

但是经常会遇到某个模型没有一个显式的表达式情况. 这类模型的参数寻优就非常麻烦

同时又会给算法带来一定的不确定性和随机性. 本文基于高斯法(GP)的贝叶斯最优化算法对Xgboost框架进行参数寻优

提出了一种新的算法GP_Xgboost

并通过多组数值进行实验. 结果表明本文改进的算法分类效果要优于人工调优和穷举法

从而证明了该算法的可行性和有效性.

Abstract

When the Xgboost framework is in use

it is often involved in the adjustment of various parameters

and the selection of parameters has a great influence on the classification performance of the model. The traditional parameter optimization method usually first derives a penalty function

and then the empirical or exhaustive method is used to adjust the parameter value to maximize or minimize the penalty function

but often encounters a model without an explicit expression. The optimization of the parameters of this model is very troublesome

also bringing some uncertainty and randomness to the algorithm. The Bayesian optimization algorithm based on Gaussian method (GP) is used to optimize the parameters of the Xgboost framework. A new algorithm

GP_Xgboost

is proposed and experimented by multiple sets of numerical values. The results show that the proposed algorithm is superior to the manual tuning and exhaustive method

which proves the feasibility and effectiveness of the proposed algorithm.

关键词

Keywords

references

ZHANG H, JI H C, ZHANG H Y.Application of XGBoost algorithm in E-commerce commodity recommendation[J]. Internet of Things Technology, 2017, 2: 102-104.

江明諺. 大数据下的糖尿病医疗管理-以D診所為例[D]. 高雄: 国立中山大學企業管理學系研究所, 2016: 1-70.

GÓMEZ-RÍOS A, LUENGO J, HERRERA F. A study on the noise label influence in boosting algorithms: AdaBoost, GBM and XGBoost[C]//International Conference on Hybrid Artificial Intelligence Systems.[S.l.]: Springer, 2017: 268-280.

FANG K N, WU J B, ZHU J P, et al . Study on the research on random forest method[J ] . Journal of Statistics and Information, 2011, 26(3): 32-38.

SUN K L, DENG X R. An improved O2O E-commerce recommendation model based on gradient lifting regression algorithm[J]. Journal of Anhui University of Architecture, 2016, 24(2): 87-91.

袁浩杰. Adaboost算法的并行化及其在目标分类中的应用[D]. 广州: 华南理工大学软件学院, 2015.

HE Q, LI N, LUO W J, et al . Overview of machine learning algorithms under large data[J ] . Pattern Recognition and Artificial Intelligence, 2014, 27(4): 327-336.

WANG Z F, WANG Z H. Optimization algorithm of Bayesian network classifier based on derivative of conditional log likelihood function[J]. Journal of Computers, 2012, 35(2): 364-374.

HU Y S, TU X Y, CUI X Y, et al . Reasoning method of uncertainty knowledge based on Bayesian network[J ] . Computer Integrated Manufacturing System, 2001, 7(12): 65-68.

WANG S C, DU R J, LIU Y.Study and optimization of continuous Bayesian classifier with continuous attributes[J]. Journal of Computers, 2012, 35(10): 2129-2138.

HUTTER F, HOOS H H, LEYTONBROWN K. Sequential model-based optimization for general algorithm configuration[C]//International Conference on Learning and Intelligent Optimization.[S.l.]: Springer, 2011.

BERGSTRA J, BARDENET R, BENGIO Y, et al. Algorithms for hyperparameter optimization[C]//NIPS'11 Proceedings of the 24th International Conference on Neural Information Processing Systems. Granada, Spain: ACM, 2011.

ILIEVSKI I, AKHTAR T, FENG J, et al . Efficient hyperparameter optimization for deep learning algorithms using deterministic RBF surrogates[J ] . AAAI, 2017: 822-829.

FRANCESCHI L, DONINI M, FRASCONI P, et al . Forward and Reverse Gradient-Based Hyperparameter Optimization[J ] . arXiv preprint arXiv: 1703.01785, 2017.

XIA Y, LIU C, LI Y Y, et al . A boosted decision tree approach using Bayesian hyper-parameter optimization for credit scoring[J ] . Expert Systems with Applications, 2017, 78: 225-241.

CHEN T, GUESTRIN C. Xgboost: A scalable tree boosting system[C]//Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining.[S.l.]: ACM, 2016: 785-794.

TIANQI C, GUESTRIN C. XGBoost: A Scalable Tree Boosting System[C]//KDD '16 Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. USA, California San Francisco: AMC, 2016.

TORLAY L, PERRONE-BERTOLOTTI M, THOMAS E, et al . Machine learning-XGBoost analysis of language networks to classify patients with epilepsy[J ] . Brain Informatics, 2017: 1-11.

KEJELA G, RONG C. Cross-device consumer identification[C]//Data Mining Workshop (ICDMW), 2015 IEEE International Conference on. USA, Atlantic City: IEEE, 2015: 1687-1689.

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

基于数字混频的脉冲式激光测距方法的研究与实现

基于PSI技术的福州市地面形变监测与影响因素分析

人工噪声辅助无人机太赫兹安全通信的鲁棒设计

具守恒量波动方程的循环双子网物理信息神经网络

面向灾后救援的多无人机通感算一体化资源分配优化方法