The Improvement and Application of Xgboost Method Based on the Bayesian Optimization

Li Ye-zi; Wang Zhen-you; Zhou Yi-lu; Han Xiao-zhuo

doi:10.12052/gdutxb.170124

English Version Journal Login

您当前的位置：

首页 >

文章列表页 >

The Improvement and Application of Xgboost Method Based on the Bayesian Optimization

更新时间：2026-05-27

- The Improvement and Application of Xgboost Method Based on the Bayesian Optimization
- Vol. 35, Issue 1, Pages: 23-28(2018)
- 作者机构：
  
  广东工业大学　应用数学学院,广东,广州,510520
- 作者简介：
- 基金信息：
- DOI：10.12052/gdutxb.170124
  CLC： TP18
- Online First：09 January 2018，
  
  Published：2018
- 稿件说明：
移动端阅览
李叶紫, 王振友, 周怡璐, 韩晓卓. 基于贝叶斯最优化的Xgboost算法的改进及应用[J]. 广东工业大学学报, 2018, 35(1): 23-28.

Li Ye-zi, Wang Zhen-you, Zhou Yi-lu, et al. The Improvement and Application of Xgboost Method Based on the Bayesian Optimization[J]. 2018, 35(1): 23-28.
李叶紫, 王振友, 周怡璐, 韩晓卓. 基于贝叶斯最优化的Xgboost算法的改进及应用[J]. 广东工业大学学报, 2018, 35(1): 23-28. DOI： 10.12052/gdutxb.170124.

Li Ye-zi, Wang Zhen-you, Zhou Yi-lu, et al. The Improvement and Application of Xgboost Method Based on the Bayesian Optimization[J]. 2018, 35(1): 23-28. DOI： 10.12052/gdutxb.170124.

摘要

在使用Xgboost框架时

经常涉及各种参数的调整

并且参数组合的选取对模型的分类性能影响较大. 传统的参数寻优方法

通常先导出一个惩罚函数

然后运用经验或者穷举法调整参数值来最大化或最小化这个惩罚函数

但是经常会遇到某个模型没有一个显式的表达式情况. 这类模型的参数寻优就非常麻烦

同时又会给算法带来一定的不确定性和随机性. 本文基于高斯法(GP)的贝叶斯最优化算法对Xgboost框架进行参数寻优

提出了一种新的算法GP_Xgboost

并通过多组数值进行实验. 结果表明本文改进的算法分类效果要优于人工调优和穷举法

从而证明了该算法的可行性和有效性.

Abstract

When the Xgboost framework is in use

it is often involved in the adjustment of various parameters

and the selection of parameters has a great influence on the classification performance of the model. The traditional parameter optimization method usually first derives a penalty function

and then the empirical or exhaustive method is used to adjust the parameter value to maximize or minimize the penalty function

but often encounters a model without an explicit expression. The optimization of the parameters of this model is very troublesome

also bringing some uncertainty and randomness to the algorithm. The Bayesian optimization algorithm based on Gaussian method (GP) is used to optimize the parameters of the Xgboost framework. A new algorithm

GP_Xgboost

is proposed and experimented by multiple sets of numerical values. The results show that the proposed algorithm is superior to the manual tuning and exhaustive method

which proves the feasibility and effectiveness of the proposed algorithm.

关键词

Keywords

references

ZHANG H, JI H C, ZHANG H Y.Application of XGBoost algorithm in E-commerce commodity recommendation[J]. Internet of Things Technology, 2017, 2: 102-104.

江明諺. 大数据下的糖尿病医疗管理-以D診所為例[D]. 高雄: 国立中山大學企業管理學系研究所, 2016: 1-70.

GÓMEZ-RÍOS A, LUENGO J, HERRERA F. A study on the noise label influence in boosting algorithms: AdaBoost, GBM and XGBoost[C]//International Conference on Hybrid Artificial Intelligence Systems.[S.l.]: Springer, 2017: 268-280.

FANG K N, WU J B, ZHU J P, et al . Study on the research on random forest method[J ] . Journal of Statistics and Information, 2011, 26(3): 32-38.

SUN K L, DENG X R. An improved O2O E-commerce recommendation model based on gradient lifting regression algorithm[J]. Journal of Anhui University of Architecture, 2016, 24(2): 87-91.

袁浩杰. Adaboost算法的并行化及其在目标分类中的应用[D]. 广州: 华南理工大学软件学院, 2015.

HE Q, LI N, LUO W J, et al . Overview of machine learning algorithms under large data[J ] . Pattern Recognition and Artificial Intelligence, 2014, 27(4): 327-336.

WANG Z F, WANG Z H. Optimization algorithm of Bayesian network classifier based on derivative of conditional log likelihood function[J]. Journal of Computers, 2012, 35(2): 364-374.

HU Y S, TU X Y, CUI X Y, et al . Reasoning method of uncertainty knowledge based on Bayesian network[J ] . Computer Integrated Manufacturing System, 2001, 7(12): 65-68.

WANG S C, DU R J, LIU Y.Study and optimization of continuous Bayesian classifier with continuous attributes[J]. Journal of Computers, 2012, 35(10): 2129-2138.

HUTTER F, HOOS H H, LEYTONBROWN K. Sequential model-based optimization for general algorithm configuration[C]//International Conference on Learning and Intelligent Optimization.[S.l.]: Springer, 2011.

BERGSTRA J, BARDENET R, BENGIO Y, et al. Algorithms for hyperparameter optimization[C]//NIPS'11 Proceedings of the 24th International Conference on Neural Information Processing Systems. Granada, Spain: ACM, 2011.

ILIEVSKI I, AKHTAR T, FENG J, et al . Efficient hyperparameter optimization for deep learning algorithms using deterministic RBF surrogates[J ] . AAAI, 2017: 822-829.

FRANCESCHI L, DONINI M, FRASCONI P, et al . Forward and Reverse Gradient-Based Hyperparameter Optimization[J ] . arXiv preprint arXiv: 1703.01785, 2017.

XIA Y, LIU C, LI Y Y, et al . A boosted decision tree approach using Bayesian hyper-parameter optimization for credit scoring[J ] . Expert Systems with Applications, 2017, 78: 225-241.

CHEN T, GUESTRIN C. Xgboost: A scalable tree boosting system[C]//Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining.[S.l.]: ACM, 2016: 785-794.

TIANQI C, GUESTRIN C. XGBoost: A Scalable Tree Boosting System[C]//KDD '16 Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. USA, California San Francisco: AMC, 2016.

TORLAY L, PERRONE-BERTOLOTTI M, THOMAS E, et al . Machine learning-XGBoost analysis of language networks to classify patients with epilepsy[J ] . Brain Informatics, 2017: 1-11.

KEJELA G, RONG C. Cross-device consumer identification[C]//Data Mining Workshop (ICDMW), 2015 IEEE International Conference on. USA, Atlantic City: IEEE, 2015: 1687-1689.

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Multi-Scale Contrastive Embedding Enhanced Adaptive Intrusion Detection Model

Energy Management Strategy of Hybrid Locomotive Based on Road Condition Parameters

A Study of Power Quality Regulation Strategies of Energy Storage Converters

A Resource Allocation Optimization Method for Multi-UAV Integrated Sensing, Computing, and Communication Towards Disaster Relief

The Cross-twin Physics-informed Neural Network for Wave Equations with Conserved Quantities

Related Author

No data

Related Institution

No data

AI问答

⁰