Research on Inductive Transfer Learning Model Based on Self-paced Learning Strategy

Zhang Yu; Liu Bo

doi:10.12052/gdutxb.220111

English Version Journal Login

您当前的位置：

首页 >

文章列表页 >

Research on Inductive Transfer Learning Model Based on Self-paced Learning Strategy

更新时间：2026-05-27

- Research on Inductive Transfer Learning Model Based on Self-paced Learning Strategy
- Vol. 40, Issue 4, Pages: 31-36(2023)
- 作者机构：
  
  1. 广东工业大学计算机学院,广东,广州,510006
  2. 广东工业大学自动化学院,广东,广州,510006
- 作者简介：
- 基金信息：
- DOI：10.12052/gdutxb.220111
  CLC： TP301
- Online First：25 July 2023，
  
  Published：2023
- 稿件说明：
移动端阅览
张宇, 刘波. 基于自步学习策略的归纳式迁移学习模型研究[J]. 广东工业大学学报, 2023, 40(4): 31-36.

Zhang Yu, Liu Bo. Research on Inductive Transfer Learning Model Based on Self-paced Learning Strategy[J]. 2023, 40(4): 31-36.
张宇, 刘波. 基于自步学习策略的归纳式迁移学习模型研究[J]. 广东工业大学学报, 2023, 40(4): 31-36. DOI： 10.12052/gdutxb.220111.

Zhang Yu, Liu Bo. Research on Inductive Transfer Learning Model Based on Self-paced Learning Strategy[J]. 2023, 40(4): 31-36. DOI： 10.12052/gdutxb.220111.

摘要

在机器学习任务中很多时候是单任务学习，所以往往会忽略学习任务之间的相关性，并且在单任务学习中容易忽略样本的复杂度，为此，本文提出了一种新的基于自步学习策略的归纳式迁移学习模型，通过对当前多个相关的源任务共享参数学习构建一个预测模型，从而解决目标任务的分类问题。首先提出模型基于自步学习的策略，按照预先设定的自步学习模型参数对多个相关的源任务进行联合学习，利用源任务中样本的损失大小与难易程度对学习的样本赋予一个权重，在迭代的过程中更新自步学习的参数从而挑选出比较合适的样本(损失较小的样本)，然后使用在多个相关的源任务中学习到的知识帮助学习目标任务，构建多个相关迁移学习目标任务的模型，将多个源任务学习到的模型迁移到相关的目标任务中从而提高模型的泛化能力，最后通过拉格朗日函数进一步优化目标模型以提高分类器的性能。实验结果表明，提出的模型在相同的实验条件下优于现有的归纳式迁移学习模型。

Abstract

Machine learning tasks are usually single-task learning. However

in practical applications

the learning tasks are often related. As a result

the correlation between tasks is often ignored

and the complexity of samples is sometimes not considered in single-task learning. To address this

this paper proposes a new inductive transfer learning model based on the self-paced learning strategyby constructing a prediction model to learn the shared parameters of multiple related source tasks. First

the proposed model uses the strategy of self-paced learning to jointly learns the multiple related source tasks

and weights the learning samples according to the loss and difficulty degree of the samples in the source tasks. The parameters of the self-paced learning are iteratively updated to select the optimal samples with the less loss. Then

the knowledge learned from multiple related source tasks guide the learning of target tasks to construct multiple models for related transfer learning target tasks

and transfer these models to related target tasks to improve their generalization ability. Finally

we optimize the target model by using the Lagrange function to improve the performance of the classifiers. Experimental results show that the proposed model is superior to the existing transfer learning model under the same experimental conditions.

关键词

Keywords

references

BENGIO Y, LOURADOUR J, COLLOBERT R, et al. Curriculum learning[C]//Proceedings of the 26th Annual International Conference on Machine Learning. Montreal: ACM, 2009, 382: 41-48.

KUMAR M, PACKER B, KOLLER D. Self-paced learning for latent variable models[C]//Advances in Neural Information Processing Systems 23: 24th Annual Conference on Neural Information Processing Systems 2010. Vancouver: Curran Associates, Inc., 2010: 1189-1197.

BENGIO Y, COURVILLE A C, VINCENT P. Representation learning: a review and new perspectives [J]. Journal of Electronic Imaging, IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(8): 1798-1828.

WANG Y, GAN W, YANG J, et al. Dynamic curriculum learning for imbalanced data classification[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. Seoul: IEEE, 2019: 5017-5026.

SOVIANY P, TUDOR R, IONESCU. Curriculum learning: a survey [J]. International Journal of Computer Vision, 2022, 130(6): 1526-1565.

YIN T, LIU N, SUN H. Self-paced active learning for deep CNNs via effective loss function [J]. Neurocomputing, 2021, 424: 1-8.

YU H, WEN G, GAN J. Self-paced learning for K-means clustering algorithm [J]. Pattern Recognition Letters, 2020, 132: 69-75.

XIAO Y, YANG X, LIU B. A new self-paced method for multiple instance boosting learning [J]. Information Sciences, 2020, 515: 80-90.

PAN S J, YANG Q. A Survey on transfer learning [J]. IEEE Transactions on Knowledge and Data Engineering, 2010, 22(10): 1345-1359.

CRAWFORD M, KHOSHGOFTAAR T M. Using inductive transfer learning to improve hotel review spam detection[C]//2021 IEEE 22nd International Conference on Information Reuse and Integration for Data Science (IRI) . Las Vegas, NV: IEEE, 2021: 248-254.

YUN H, ZHANG C, HOU H, et al . An adaptive approach for ice detection in wind turbine with inductive transfer learning [J ] . IEEE Access, 2019, 7: 122205-122213.

DENG Z, JIANG Y, WANG S, et al. Enhanced knowledge leverage-based TSK fuzzy system modeling for inductive transfer learning[J]. ACM Transactions on Intelligent Systems and Technology, 2016, 8(1) : 11: 1-11: 21.

LI Q X, XIAO Y S, HAO Z F, et al . An algorithm based on multi-task multi-instance anti-noise learning. [J ] . Journal of Guangdong University of Technology, 2018, 35(3): 47-53.

XIAO Y, FENG J, LIU B. A new transductive learning method with universum data [J]. Applied Intelligence, 2021, 51(8): 5571-5583.

XIAO F, PANG L, LAN Y, et al. Transductive learning for unsupervised text style transfer[C]// Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. Punta Cana: Association for Computational Linguistics, 2021, 2510-2521.

ZHAO P, WU G, YAO S, et al . A transductive transfer learning approach based on manifold learning [J ] . Computing in Science and Engineering, 2020, 22(1): 77-87.

MICHAU G, FINK O. Unsupervised transfer learning for anomaly detection: application to complementary operating condition transfer [J]. Knowledge-Based Systems, 2021, 216: 106816.

CHEN X, YANG R, WEN H. Transfer learning with unsupervised domain adaptation method for bearing fault diagnosis[C]//2021 CAA Symposium on Fault Detection, Supervision, and Safety for Technical Processes (SAFEPROCESS) . Chengdu: IEEE, 2021: 1-6.

ROCHA G, CARDOSO H L. Improving transfer learning in unsupervised language adaptation[C]//International Conference on Artificial Neural Networks. Bratislava: Springer, 2021, 12895: 588-599.

MEI B, XU Y. Multi-task least squares twin support vector machine for classification [J]. Neurocomputing, 2019, 338: 26-33.

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Support Vector Machines Based Method to Solve Discrete Linear Differential Equations

Bimodal Iterative Cross-attention Fusion Ensemble Framework

Robust Design for Artificial Noise-assisted UAV Terahertz Secure Communication

SCA-YOLO: Real-time Target Detection Based on Deformable Convolution and Context-aware Attention

Energy Management Strategy of Hybrid Locomotive Based on Road Condition Parameters

Related Author

Wang Dan-rong

Mo Yan

Related Institution

School of Applied Mathematics, Guangdong University of Technology

AI问答

⁰