Reinforcement Learning-driven Multi-round Data Augmentation for Web Service Classification

He Huimin; Chen Chong; Wang Tao; Cheng Lianglun

doi:10.12052/gdutxb.250184

English Version Journal Login

您当前的位置：

首页 >

文章列表页 >

Reinforcement Learning-driven Multi-round Data Augmentation for Web Service Classification

更新时间：2026-08-04

- Reinforcement Learning-driven Multi-round Data Augmentation for Web Service Classification
- Vol. 43, Issue 4, Pages: 80-90(2026)
- 作者机构：
  
  1. 广东工业大学计算机学院,广东,广州,510006
  2. 广东工业大学自动化学院,广东,广州,510006
- 作者简介：
- 基金信息：
- DOI：10.12052/gdutxb.250184
  CLC： TP391
- Published：2026
- 稿件说明：
移动端阅览
何慧敏, 陈翀, 王涛, 等. 多轮强化学习驱动的Web服务增强分类[J]. 广东工业大学学报, 2026, 43(4): 80-90.
何慧敏, 陈翀, 王涛, 等. 多轮强化学习驱动的Web服务增强分类[J]. 广东工业大学学报, 2026, 43(4): 80-90. DOI： 10.12052/gdutxb.250184.

DOI： 10.12052/gdutxb.250184

摘要

针对Web服务分类中长尾分布导致的尾部类别识别困难问题，本文提出一种融合强化学习的多轮数据增强与自适应损失优化方法。该方法以大语言模型为语义生成核心，构建强化学习智能体，在每一轮迭代中根据环境状态自适应地调整不同类别的增强比例与筛选阈值，增强“生成−评估−筛选−回流”的多轮闭环数据；同时结合思维链推理机制，从新颖性、一致性和推理质量等多维度评估生成样本，过滤模板化和语义漂移样本，逐轮优化训练数据分布。在分类器训练阶段，本文基于当轮增强后训练集的类别频次动态计算类权重，并设计Top-

Near-Miss Focal Loss损失函数，联合刻画长尾类别与近错样本，对边界语义进行重点惩罚，从而实现面向长尾与难例的自适应损失优化。在真实长尾Web服务数据集、PMTD(Productive Math Tutoring Dialogue)教学对话数据集上进行实验，结果表明，本文方法在多项评估指标上均优于NCAL(Neural-collapse-advanced Personalized Learning)、RGPT(Recurrent Generative Pre-trained Transformer)、SRaSLR(Social Relation Aware Service Label Recommendation Model)、LLMEmbed等基线模型，尤其在尾部类别识别方面表现突出：在多个轻量级模型上，Web数据集上Macro-$ {{F}}_{1} $提升最高达5个百分点，Weighted-$ {{F}}_{1} $提升约2~3个百分点。本文研究成果有效缓解了长尾数据分布带来的类别偏置问题，为开放环境下语义稀疏服务的智能识别与分类提供了可行路径。

Abstract

To address the difficulty of recognizing tail classes in Web service classification caused by long-tailed data distributions

a reinforcement learning-enhanced framework is proposed

which integrates multi-round data augmentation with adaptive loss optimization

. A large language model (LLM) is employed as the core semantic generator

and a reinforcement learning agent adaptively adjusts class-specific augmentation ratios and filtering thresholds at each iteration based on the observed environment state

driving a closed-loop multi-round process of “generation-evaluation-filtering-feedback.” In parallel

a Chain-of-Thought-based reasoning mechanism is introduced to evaluate generated samples from multiple dimensions－including novelty

semantic consistency

and reasoning quality－thereby filtering out template-like and semantically drifting instances and progressively improving the training data distribution. During classifier training

class weights are dynamically computed from the frequency statistics of the augmented dataset at each iteration. A Top-

Near-Miss Focal Loss is further designed to jointly emphasize long-tailed classes and near-miss boundary samples

penalizing ambiguous semantic regions and enabling adaptive loss optimization tailored to long-tailed and hard examples. Experiments conducted on a real-world long-tailed Web service dataset and the PMTD (Productive Math Tutoring Dialogue) instructional dialogue dataset demonstrate that the proposed method outperforms mainstream baselines such as NCAL (Neural-Collapse-Advanced Personalized Learning)

RGPT

SRaSLR (Social Relation Aware Service Label Recommendation Model) and LLMEmbed across multiple evaluation metrics. In particular

substantial improvements are observed for tail-class recognition: on several lightweight models

Macro-$ {{F}}_{1} $ improves by up to 5 percentage points on the Web service dataset

and Weighted-$ {{F}}_{1} $ increases by approximately 2-3 percentage points. These results verify the effectiveness of the proposed approach in mitigating the bias introduced by long-tailed distributions and provide a practical solution for intelligent recognition and classification of semantically sparse services in open environments.

关键词

Keywords

references

ZOU G B, YANG S, DUAN S Y, et al . Deep LTSC: long-tail service classification via integrating category attentive deep neural network and feature augmentation[J ] . IEEE Transactions on Network and Service Management, 2022, 19(2): 922-935.

WANG K, HU Q, WANG H D, et al . Web service clustering method integrating functional semantics and service collaboration relationships[J ] . Computer Integrated Manufacturing Systems, 2023, 29(4): 1336-1345.

CHEN W X, YANG K X, YU Z W, et al . A survey on imbalanced learning: latest research, applications and future directions[J ] . Artificial Intelligence Review, 2024, 57(6): 137.

KANIKA, SANGEETA. Applying machine learning algorithms for news articles categorization: using SVM and KNN with TF-IDF approach [C] // InSmart Computational Strategies: Theoretical and Practical Aspects. Singapore: Springer Singapore, 2019: 95-105.

YE H F, CAO B Q, PENG Z L, et al . Web services classification based on wide and Bi-LSTM model[J ] . IEEE Access, 2019, 7: 43697-43706.

SAIGAL P, KHANNA V. Multi-category news classification using support vector machine based classifiers[J]. SN Applied Sciences, 2020, 2(3): 458.

YUNITARINI R, GULTOM J F, STEFANY E M. Klasifikasi jamu tradisional Madura menggunakan metode K-nearest neighbors (KNN) dan term frequency-inverse document frequency (TF-IDF) sebagai representasi teks[J]. Jurnal Informatika Polinema, 2024, 11(1): 99-106.

YANG Y L, KE W, WANG W R, et al . Deep learning for web services classification [C ] // 2019 IEEE International Conference on Web Services (ICWS). Milan: IEEE, 2019: 440-442.

TENG S H, DONG P, ZHANG W. Attention-based text summarization model integrating semantic structures[J]. Journal of Guangdong University of Technology, 2021, 38(3): 1-8.

LU H X, EHWERHEMUEPHA L, RAKOVSKI C. A comparative study on deep learning models for text classification of unstructured medical notes with various levels of class imbalance[J]. BMC Medical Research Methodology, 2022, 22(1): 181.

KAMATH C N, BUKHARI S S, DENGEL A. Comparative study between traditional machine learning and deep learning approaches for text classification [C] // Proceedings of the ACM Symposium on Document Engineering 2018. New York: ACM, 2018: 1-11.

NABI H A, FARAJ K H A. Enhanced classification of web services using hybrid meta-heuristic algorithms and deep learning[J]. Expert Systems with Applications, 2025, 279: 127281.

QU P, ZHANG B B, WU J W, et al . Comparison of text classification algorithms based on deep learning[J ] . Journal of Computer Technology and Applied Mathematics, 2024, 1(2): 35-42.

PENG F, PAN G Q, REN Z K, et al . Web service category label recommendation via integrating multi-channel semantic information and attention mechanism[J ] . Computer Integrated Manufacturing Systems, 2025, 31(6): 2215-2225.

ZHU Y Q, LIU M Y, TU Z Y, et al . SRASLR: a novel social relation aware service label recommendation model [C ] // 2021 IEEE International Conference on Web Services (ICWS). Chicago: IEEE, 2021: 87-96.

ZHANG J, CHEN Y, YANG Y L, et al . ServNet-LT: a normalized multi-head deep neural network for long-tailed web services classification [C ] // 2021 IEEE International Conference on Web Services (ICWS). Chicago: IEEE, 2021: 97-106.

ZHAO K S, LIU J, XU Z, et al . Graph4Web: a relation-aware graph attention network for web service classification[J ] . Journal of Systems and Software, 2022, 190: 111324.

ZHANG L, LI R Z, ZHENG S. Short text feature expansion and classification method via label semantic embedding and graph convolution[J]. Journal of Guangdong University of Technology, 2024, 41(1): 69-78.

PENG M, CAO B, CHEN J J, et al . SC-GAT: web services classification based on graph attention network [C ] // International Conference on Collaborative Computing: Networking, Applications and Worksharing. Cham: Springer International Publishing, 2020: 513-529.

LIANG B W, KANG G S, LIU J X, et al . Attentional neural factorization machine for web services classification via exploring content and structural semantics [C ] // 2022 International Joint Conference on Neural Networks (IJCNN). Padua: IEEE, 2022: 1-8.

QI L Y, YAN B Y, ZHENG J Q, et al . A deep graph convolutional network based web API classification approach for mashup creation [C ] // 2023 IEEE International Conference on High Performance Computing Communications, Data Science Systems, Smart City Dependability in Sensor, Cloud Big Data Systems Application (HPCC/DSS/SmartCity/DependSys). Melbourne: IEEE, 2023: 265-272.

ZHANG Z, ZHANG Y, DONG M, et al . Collaborative tag-aware graph neural network for long-tail service recommendation[J ] . IEEE Transactions on Services Computing, 2024, 17(5): 2124-2138.

YIN H R. An industrial short text classification method based on large language model and knowledge base [C] // 2024 International Joint Conference on Neural Networks (IJCNN). Yokohama: IEEE, 2024: 1-7.

LI Y C, DING K Z, WANG J L, et al . Empowering large language models for textual data augmentation [C ] // Findings of the Association for Computational Linguistics: ACL 2024. Bangkok, Thailand: Association for Computational Linguistics, 2024: 12734-12751.

CHEN J F, ZHANG R C, LUO Z Y, et al . Adversarial word dilution as text data augmentation in low-resource regime [C ] // Proceedings of the AAAI Conference on Artificial Intelligence. Washington DC: AAAI Press, 2023, 37(11): 12626-12634.

MEGUELLATI E, ZEGHINA A, SADIQ S, et al . LLM-based semantic augmentation for harmful content detection [C ] // Proceedings of the International AAAI Conference on Web and Social Media. Philadelphia: AAAI Press, 2025, 19: 1190-1209.

PENG L T, ZHANG Y W, SHANG J B. Controllable data augmentation for few-shot text mining with chain-of-thought attribute manipulation [C] // Findings of the Association for Computational Linguistics: ACL 2024. Bangkok: Association for Computational Linguistics, 2024: 1-16.

MOU G Y, LI Y C, LEE K. Reducing and exploiting data augmentation noise through meta reweighting contrastive learning for text classification [C] // 2021 IEEE International Conference on Big Data (Big Data). Orlando: IEEE, 2021: 876-887.

WU H, ZHANG Y B, HAN Z H, et al . CoT-driven framework for short text classification: enhancing and transferring capabilities from large to smaller model[J ] . Knowledge-Based Systems, 2025, 311: 113057.

DUAN Z H, WANG J L. Implicit sentiment analysis based on chain-of-thought prompting [C] // 2024 7th International Conference on Advanced Algorithms and Control Engineering (ICAACE). Shanghai: IEEE, 2024: 368-371.

LIU C, ZHANG H G, ZHAO K N, et al . LLMEmbed: rethinking lightweight LLM’s genuine function in text classification [C ] // Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Bangkok: Association for Computational Linguistics, 2024: 7994-8004.

ZHANG Y Z, WANG M Y, LI Q C, et al . Pushing the limit of LLM capacity for text classification [C ] // Companion Proceedings of the ACM on Web Conference 2025. New York: ACM, 2025: 1524-1528.

HU H L, GUO Y Y, CHEN Z K, et al . Advancing personalized learning with neural collapse for long-tail challenge[C ] // International Conference on Machine Learning. Vancouver : PMLR, 2025: 24314-24327.

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

A Method for Generating Test Cases for Component-based Industrial Software Systems Based on Large Language Models

Deep Reinforcement Learning Based Dialogue Policy Network

Deep Reinforcement Learning-based Spectrum Allocation in Vehicular Networks

Reinforcement Learning-based Automatic Tuning Method and Application of Injection Molding Process Parameters

Meta Reinforcement Learning Based on Skill Discovery

Related Author

Chen Chong

Cheng Lianglun

Wang Tao

Luo Jiale

Xu Boyan

Hao Zhifeng

Zou Zhibin

Wan Pin

Related Institution

College of Engineering, Shantou University

School of Intelligent Manufacturing and Electrical Engineering, Guangzhou Institute of Science and Technology

College of Science, Shantou University

Guangzhou Xuanyuan Research Institute Co., Ltd.

School of Information Engineering, Guangdong University of Technology

AI问答

⁰