基于文本块密度与标签路径等特征的正文提取

杨贤; 唐超兰; 李航

doi:10.12052/gdutxb.170152

欢迎来到能源电力期刊网！

English Version 期刊登录

您当前的位置：

首页 >

文章列表页 >

基于文本块密度与标签路径等特征的正文提取

更新时间：2026-05-27

- 基于文本块密度与标签路径等特征的正文提取
- Text Extraction Based on Text Block Density with Tag Path and Other Features
- 广东工业大学学报 2018年35卷第2期页码：51-56
- 作者机构：
  
  1. 广东工业大学　艺术与设计学院,广东,广州,510090
  2. 广东工业大学　计算机学院,广东,广州,510006
- 作者简介：
  
  杨贤(1982–)，男，助理研究员，主要研究方向为数据挖掘、人机交互
  唐超兰(1969–)，女，教授，主要研究方向为交互设计. E-mail：tangchl@gdut.edu.cn
- 基金信息：
  
  广东省部产学研专项资金企业创新平台资助项目(2013B090800042)
- DOI：10.12052/gdutxb.170152
  中图分类号： TP391
- 纸质出版：2018
- 稿件说明：
移动端阅览
杨贤, 唐超兰, 李航. 基于文本块密度与标签路径等特征的正文提取[J]. 广东工业大学学报, 2018, 35(2): 51-56.

Yang Xian, Tang Chao-lan, Li Hang. Text Extraction Based on Text Block Density with Tag Path and Other Features[J]. 2018, 35(2): 51-56.
杨贤, 唐超兰, 李航. 基于文本块密度与标签路径等特征的正文提取[J]. 广东工业大学学报, 2018, 35(2): 51-56. DOI： 10.12052/gdutxb.170152.

Yang Xian, Tang Chao-lan, Li Hang. Text Extraction Based on Text Block Density with Tag Path and Other Features[J]. 2018, 35(2): 51-56. DOI： 10.12052/gdutxb.170152.

摘要

为了解决网页中除正文信息外还包含网页导航、广告和免责声明等噪声信息的问题

本文提出一种基于标签路径等多特征和文本块密度的正文提取方法. 首先根据文本块密度特征确定正文区域

然后在区域内使用标签路径等特征剔去噪音节点

最后抽取该文本块中的正文节点内容. 该方法有效解决了网页正文块中噪声信息难以过滤和标签路径等特征易对正文部分外较长文本误抽取的问题

且无须训练和人工处理. 从知名网站上随机选取新闻网页数据集进行实验

验证了该方法在不同数据源上都具有很好的适用性

抽取精确度优于CETR、CETD等方法.

Abstract

Most of web pages contain content information as well as a lot of noisy information. In order to address this problem and improve the accuracy of web page extraction

a web page extraction method is proposed via text block density with tap path and other features. The proposed method mostly combines the advantages of text block extraction method and label path extraction method. First

the block of the text is determined according to the density feature of the text block

and then the tag path method is used to remove the noisy node in the block

the text node in the text block is extracted from the content finally. This solution effectively solves the problem that the noisy information in the text block is difficult to filter and the tag path method is easy to extract the long text from the noisy block. In the end

experiments show that the solution is better than CETR and CETD in most cases.

关键词

Keywords

references

HE K D, ZHU Z T, CHENG Y. A research on text classification method based on improved TF-IDF algorithm[J]. Journal of Guangdong University of Technology, 2016, 33(5): 49-53.

LIU P F, QIU X P, HUANG X J, Adversarial multi-task learning for text classification[C] //Annual meeting of the association for computational linguistics. Vancouver: Transactions of The ACL Journal, 2017: 1-10.

FAZZINGA B, FLESCA S, TAGARELLI A. Schema-based web wrapping[J]. Knowledge Information Systems, 2011, 26(1): 127-173.

KIM M, KIM Y, SONG W, et al. Main content extraction from web documents using text block context[C] //Database and expert systems applications. Heidelberg: Springer, 2013: 81-93.

LI P, ZHU J B, ZHOU L X, et al . Shopping information extraction method based on rapid construction of template[J ] . Journal of Computer Applications. 2014, 34(3): 733-737.

YANG X, HE H W. Internet text mining for user intent perception[J]. Journal of Guangdong University of Technology, 2017, 34(3): 54-58.

WENINGER T, HSU W H, HAN J. CETR: content extraction via tag ratios[C] //International conference on world wide web. Raleigh: ACM, 2010: 971-980.

WENINGER T, HSU W H. Text extraction from the web via text-to-tag ratio[C] //International workshop on database and expert systems application. Turin: IEEE, 2008: 23-28.

SUN F, SONG D, LIAO L. DOM based content extraction via text density[C] //International ACM SIGIR conference on research and development in information retrieval. Beijing: ACM, 2011: 245-254.

WU G Q, LI L, HU X G, et al. Web news extraction via tag path feature fusion using DS theory[J]. Journal of Computer Science and Technology, 2016, 31(4): 661-672.

胡骏. 基于标签路径特征的网页正文自适应抽取方法研究[D]. 合肥: 合肥工业大学计算机与信息学院, 2016.

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

人工噪声辅助无人机太赫兹安全通信的鲁棒设计

融合沟道倾斜角的3D NAND存储单元建模及编程策略优化

基于PSI技术的福州市地面形变监测与影响因素分析

储能变流器电能质量治理策略研究

基于数字混频的脉冲式激光测距方法的研究与实现