排序方式： 共有15条查询结果，搜索用时 171 毫秒
1.
基于BERT嵌入BiLSTM-CRF模型的中文专业术语抽取研究 总被引：2，自引：0，他引：2
专业术语的识别与自动抽取对于提升专业信息检索精度,构建领域知识图谱发挥着重要基础性作用。为进一步提升中文专业术语识别的精确率和召回率,提出一种端到端的不依赖人工特征选择和领域知识,基于谷歌BERT预训练语言模型及中文预训练字嵌入向量,融合BiLSTM和CRF的中文专业术语抽取模型。以自建的1278条深度学习语料数据为实验对象,该模型对术语提取的F1值为92.96%,相对于传统的浅层机器学习模型(如左右熵与互信息算法、word2vec相似词算法等)和BiLSTM-CRF深度神经网络模型的性能有较为显著的提升。本文也给出了模型应用的具体流程,能够为中文专业术语库的构建提供实践指南。 相似文献
2.
[目的/意义] 针对SAO结构短文本分类时面临的语义特征短缺和领域知识不足问题,提出一种融合语义联想和BERT的SAO分类方法,以期提高短文本分类效果。[方法/过程] 以图情领域SAO短文本为数据源,首先设计了一种包含"扩展-重构-降噪"三环节的语义联想方案,即通过语义扩展和SAO重构延展SAO语义信息,通过语义降噪解决扩展后的噪声干扰问题;然后利用BERT模型对语义联想后的SAO短文本进行训练;最后在分类部分实现自动分类。[结果/结论] 在分别对比了不同联想值、学习率和分类器后,实验结果表明当联想值为10、学习率为4e-5时SAO短文本分类效果达到最优,平均F1值为0.852 2,与SVM、LSTM和单纯的BERT相比,F1值分别提高了0.103 1、0.153 8和0.140 5。 相似文献
3.
[目的/意义] 政府网络问政平台是政府部门知晓民意的重要途径之一,为提高问政留言分类的精度以及处理留言数据质量差、数量少等问题,对比多种基于BERT改进模型与文本增强技术结合的分类效果并探究其差异原因。[方法/过程] 设计网络问政留言分类集成对比模型,文本增强方面采用EDA技术与SimBERT文本增强技术进行对比实验,文本分类模型方面则采用多种基于BERT改进的预训练语言模型（如ALBERT、RoBERTa）进行对比实验。[结果/结论] 实验结果表明,基于RoBERTa与SimBERT文本增强的文本分类模型效果最佳,在测试集上的F1值高达92.05%,相比于未进行文本增强的BERT-base模型高出2.89%。同时,SimBERT文本增强后F1值相比未增强前平均提高0.61%。实验证明了基于RoBERTa与SimBERT文本增强模型能够有效提升多类别文本分类的效果,在解决同类问题时具有较强可借鉴性。 相似文献
4.
《Information processing & management》2022,59(5):103058
In this work, we propose BERT-WMAL, a hybrid model that brings together information coming from data through the recent transformer deep learning model and those obtained from a polarized lexicon. The result is a model for sentence polarity that manages to have performances comparable with those at the state-of-the-art, but with the advantage of being able to provide the end-user with an explanation regarding the most important terms involved with the provided prediction. The model has been evaluated on three polarity detection Italian dataset, i.e., SENTIPOLC, AGRITREND and ABSITA. While the first contains 7,410 tweets released for training and 2,000 for testing, the second and the third respectively include 1,000 tweets without splitting , and 2,365 reviews for training, 1,171 for testing. The use of lexicon-based information proves to be effective in terms of the F1 measure since it shows an improvement of F1 score on all the observed dataset: from 0.664 to 0.669 (i.e, 0.772%) on AGRITREND, from 0.728 to 0.734 (i.e., 0.854%) on SENTIPOLC and from 0.904 to 0.921 (i.e, 1.873%) on ABSITA. The usefulness of this model not only depends on its effectiveness in terms of the F1 measure, but also on its ability to generate predictions that are more explainable and especially convincing for the end-users. We evaluated this aspect through a user study involving four native Italian speakers, each evaluating 64 sentences with associated explanations. The results demonstrate the validity of this approach based on a combination of weights of attention extracted from the deep learning model and the linguistic knowledge stored in the WMAL lexicon. These considerations allow us to regard the approach provided in this paper as a promising starting point for further works in this research area. 相似文献
5.
《Information processing & management》2023,60(1):103139
Conversational Recommendation Systems (CRSs) have recently started to leverage pretrained language models (LM) such as BERT for their ability to semantically interpret a wide range of preference statement variations. However, pretrained LMs are prone to intrinsic biases in their training data, which may be exacerbated by biases embedded in domain-specific language data (e.g., user reviews) used to fine-tune LMs for CRSs. We study a simple LM-driven recommendation backbone (termed LMRec) of a CRS to investigate how unintended bias — i.e., bias due to language variations such as name references or indirect indicators of sexual orientation or location that should not affect recommendations — manifests in substantially shifted price and category distributions of restaurant recommendations. For example, offhand mention of names associated with the black community substantially lowers the price distribution of recommended restaurants, while offhand mentions of common male-associated names lead to an increase in recommended alcohol-serving establishments. While these results raise red flags regarding a range of previously undocumented unintended biases that can occur in LM-driven CRSs, there is fortunately a silver lining: we show that train side masking and test side neutralization of non-preferential entities nullifies the observed biases without significantly impacting recommendation performance. 相似文献
6.
《Information processing & management》2023,60(5):103452
Word embeddings, which represent words as numerical vectors in a high-dimensional space, are contextualized by generating a unique vector representation for each sense of a word based on the surrounding words and sentence structure. They are typically generated using such deep learning models as BERT and trained on large amounts of text data and using self-supervised learning techniques. Resulting embeddings are highly effective at capturing the nuances of language, and have been shown to significantly improve the performance of numerous NLP tasks. Word embeddings represent textual records of human thinking, with all the mental relations that we utilize to produce the succession of sentences that make up texts and discourses. Consequently, the distributed representation of words within embeddings ought to capture the reasoning relations that hold texts together. This paper makes its contribution to the field by proposing a benchmark for the assessment of contextualized word embeddings that probes into their capability for true contextualization by inspecting how well they capture resemblance, contrariety, comparability, identity, relations in time and space, causation, analogy, and sense disambiguation. The proposed metrics adopt a triangulation approach, so they use (1) Hume’s reasoning relations, (2) standard analogy, and (3) sense disambiguation. The benchmark has been evaluated against 22 Arabic contextualized embeddings and has proven to be capable of quantifying their differential performance in terms of these reasoning relations. Results of evaluation of the target embeddings revealed that they do take context into account and that they do reasonably well in sense disambiguation but have weakness in their identification of converseness, synonymy, complementarity, and analogy. Results also show that size of an embedding has diminishing returns because the highly frequent language patterns swamp low frequency patterns. Furthermore, the suggest that future research endeavors should not be concerned with the quantity of data as much as its quality, and that it should focus more on the representativeness of data, and on model architecture, design, and training. 相似文献
7.
[目的/意义]为了帮助情报学学科背景的就业人员掌握市场对情报学人才的具体需要,为情报学的教育者拟定情报学的教育体系和人才培养的目标提供指导。[方法/过程]采集国内各大招聘网站情报学相关职位招聘公告,构建情报学招聘语料库,基于CRF机器学习模型和Bi-LSTM-CRF、BERT、BERT-Bi-LSTM-CRF深度学习模型,从语料库中抽取5类情报学招聘实体进行挖掘分析。[结果/结论]通过在已有2000篇经过标注的职位招聘公告语料库上开展情报学招聘实体自动抽取对比实验,识别效果最佳的CRF模型的整体F值为85.07%,其中对"专业要求"实体的识别F值达到了91.67%。BERT模型在"专业要求"实体识别任务中更是取得了92.10%的F值。使用CRF模型对全部符合要求的5287篇招聘公告进行实体抽取,构建了情报学招聘实体社会网络,并通过信息计量分析与社会网络分析的方式挖掘隐含知识。 相似文献
8.
《Information processing & management》2023,60(1):103135
The pre-trained language models (PLMs), such as BERT, have been successfully employed in two-phases ranking pipeline for information retrieval (IR). Meanwhile, recent studies have reported that BERT model is vulnerable to imperceptible textual perturbations on quite a few natural language processing (NLP) tasks. As for IR tasks, current established BERT re-ranker is mainly trained on large-scale and relatively clean dataset, such as MS MARCO, but actually noisy text is more common in real-world scenarios, such as web search. In addition, the impact of within-document textual noises (perturbations) on retrieval effectiveness remains to be investigated, especially on the ranking quality of BERT re-ranker, considering its contextualized nature. To mitigate this gap, we carry out exploratory experiments on the MS MARCO dataset in this work to examine whether BERT re-ranker can still perform well when ranking text with noise. Unfortunately, we observe non-negligible effectiveness degradation of BERT re-ranker over a total of ten different types of synthetic within-document textual noise. Furthermore, to address the effectiveness losses over textual noise, we propose a novel noise-tolerant model, De-Ranker, which is learned by minimizing the distance between the noisy text and its original clean version. Our evaluation on the MS MARCO and TREC 2019–2020 DL datasets demonstrates that De-Ranker can deal with synthetic textual noise more effectively, with 3%–4% performance improvement over vanilla BERT re-ranker. Meanwhile, extensive zero-shot transfer experiments on a total of 18 widely-used IR datasets show that De-Ranker can not only tackle natural noise in real-world text, but also achieve 1.32% improvement on average in terms of cross-domain generalization ability on the BEIR benchmark. 相似文献
9.
Tingting Yang Fei Li Donghong Ji Xiaohui Liang Tian Xie Shuwan Tian Bobo Li Peitong Liang 《Information processing & management》2021,58(6):102681
Depression is a widespread and intractable problem in modern society, which may lead to suicide ideation and behavior. Analyzing depression or suicide based on the posts of social media such as Twitter or Reddit has achieved great progress in recent years. However, most work focuses on English social media and depression prediction is typically formalized as being present or absent. In this paper, we construct a human-annotated dataset for depression analysis via Chinese microblog reviews which includes 6,100 manually-annotated posts. Our dataset includes two fine-grained tasks, namely depression degree prediction and depression cause prediction. The object of the former task is to classify a Microblog post into one of 5 categories based on the depression degree, while the object of the latter one is selecting one or multiple reasons that cause the depression from 7 predefined categories. To set up a benchmark, we design a neural model for joint depression degree and cause prediction, and compare it with several widely-used neural models such as TextCNN, BiLSTM and BERT. Our model outperforms the baselines and achieves at most 65+% F1 for depression degree prediction, 70+% F1 and 90+% AUC for depression cause prediction, which shows that neural models achieve promising results, but there is still room for improvement. Our work can extend the area of social-media-based depression analyses, and our annotated data and code can also facilitate related research. 相似文献
10.
在智慧政务的应用背景下,利用深度学习的方法对海量的科技政策文本数据进行自动分类,可以降低人工处理的成本,提高政策匹配的效率。利用BERT深度学习模型对科技政策进行自动分类实验,通过TextRank算法和TF-IDF算法提取政策文本关键词,将关键词与政策标题融合后输入BERT模型中以优化实验,并对比不同深度学习模型的分类效果来验证该方法的有效性。结果表明,通过BERT模型,融合标题和TF-IDF政策关键词的分类效果最佳,其准确率可达94.41%,证明利用BERT模型在标题的基础上加入政策关键词能够提高政策文本自动分类的准确率,实现对科技政策文本的有效分类。 相似文献