Tailoring an Interpretable Neural Language Model | |
Zhang, YK (Zhang, Yike)[ 1,2 ]; Zhang, PY (Zhang, Pengyuan)[ 1,2 ]; Yan, YH (Yan, Yonghong)[ 1,2,3 ] | |
刊名 | IEEE-ACM TRANSACTIONS ON AUDIO SPEECH AND LANGUAGE PROCESSING |
2019 | |
卷号 | 27期号:7页码:1164-1178 |
关键词 | Neural language models interpretability autoregressive moving average speech recognition |
ISSN号 | 2329-9290 |
DOI | 10.1109/TASLP.2019.2913087 |
英文摘要 | Neural networks have shown great potential in language modeling. Currently, the dominant approach to language modeling is based on recurrent neural networks (RNNs) and convolutional neural networks (CNNs). Nonetheless, it is not clear why RNNs and CNNs are suitable for the language modeling task since these neural models are lack of interpretability. The goal of this paper is to tailor an interpretable neural model as an alternative to RNNs and CNNs for the language modeling task. This paper proposes a unified framework for language modeling, which can partly interpret the rationales behind existing language models (LMs). Based on the proposed framework, an interpretable neural language model (INLM) is proposed, including a tailored architectural structure and a tailored learning method for the language modeling task. The proposed INLM can be approximated as a parameterized auto-regressive moving average model and provides interpretability in two aspects: component interpretability and prediction interpretability. Experiments demonstrate that the proposed INLM outperforms some typical neural LMs on several language modeling datasets and on the switchboard speech recognition task. Further experiments also show that the proposed INLM is competitive with the state-of-the-art long short-term memory LMs on the Penn Treebank andWikiText-2 datasets. |
WOS记录号 | WOS:000467568600005 |
内容类型 | 期刊论文 |
源URL | [http://ir.xjipc.cas.cn/handle/365002/5751] |
专题 | 新疆理化技术研究所_多语种信息技术研究室 |
作者单位 | 1.Chinese Acad Sci, Inst Acoust, Beijing 100190, Peoples R China 2.Univ Chinese Acad Sci, Beijing 100049, Peoples R China 3.Chinese Acad Sci, Xinjiang Tech Inst Phys & Chem, Urumqi 830011, Peoples R China |
推荐引用方式 GB/T 7714 | Zhang, YK ,Zhang, PY ,Yan, YH . Tailoring an Interpretable Neural Language Model[J]. IEEE-ACM TRANSACTIONS ON AUDIO SPEECH AND LANGUAGE PROCESSING,2019,27(7):1164-1178. |
APA | Zhang, YK ,Zhang, PY ,&Yan, YH .(2019).Tailoring an Interpretable Neural Language Model.IEEE-ACM TRANSACTIONS ON AUDIO SPEECH AND LANGUAGE PROCESSING,27(7),1164-1178. |
MLA | Zhang, YK ,et al."Tailoring an Interpretable Neural Language Model".IEEE-ACM TRANSACTIONS ON AUDIO SPEECH AND LANGUAGE PROCESSING 27.7(2019):1164-1178. |
个性服务 |
查看访问统计 |
相关权益政策 |
暂无数据 |
收藏/分享 |
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。
修改评论