Multimodal Transformer Learning for Continuous Emotion Recognition | |
Huang, Jian2,3; Tao, Jianhua1,2,3; Liu, Bin3; Lian, Zheng2,3; Niu, Mingyue2,3 | |
2020-05 | |
会议日期 | 2020.5.4-2020.5.8 |
会议地点 | Barcelona, Spain |
英文摘要 | Multimodal fusion increases the performance of emotion recognition because of the complementarity of different modalities. Compared with decision level and feature level fusion, model level fusion makes better use of the advantages of deep neural networks. In this work, we utilize the Transformer model to fuse audio-visual modalities on the model level. Specifically, the multi-head attention produces multimodal emotional intermediate representations from common semantic feature space after encoding audio and visual modalities. Meanwhile, it also can learn long-term temporal dependencies with self-attention mechanism effectively. The experiments, on the AVEC 2017 database, shows the superiority of model level fusion than other fusion strategies. Moreover, we combine the Transformer model and LSTM to further improve the performance, which achieves better results than other methods. |
语种 | 英语 |
内容类型 | 会议论文 |
源URL | [http://ir.ia.ac.cn/handle/173211/39299] |
专题 | 模式识别国家重点实验室_智能交互 |
作者单位 | 1.CAS Center for Excellence in Brain Science and Intelligence Technology, Beijing, China 2.School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China 3.National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences, Beijing, China |
推荐引用方式 GB/T 7714 | Huang, Jian,Tao, Jianhua,Liu, Bin,et al. Multimodal Transformer Learning for Continuous Emotion Recognition[C]. 见:. Barcelona, Spain. 2020.5.4-2020.5.8. |
个性服务 |
查看访问统计 |
相关权益政策 |
暂无数据 |
收藏/分享 |
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。
修改评论