Cross-Modal Multitask Transformer for End-to-End Multimodal Aspect-Based Sentiment Analysis |
| |
| Institution: | 1. West China Biomedical Big Data Center, West China Hospital, Sichuan University, No.37 Guoxue Alley, Chengdu 610041, China;2. Department of Radiology, West China Hospital, Sichuan University, No.37 Guoxue Alley, Chengdu 610041, China;3. West China Periodicals, West China Hospital, Sichuan University, No.37 Guoxue Alley, Chengdu 610041, China;4. Department of Bile Duct Surgery, West China Hospital, Sichuan University, No.37 Guoxue Alley, Chengdu 610041, China |
| |
| Abstract: | As an emerging task in opinion mining, End-to-End Multimodal Aspect-Based Sentiment Analysis (MABSA) aims to extract all the aspect-sentiment pairs mentioned in a pair of sentence and image. Most existing methods of MABSA do not explicitly incorporate aspect and sentiment information in their textual and visual representations and fail to consider the different contributions of visual representations to each word or aspect in the text. To tackle these limitations, we propose a multi-task learning framework named Cross-Modal Multitask Transformer (CMMT), which incorporates two auxiliary tasks to learn the aspect/sentiment-aware intra-modal representations and introduces a Text-Guided Cross-Modal Interaction Module to dynamically control the contributions of the visual information to the representation of each word in the inter-modal interaction. Experimental results demonstrate that CMMT consistently outperforms the state-of-the-art approach JML by 3.1, 3.3, and 4.1 absolute percentage points on three Twitter datasets for the End-to-End MABSA task, respectively. Moreover, further analysis shows that CMMT is superior to comparison systems in both aspect extraction (AE) and sentiment classification (SC), which would move the development of multimodal AE and SC algorithms forward with improved performance. |
| |
| Keywords: | Fine-grained opinion mining Aspect-Based Sentiment Analysis Multimodal Sentiment Analysis |
| 本文献已被 ScienceDirect 等数据库收录！ |
|
