An image-text consistency driven multimodal sentiment analysis approach for social media |
| |
| Institution: | 1. Institute of Systems Science, National University of Singapore, Singapore 119615, Singapore;2. Hubei Province Key Laboratory of Intelligent Information Processing and Real-time Industrial System, School of Computer Science and Technology, Wuhan University of Science and Technology, Wuhan, 430081, China;1. Department of Computing Science and Mathematics, University of Stirling, UK;2. School of Computer Science and Engineering, Nanyang Technological University, Singapore;3. Computational Neuroscience and Functional Neurosurgery, University of Oxford, UK;1. University of Science and Technology of China, Hefei, Anhui 230027, China;2. Microsoft Research Asia, Beijing 100080, China;3. JD AI Research, Beijing 101111, China;1. College of Computer Science and Technology, Hubei Province Key Laboratory of Intelligent Information Processing and Real-time Industrial System, Wuhan University of Science and Technology, Wuhan 430065, China;2. School of Computer and Information, Hefei University of Technology, Hefei 230009, China;3. Jiangsu Engineering Center of Network Monitoring, Nanjing University of Information Science and Technology, Nanjing 210044, China;1. Department of Computer Science & Engineering, Delhi Technological University, Delhi, India;2. School of Information Technology and Engineering, Vellore Institute of Technology, Vellore, Tamil Nadu, India;3. Department of Electronics Engineering & Institute of Electronics, National Chiao Tung University, Taiwan;4. School of Computer Science, University of Sydney, NSW, Australia |
| |
| Abstract: | Social media users are increasingly using both images and text to express their opinions and share their experiences, instead of only using text in the conventional social media. Consequently, the conventional text-based sentiment analysis has evolved into more complicated studies of multimodal sentiment analysis. To tackle the challenge of how to effectively exploit the information from both visual content and textual content from image-text posts, this paper proposes a new image-text consistency driven multimodal sentiment analysis approach. The proposed approach explores the correlation between the image and the text, followed by a multimodal adaptive sentiment analysis method. To be more specific, the mid-level visual features extracted by the conventional SentiBank approach are used to represent visual concepts, with the integration of other features, including textual, visual and social features, to develop a machine learning sentiment analysis approach. Extensive experiments are conducted to demonstrate the superior performance of the proposed approach. |
| |
| Keywords: | |
| 本文献已被 ScienceDirect 等数据库收录！ |
|
