Informed Patch Enhanced HyperGCN for skeleton-based action recognition |
| |
| Institution: | 1. School of Management and Economics, University of Electronic Science and Technology of China, No.2006, Xiyuan Ave, West Hi-Tech Zone, 611731 Chengdu, Sichuan, PR China;2. The Walker School of Business and Technology, Webster University, 470 E Lockwood Ave, Webster Groves, MO 63119, United States;3. The James F. Dicke College of Business Administration, Ohio Northern University, 525 S Main St, Ada, OH, United States;1. Center for Studies of Information Resources, Wuhan University, Bayi Rd. 299, Wuhan, Hubei 430072, China;2. School of Information Management, Wuhan University, Bayi Rd 299, Wuhan 430072, China |
| |
| Abstract: | Human skeleton, as a compact representation of action, has attracted numerous research attentions in recent years. However, skeletal data is too sparse to fully characterize fine-grained human motions, especially for hand/finger motions with subtle local movements. Besides, without containing any information of interacted objects, skeleton is hard to identify human–object interaction actions accurately. Hence, many action recognition approaches that purely rely on skeletal data have met a bottleneck in identifying such kind of actions. In this paper, we propose an Informed Patch Enhanced HyperGraph Convolutional Network that jointly employs human pose skeleton and informed visual patches for multi-modal feature learning. Specifically, we extract five informed visual patches around head, left hand, right hand, left foot and right foot joints as the complementary visual graph vertices. These patches often exhibit many action-related semantic information, like facial expressions, hand gestures, and interacted objects with hands or feet, which can compensate the deficiency of skeletal data. This hybrid scheme can boost the performance while keeping the computation and memory load low since only five extra vertices are appended to the original graph. Evaluation on two widely used large-scale datasets for skeleton-based action recognition demonstrates the effectiveness of the proposed method compared to the state-of-the-art methods. Significant accuracy improvements are reported using X-Sub protocol on NTU RGB+D 120 dataset. |
| |
| Keywords: | Informed patch Hypergraph HyperGCN Multi-modal Skeleton-based action recognition |
| 本文献已被 ScienceDirect 等数据库收录！ |
|
