Thursday, July 20, 2017

 

We aim to resolve the difficulties of action recognition arising from the large intra-class variations. These unfavorable variations make it infeasible to represent one action instance by other ones of the same action. We hence propose to extract both instance-specific and class-consistent features to facilitate action recognition. Specifically, the instance-specific features explore the self-similarities among frames of each video instance, while class-consistent features summarize withinclass similarities. We introduce a generative formulation to combine the two diverse types of features. The experimental results demonstrate the effectiveness of our approach.

 

Instance-specific and Class-consistent Cues

  • We aim to resolve the difficulties of action recognition arising from the large intra-class variations. These unfavorable variations make it infeasible to represent one action instance by other ones of the same action. We hence propose to extract both instance-specific and class-consistent features to facilitate action recognition.
  • Credits
    • Instance-specific features: Self-similarities among frames of an action sequence. Multivariate linear prediction (MLP) is adopted to aggregate all the causalities among frames.
    • Class-consistent features: Characteristics shared by instances of the same action. Support vector machines (SVMs) are used to discover these features based on the bag-of-words model.
    • We propose a generative formulation to integrate the two complementary types of features, and boost the performance.

 

Core Techniques

  • We view actions as multivariate time signals. For signal processing in our approach, there are several essential techniques:
    • Wide-Sense Stationary Process
    • Linear Prediction
    • Support Vector Machine
  • We propose a generative model. The main idea is to consider the static and dynamic information of a multivariate time signal separately. This is based on the assumption that these two information are independent for action recognition.
    • Instance-specific cue via MLP
    • Class-consistent cue via SVM
    • Linear fusion
       

 

Publications

 

Action Recognition using Instance-specific and Class-consistent Cues

Chin-An Lin, Yen-Yu Lin, Hong-Yuan Mark Liao, and Shyh-Kang Jeng


IEEE International Conference on Image Processing (ICIP), September 2012


Paper