TasksSotADatasetsPapersMethodsSubmitAbout
Papers With Code 2

A community resource for machine learning research: papers, code, benchmarks, and state-of-the-art results.

Explore

Notable BenchmarksAll SotADatasetsPapersMethods

Community

Submit ResultsAbout

Data sourced from the PWC Archive (CC-BY-SA 4.0). Built by the community, for the community.

Methods/GER

GER

Gait Emotion Recognition

Introduced 200018 papers
Source Paper

Description

We present a novel classifier network called STEP, to classify perceived human emotion from gaits, based on a Spatial Temporal Graph Convolutional Network (ST-GCN) architecture. Given an RGB video of an individual walking, our formulation implicitly exploits the gait features to classify the perceived emotion of the human into one of four emotions: happy, sad, angry, or neutral. We train STEP on annotated real-world gait videos, augmented with annotated synthetic gaits generated using a novel generative network called STEP-Gen, built on an ST-GCN based Conditional Variational Autoencoder (CVAE). We incorporate a novel push-pull regularization loss in the CVAE formulation of STEP-Gen to generate realistic gaits and improve the classification accuracy of STEP. We also release a novel dataset (E-Gait), which consists of 4,227 human gaits annotated with perceived emotions along with thousands of synthetic gaits. In practice, STEP can learn the affective features and exhibits classification accuracy of 88% on E-Gait, which is 14--30% more accurate over prior methods.

Papers Using This Method

LLM-based Generative Error Correction for Rare Words with Synthetic Data and Phonetic Context2025-05-23Listening and Seeing Again: Generative Error Correction for Audio-Visual Speech Recognition2025-01-03Benchmarking Japanese Speech Recognition on ASR-LLM Setups with Multi-Pass Augmented Generative Error Correction2024-08-29A Survey of Deep Learning for Group-level Emotion Recognition2024-08-13A Discrete Perspective Towards the Construction of Sparse Probabilistic Boolean Networks2024-07-16Listen Again and Choose the Right Answer: A New Paradigm for Automatic Speech Recognition with Large Language Models2024-05-16MMGER: Multi-modal and Multi-granularity Generative Error Correction with LLM for Joint Accent and Speech Recognition2024-05-06A Generative Approach for Wikipedia-Scale Visual Entity Recognition2024-03-04It's Never Too Late: Fusing Acoustic Information into Large Language Models for Automatic Speech Recognition2024-02-08Large Language Models are Efficient Learners of Noise-Robust Speech Recognition2024-01-19GPT-4V with Emotion: A Zero-shot Benchmark for Generalized Emotion Recognition2023-12-07Generative error correction for code-switching speech recognition using large language models2023-10-17Towards A Robust Group-level Emotion Recognition via Uncertainty-Aware Learning2023-10-06Design and Planning of Flexible Mobile Micro-Grids Using Deep Reinforcement Learning2022-12-08Modeling Fine-grained Information via Knowledge-aware Hierarchical Graph for Zero-shot Entity Retrieval2022-11-20Distributed Privacy-Preserving Electric Vehicle Charging Control Based on Secret Sharing2021-10-05Generative Ensemble Regression: Learning Particle Dynamics from Observations of Ensembles with Physics-Informed Deep Generative Models2020-08-05STEP: Spatial Temporal Graph Convolutional Networks for Emotion Perception from Gaits2019-10-28