2

The topic of extracting the Facial Action Coding System (FACS) Action Units (AUs) [1] from images and it's translation into emotion prediction [2] is pretty well studied, but I'm not clear on how it stacks up against alternative approaches such as Convolutional Neural Networks (CNN). How much does the accuracy of AUs effect the accuracy of emotion detection?

  1. "Joint Facial Action Unit Detection and Feature Fusion: A Multi-conditional Learning Approach", Eleftheriadis S, Rudovic O, Pantic M. (2016)
  2. "Emotion Detection From Facial Expression And Its Use In The Evaluation Of Stress" Suvashis Das (2013)
Seanny123
  • 651
  • 8
  • 23

1 Answers1

2

As a preliminary answer, I can't imagine the extraction of AUs affecting their ability to detect emotions, since according to Do Deep Neural Networks Learn Facial Action Units When Doing Expression Recognition?, CNNs trained to extract emotions already correspond to AUs (called FAUs in the paper), as shown in the table below taken from their paper:

CNN filter to

Additionally, this is a very basic CNN with only three convolutional layers and grayscaled inputs. Given a larger CNN using Inception-layers and residual connections, more specialisation would probably be seen.

However, this conclusion also makes me feel like I'm dismissing a whole area of research, so it's probably a little hasty.

Seanny123
  • 651
  • 8
  • 23