Computer Graphics
TU Braunschweig

Towards Understanding Perceptual Differences between Genuine and Face-Swapped Videos


Towards Understanding Perceptual Differences between Genuine and Face-Swapped Videos

In this paper, we report on perceptual experiments indicating that there are distinct and quantitatively measurable differences in the way we visually perceive genuine versus face-swapped videos. Recent progress in deep learning has made face-swapping techniques a powerful tool for creative purposes, but also a means for unethical forgeries. Currently, it remains unclear why people are misled, and which indicators they use to recognize potential manipulations. Here, we conduct three perceptual experiments focusing on a wide range of aspects: the conspicuousness of artifacts, the viewing behavior using eye tracking, the recognition accuracy for different video lengths, and the assessment of emotions.

Our experiments show that responses differ distinctly when watching manipulated as opposed to original faces, from which we derive perceptual cues to recognize face swaps. By investigating physiologically measurable signals, our findings yield valuable insights that may also be useful for advanced algorithmic detection.


Author(s):Leslie Wöhler, Martin Zembaty, Susana Castillo, Marcus Magnor
Published:to appear
Type:Article in conference proceedings
Book:Proc. ACM Human Factors in Computing Systems (CHI)
DOI:10.1145/3411764.3445627
Project(s): Perception of Video Manipulation   Immersive Digital Reality 


@inproceedings{wohler2021towards,
  title = {Towards Understanding Perceptual Differences between Genuine and Face-Swapped Videos},
  author = {W{\"o}hler, Leslie and Zembaty, Martin and Castillo, Susana  and Magnor, Marcus},
  booktitle = {Proc. {ACM} Human Factors in Computing Systems ({CHI})},
  doi = {10.1145/3411764.3445627},
  year = {2021}
}

Authors