Computer Graphics
TU Braunschweig

EEG-based Analysis of the Impact of Familiarity in the Perception of Deepfake Videos


EEG-based Analysis of the Impact of Familiarity in the Perception of Deepfake Videos

We investigate the brain’s subliminal response to fake portrait videos using electroencephalography, with a special emphasis on the viewer’s familiarity with the depicted individuals. Deepfake videos are increasingly becoming popular but, while they are entertaining, they can also pose a threat to society. These face-swapped videos merge physiognomy and behaviour of two different individuals, both strong cues used for recognizing a person. We show that this mismatch elicits different brain responses depending on the viewer’s familiarity with the merged individuals.

Using EEG, we classify perceptual differences of familiar and unfamiliar people versus their face-swapped counterparts. Our results show that it is possible to discriminate fake videos from genuine ones when at least one face-swapped actor is known to the observer. Furthermore, we indicate a correlation of classification accuracy with level of personal engagement between participant and actor, as well as with the participant’s familiarity with the used dataset.


Author(s):Jan-Philipp Tauscher, Susana Castillo, Sebastian Bosse, Marcus Magnor
Published:to appear
Type:Article in conference proceedings
Book:Proc. IEEE International Conference on Image Processing (ICIP)
ISBN:978-1-6654-3102-6
DOI:10.1109/ICIP42928.2021.9506082
Presented at:IEEE International Conference on Image Processing (ICIP) 2021
Project(s): ElectroEncephaloGraphics 


@inproceedings{tauscher2021eeg-based,
  title = {{EEG}-based Analysis of the Impact of Familiarity in the Perception of Deepfake Videos},
  author = {Tauscher, Jan-Philipp and Castillo, Susana  and Bosse, Sebastian and Magnor, Marcus},
  booktitle = {Proc. {IEEE} International Conference on Image Processing ({ICIP})},
  isbn = {978-1-6654-3102-6},
  doi = {10.1109/{ICIP}42928.2021.9506082},
  pages = {160--164},
  year = {2021}
}

Authors