Recent advances in generative models have enabled the production of increasingly realistic audiovisual deepfakes, raising important challenges for media authenticity and identity verification. Most existing deepfake detectors rely on artifacts introduced during the generation process to distinguish genuine from manipulated content. However, such artifacts are often specific to particular synthesis methods and can be strongly affected by benign transformations applied during content dissemination, limiting the robustness and generalization of these approaches.
This thesis investigates a complementary paradigm for deepfake detection based on behavioral identity. Rather than focusing primarily on how manipulated content is generated, the proposed approaches assess whether the observed behavior is consistent with the identity represented in the content. A first study demonstrates the fragility of conventional deepfake detectors under common social-media transformations, motivating the search for more intrinsic authenticity cues. We then show that facial dynamics contain identity-specific behavioral information that can be learned from genuine recordings and exploited to detect manipulations without relying on fake training data. Beyond detection, the learned facial representations also reveal that deepfakes can preserve behavioral traces of the source identity driving the manipulation, enabling, under certain conditions, the recognition of the individual whose behavior was transferred to the target.
The behavioral perspective is subsequently extended to speech through the modeling of identity-related prosodic patterns. In contrast to the initial person-specific formulation, this approach enables zero-shot detection using a single model that can be applied to previously unseen identities. Finally, facial and vocal behavioral representations are combined in a multimodal framework, showing that the two modalities provide complementary information for authenticity assessment.
Overall, the results highlight that the effectiveness of person-of-interest deepfake detection depends not only on extracting behavioral information, but also on learning representations that capture identity-specific behavior while remaining disentangled from the physical characteristics of the person. These findings support behavioral identity as a complementary direction for deepfake detection and suggest that, as synthetic media become increasingly realistic, authenticity assessment may benefit from reasoning about the consistency between who a person appears to be and how that person behaves.
