Deepfake Detection
This repository contains the model for the paper:
Model source
- Kind
- Other
- File size
- 1.13 GB
- Version
- vRevision 521c54a
- Updated
- Sep 28, 2026
- Downloads
- 0
- Source
- Hugging Face
Source description
Unlocking the Hidden Potential of CLIP in Generalizable Deepfake Detection
This repository contains the model for the paper:
Unlocking the Hidden Potential of CLIP in Generalizable Deepfake Detection
Abstract
This paper tackles the challenge of detecting partially manipulated facial deepfakes, which involve subtle alterations to specific facial features while retaining the overall context, posing a greater detection difficulty than fully synthetic faces. We leverage the Contrastive Language-Image Pre-training (CLIP) model, specifically its ViT-L/14 visual encoder, to develop a generalizable detection method that performs robustly across diverse datasets and unknown forgery techniques with minimal modifications to the original model. The proposed approach utilizes parameter-efficient fine-tuning (PEFT) techniques, such as LN-tuning, to adjust a small subset of the model's parameters, preserving CLIP's pre-trained knowledge and reducing overfitting. A tailored preprocessing pipeline optimizes the method for facial images, while regularization strategies, including L2 normalization and metric learning on a hyperspherical manifold, enhance generalization. Trained on the FaceForensics++ dataset and evaluated in a cross-dataset fashion on Celeb-DF-v2, DFDC, FFIW, and others, the proposed method achieves competitive detection accuracy comparable to or outperforming much more complex state-of-the-art techniques. This work highlights the efficacy of CLIP's visual encoder in facial deepfake detection and establishes a simple, powerful baseline for future research, advancing the field of generalizable deepfake detection.
Results
Generalization of models trained on the FF++ dataset to unseen datasets and forgery methods. Reported values are video-level AUROC. Results of other methods are taken from their original papers. Values with * are taken from the other papers.
| Model | Year | Publication | CDFv2 | DFD | DFDC | FFIW | DSv1 |
|---|---|---|---|---|---|---|---|
| LipForensics | 2021 | CVPR | 82.4 | -- | 73.5 | -- | -- |
| FTCN | 2021 | ICCV | 86.9 | -- | 74.0 | 74.47* | -- |
| RealForensics | 2022 | CVPR | 86.9 | -- | 75.9 | -- | -- |
| SBI | 2022 | CVPR | 93.18 | 82.68 | 72.42 | 84.83 | -- |
| AUNet | 2023 | CVPR | 92.77 | 99.22 | 73.82 | 81.45 | -- |
| StyleDFD | 2024 | CVPR | 89.0 | 96.1 | -- | -- | -- |
| LSDA | 2024 | CVPR | 91.1 | -- | 77.0 | 72.4* | -- |
| LAA-Net | 2024 | CVPR | 95.4 | 98.43 | 86.94 | -- | -- |
| AltFreezing | 2024 | CVPR | 89.5 | 98.5 | 99.4 | -- | -- |
| NACO | 2024 | ECCV | 89.5 | -- | 76.7 | -- | -- |
| TALL++ | 2024 | IJCV | 91.96 | -- | 78.51 | -- | -- |
| UDD | 2025 | arXiv | 93.13 | 95.51 | 81.21 | -- | -- |
| Effort | 2025 | arXiv | 95.6 | 96.5 | 84.3 | 92.1 | -- |
| KID | 2025 | arXiv | 95.74 | 99.46 | 75.77 | 82.53 | -- |
| ForensicsAdapter | 2025 | arXiv | 95.7 | 97.2 | 87.2 | -- | -- |
| Proposed | 2025 | arXiv | 96.62 | 98.0 | 87.15 | 91.52 | 92.01 |
Example
See usage examples in our github project
Cite
@article{yermakov-2025-deepfake-detection,
title={Unlocking the Hidden Potential of CLIP in Generalizable Deepfake Detection},
author={Andrii Yermakov and Jan Cech and Jiri Matas},
year={2025},
eprint={2503.19683},
archivePrefix={arXiv},
primaryClass={cs.CV},
url={https://arxiv.org/abs/2503.19683},
}
Sources
1 source
Sources
1 sourceVerified Sep 28
Model artifacts
1 artifact
Model artifacts
1 artifactSource excerpts
2 excerpts
Source excerpts
2 excerpts--- license: mit tags: - pytorch - endpoints-template - deepfake-detection pipeline_tag: image-classification widget: - src: "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/hub/infrence-examples-text2img-dark.png" example_title: "Deepfake Image Example" --- # Unlocking the Hidden Potential of CLIP in Generalizable Deepfake Detection [](https://arxiv.org/abs/2503.19683) [](https://github.com/yermandy/deepfake-detection) This repository contains the model for the paper: **[Unlocking the Hidden Potential of CLIP in Generalizable Deepfake Detection](https://arxiv.org/abs/2503.19683)** ## Abstract > This paper tackles the challenge of detecting partially manipulated facial deepfakes, which involve subtle alterations to specific facial features while retaining the overall context, posing a greater detection difficulty than fully synthetic faces. We leverage the Contrastive Language-Image Pre-training (CLIP) model, specifically its ViT-L/14 visual encoder, to develop a generalizable detection method that performs robustly across diverse datasets and unknown forgery techniques with minimal modifications to the original model. The proposed approach utilizes parameter-efficient fine-tuning (PEFT) techniques, such as LN-tuning, to adjust a small subset of the model's parameters, preserving CLIP's pre-trained knowledge and reducing overfitting. A tailored preprocessing pipeline optimizes the method for facial images, while regularization strategies, including L2 normalization and metric learning on a hyperspherical manifold, enhance generalization. Trained on the FaceForensics++ dataset and evaluated in a cross-dataset fashion on Celeb-DF-v2, DFDC, FFIW, and others, the proposed method achieves competitive detection accuracy comparable to or outperforming much more complex state-of-the-art techniques. This work highlights the efficacy of CLIP's visual encoder in facial deepfake detection and establishes a simple, powerful baseline for future research, advancing the field of generalizable deepfake detection. ## Results Generalization of models trained on the FF++ dataset to unseen datasets and forgery methods. Reported values are **video-level AUROC**. Results of...
Source context: 0 downloads · 0 likes · Pipeline image-classification · Repo morhadi/deepfake-detection
