ModelListedExternal

Deepfake Detection Ff Cn Transformer

This repository contains the trained checkpoints for a controlled study of video-level deepfake detection on FaceForensics++ (C23). It hosts five models: four established baselines and one proposed Hybrid...

Sourcehuggingface.co/MUmairAB/deepfake-detection-ff-cn-transformer/tree/1ff717adad4f4f1e1a6d98c231af0bb1ff534f74

KindOtherVersionBest modelBgrade

Model source

Kind: Other
File size: 6.89–165 MB
Version: vBest model
Updated: Oct 31, 2026
Source: Hugging Face

Source description

Deepfake Detection on FaceForensics++ — CNN, Transformer, and Hybrid Models

This repository contains the trained checkpoints for a controlled study of video-level deepfake detection on FaceForensics++ (C23). It hosts five models: four established baselines and one proposed Hybrid CNN–Transformer.

The study is a scientific investigation with a negative result: adding a temporal Transformer on top of a strong per-frame CNN backbone does not improve in-domain detection or cross-manipulation generalization under the limited data and compute budget used here. These weights are released for reproducibility and educational purposes, not as a production-ready detector.

Models

The five models form a 2×2 design matrix over {pure-CNN, has-Transformer} × {spatial-only, spatiotemporal}:

Checkpoint	Architecture	Params	Temporal?
`xception.pt`	Xception (frame-averaged logits)	20.8M	No (spatial-only)
`efficientnet_b3.pt`	EfficientNet-B3 (frame-averaged logits)	10.7M	No (spatial-only)
`cnn_lstm.pt`	ResNet-18 + 2-layer BiLSTM	13.7M	Yes (recurrent)
`vit_base.pt`	ViT-Base/16 (frame-averaged logits)	85.8M	No (spatial-only)
`hybrid_cnn_transformer.pt`	EfficientNet-B3 + 4-layer Transformer encoder	74.9M	Yes (attention)

The proposed hybrid uses the same EfficientNet-B3 backbone as the EfficientNet baseline, so comparing the two isolates the effect of the temporal Transformer head (a controlled ablation).

Results

In-domain test (5 trained classes, shared test split):

Model	AUC	Accuracy	Precision	Recall	F1
Xception	0.9944	0.9744	0.9823	0.9858	0.9840
EfficientNet-B3	0.9976	0.9829	0.9825	0.9964	0.9894
CNN+BiLSTM	0.9805	0.9744	0.9823	0.9858	0.9840
ViT-Base/16	0.7066	0.6695	0.8910	0.6690	0.7642
Hybrid (ours)	0.9458	0.8462	0.9710	0.8327	0.8966

Cross-manipulation (in-domain AUC → held-out FaceShifter AUC):

Model	In-domain AUC	FaceShifter AUC	AUC drop
Xception	0.9944	0.7015	+0.2929
EfficientNet-B3	0.9976	0.7343	+0.2633
CNN+BiLSTM	0.9805	0.7591	+0.2214
ViT-Base/16	0.7066	0.6880	+0.0186
Hybrid (ours)	0.9458	0.6273	+0.3185

Key findings: EfficientNet-B3 is the strongest in-domain model; the hybrid scores below its own backbone; every model degrades sharply on the unseen manipulation; and the hybrid generalizes worst of all. ViT's small AUC drop is an artifact of its already-weak in-domain performance, not real robustness.

Intended use and scope

Intended use: research and educational reproduction of the accompanying study; baseline checkpoints for further experiments on FaceForensics++.
Out of scope: real-world or forensic deepfake detection, content moderation, or any decision affecting individuals. These models do not generalize to unseen manipulation types and must not be relied upon as a trustworthy detector.

How to use

Each checkpoint stores a model_state_dict plus the test metrics. The model class definitions live in the project notebooks (see the code repository). A typical load looks like:

import torch

ckpt = torch.load("hybrid_cnn_transformer.pt", map_location="cpu", weights_only=False)
model.load_state_dict(ckpt["model_state_dict"])   # `model` is the matching architecture
model.eval()
print(ckpt.get("metrics"))

Inputs are clips of shape [T=24, 3, 224, 224], ImageNet-normalized, produced by sampling 24 frames per video, detecting the face with MTCNN, cropping with a 30% margin, and resizing to 224×224. The frame-averaging models (Xception, EfficientNet, ViT) average per-frame logits; CNN+BiLSTM and the hybrid output a single clip-level logit directly. Apply a sigmoid to obtain the fake probability.

Training details

All five models were trained with an identical protocol for a fair architectural comparison:

Loss: binary cross-entropy on the clip-level logit
Optimizer: AdamW, lr 1e-4, weight decay 1e-4
Schedule: cosine annealing over 25 epochs; early stopping (patience 5) on validation AUC; best checkpoint by validation AUC
Mixed precision (fp16 autocast + gradient scaling)
Class imbalance (~4:1 fake:real) handled with an inverse-frequency WeightedRandomSampler
Augmentation (train only): horizontal flip (p=0.5), brightness jitter (±10%), ImageNet normalization
Seed 42; trained on a single NVIDIA RTX 5080 (16 GB) with PyTorch 2.11 / CUDA 12.8

No per-model hyperparameter sweep was performed; this is a deliberate fair-comparison choice and a stated limitation. TensorBoard training logs are included in this repository.

Dataset

FaceForensics++ at the C23 (high-quality) compression level.

Training classes: real original videos + Deepfakes, Face2Face, FaceSwap, NeuralTextures (700 videos per class)
Held out for cross-manipulation testing: FaceShifter (400 videos), never seen during training

FaceForensics++ is distributed under its own research license. Obtain the dataset from its official source and comply with its terms; these weights are a derivative of that data and inherit its usage restrictions.

Limitations and ethical considerations

Poor cross-manipulation generalization. Every model collapses on the held-out FaceShifter attack (AUC ~0.63–0.76). High in-domain accuracy does not imply a general notion of "manipulated."
Not for deployment. These models can produce confident false negatives on unseen forgeries and false positives on genuine video. Do not use them to make consequential judgments about real people or content.
Dataset bias. Performance reflects the actors, capture conditions, and manipulation methods present in FaceForensics++ and may not transfer elsewhere.
Dual-use awareness. Detection research can inform evasion; please use responsibly.

Citation

If you use these checkpoints, please cite the accompanying report and the FaceForensics++ dataset:

@misc{deepfake_ff_cnn_transformer,
  title  = {Spatiotemporal Deepfake Detection with Cross-Manipulation
            Generalization: A Hybrid CNN--Transformer Approach on FaceForensics++},
  author = {Umair Akram and Ibrahim Dodo},
  year   = {2026},
  note   = {deepfake research study}
}

@inproceedings{rossler2019ff,
  title     = {FaceForensics++: Learning to Detect Manipulated Facial Images},
  author    = {R\"ossler, Andreas and Cozzolino, Davide and Verdoliva, Luisa and
               Riess, Christian and Thies, Justus and Nie{\ss}ner, Matthias},
  booktitle = {ICCV},
  year      = {2019}
}

Sources

1 source

Hugging FacePrimaryVersion 1ff717adad4f4f1e1a6d98c231af0bb1ff534f74

huggingface.co/MUmairAB/deepfake-detection-ff-cn-transformer/tree/1ff717adad4f4f1e1a6d98c231af0bb1ff534f74

Verified Oct 11

Download files

Model artifacts

64 artifacts

cnn_lstm_baseline/best_model.pt

pt · 165 MB · SHA-256 1a8b3a0302de…14f2 · Hugging Face

Download file

efficientnet_b3_baseline/best_model.pt

pt · 123 MB · SHA-256 3d829fe6c257…7fc6 · Hugging Face

Download file

ff-c23-preprocessed/Deepfakes/000_003.pt

pt · 6.89 MB · SHA-256 7aa13058f689…259d · Hugging Face

Download file

ff-c23-preprocessed/Deepfakes/001_870.pt

pt · 6.89 MB · SHA-256 b56aee2a3ec3…2e55 · Hugging Face

Download file

ff-c23-preprocessed/Deepfakes/002_006.pt

pt · 6.89 MB · SHA-256 8c0e7d019d03…bd22 · Hugging Face

Download file

ff-c23-preprocessed/Deepfakes/003_000.pt

pt · 6.89 MB · SHA-256 41d70777f6f3…e653 · Hugging Face

Download file

ff-c23-preprocessed/Deepfakes/004_982.pt

pt · 6.89 MB · SHA-256 cb8b6ec0f6b6…e8c6 · Hugging Face

Download file

ff-c23-preprocessed/Deepfakes/005_010.pt

pt · 6.89 MB · SHA-256 de065df53f49…5077 · Hugging Face

Download file

ff-c23-preprocessed/Deepfakes/006_002.pt

pt · 6.89 MB · SHA-256 28fffe62c667…bab9 · Hugging Face

Download file

ff-c23-preprocessed/Deepfakes/007_132.pt

pt · 6.89 MB · SHA-256 a01e94872440…a3a2 · Hugging Face

Download file

ff-c23-preprocessed/Deepfakes/008_990.pt

pt · 6.89 MB · SHA-256 47fe875d2b49…332d · Hugging Face

Download file

ff-c23-preprocessed/Deepfakes/009_027.pt

pt · 6.89 MB · SHA-256 f4d61e756f4d…b339 · Hugging Face

Download file

ff-c23-preprocessed/Deepfakes/010_005.pt

pt · 6.89 MB · SHA-256 89d518c52797…d6e0 · Hugging Face

Download file

ff-c23-preprocessed/Deepfakes/011_805.pt

pt · 6.89 MB · SHA-256 41f6d545c8bc…1d74 · Hugging Face

Download file

ff-c23-preprocessed/Deepfakes/012_026.pt

pt · 6.89 MB · SHA-256 8a5564651cb4…a963 · Hugging Face

Download file

ff-c23-preprocessed/Deepfakes/013_883.pt

pt · 6.89 MB · SHA-256 3f5c6ea6e672…ebc7 · Hugging Face

Download file

ff-c23-preprocessed/Deepfakes/014_790.pt

pt · 6.89 MB · SHA-256 b9b621470d62…fe00 · Hugging Face

Download file

ff-c23-preprocessed/Deepfakes/015_919.pt

pt · 6.89 MB · SHA-256 e521e28a73a7…ac23 · Hugging Face

Download file

ff-c23-preprocessed/Deepfakes/016_209.pt

pt · 6.89 MB · SHA-256 619ae36b5e38…0f2f · Hugging Face

Download file

ff-c23-preprocessed/Deepfakes/017_803.pt

pt · 6.89 MB · SHA-256 21f4d51790b8…d281 · Hugging Face

Download file

ff-c23-preprocessed/Deepfakes/018_019.pt

pt · 6.89 MB · SHA-256 a6ff2c6b0d37…e005 · Hugging Face

Download file

ff-c23-preprocessed/Deepfakes/019_018.pt

pt · 6.89 MB · SHA-256 fc47664eaf8e…824e · Hugging Face

Download file

ff-c23-preprocessed/Deepfakes/020_344.pt

pt · 6.89 MB · SHA-256 ad0f6996a18a…545f · Hugging Face

Download file

ff-c23-preprocessed/Deepfakes/021_312.pt

pt · 6.89 MB · SHA-256 7a461e571a58…0a4d · Hugging Face

Download file

ff-c23-preprocessed/Deepfakes/022_489.pt

pt · 6.89 MB · SHA-256 4fac08176d0e…3043 · Hugging Face

Download file

ff-c23-preprocessed/Deepfakes/023_923.pt

pt · 6.89 MB · SHA-256 0245b4f524ee…bd8e · Hugging Face

Download file

ff-c23-preprocessed/Deepfakes/024_073.pt

pt · 6.89 MB · SHA-256 e08a683396ed…ca84 · Hugging Face

Download file

ff-c23-preprocessed/Deepfakes/025_067.pt

pt · 6.89 MB · SHA-256 ebd3138e1aa8…7477 · Hugging Face

Download file

ff-c23-preprocessed/Deepfakes/026_012.pt

pt · 6.89 MB · SHA-256 f5b3f5535bd1…01e7 · Hugging Face

Download file

ff-c23-preprocessed/Deepfakes/027_009.pt

pt · 6.89 MB · SHA-256 c3f324cb37e6…5910 · Hugging Face

Download file

ff-c23-preprocessed/Deepfakes/028_068.pt

pt · 6.89 MB · SHA-256 d8dbc21af795…6981 · Hugging Face

Download file

ff-c23-preprocessed/Deepfakes/029_048.pt

pt · 6.89 MB · SHA-256 71fe1631f05b…c19f · Hugging Face

Download file

ff-c23-preprocessed/Deepfakes/030_193.pt

pt · 6.89 MB · SHA-256 f2567f974635…48d8 · Hugging Face

Download file

ff-c23-preprocessed/Deepfakes/031_163.pt

pt · 6.89 MB · SHA-256 de0761a01383…17c8 · Hugging Face

Download file

ff-c23-preprocessed/Deepfakes/032_944.pt

pt · 6.89 MB · SHA-256 49d1f618012b…6847 · Hugging Face

Download file

ff-c23-preprocessed/Deepfakes/033_097.pt

pt · 6.89 MB · SHA-256 c4eba750ef77…7a5d · Hugging Face

Download file

ff-c23-preprocessed/Deepfakes/034_590.pt

pt · 6.89 MB · SHA-256 fe82a2a661b6…8fda · Hugging Face

Download file

ff-c23-preprocessed/Deepfakes/035_036.pt

pt · 6.89 MB · SHA-256 79ec795f65fc…530b · Hugging Face

Download file

ff-c23-preprocessed/Deepfakes/036_035.pt

pt · 6.89 MB · SHA-256 bf0cd6183b21…d0ee · Hugging Face

Download file

ff-c23-preprocessed/Deepfakes/037_072.pt

pt · 6.89 MB · SHA-256 3a29cab1358c…8266 · Hugging Face

Download file

ff-c23-preprocessed/Deepfakes/038_125.pt

pt · 6.89 MB · SHA-256 853c530cd641…c81b · Hugging Face

Download file

ff-c23-preprocessed/Deepfakes/039_058.pt

pt · 6.89 MB · SHA-256 1231145b63b3…1010 · Hugging Face

Download file

ff-c23-preprocessed/Deepfakes/040_997.pt

pt · 6.89 MB · SHA-256 c7f9d68a5ccf…a2ad · Hugging Face

Download file

ff-c23-preprocessed/Deepfakes/041_063.pt

pt · 6.89 MB · SHA-256 0d55e65ef88e…3ac8 · Hugging Face

Download file

ff-c23-preprocessed/Deepfakes/042_084.pt

pt · 6.89 MB · SHA-256 8723490ab991…e6b0 · Hugging Face

Download file

ff-c23-preprocessed/Deepfakes/043_110.pt

pt · 6.89 MB · SHA-256 9a4312c109d9…5e65 · Hugging Face

Download file

ff-c23-preprocessed/Deepfakes/044_945.pt

pt · 6.89 MB · SHA-256 48c8454df7fa…fad0 · Hugging Face

Download file

ff-c23-preprocessed/Deepfakes/045_889.pt

pt · 6.89 MB · SHA-256 536474f445b2…9d9a · Hugging Face

Download file

ff-c23-preprocessed/Deepfakes/046_904.pt

pt · 6.89 MB · SHA-256 455262e62b24…f972 · Hugging Face

Download file

ff-c23-preprocessed/Deepfakes/047_862.pt

pt · 6.89 MB · SHA-256 2d821c715166…2e0c · Hugging Face

Download file

ff-c23-preprocessed/Deepfakes/048_029.pt

pt · 6.89 MB · SHA-256 0746acdd9b2c…2ef7 · Hugging Face

Download file

ff-c23-preprocessed/Deepfakes/049_946.pt

pt · 6.89 MB · SHA-256 20078e915b6e…4b8b · Hugging Face

Download file

ff-c23-preprocessed/Deepfakes/050_059.pt

pt · 6.89 MB · SHA-256 34c5f932b5b1…c57c · Hugging Face

Download file

ff-c23-preprocessed/Deepfakes/051_332.pt

pt · 6.89 MB · SHA-256 9365ee89afce…383c · Hugging Face

Download file

ff-c23-preprocessed/Deepfakes/052_108.pt

pt · 6.89 MB · SHA-256 84852a9607fa…c2ac · Hugging Face

Download file

ff-c23-preprocessed/Deepfakes/053_095.pt

pt · 6.89 MB · SHA-256 65146c4459a2…8f04 · Hugging Face

Download file

ff-c23-preprocessed/Deepfakes/054_071.pt

pt · 6.89 MB · SHA-256 aef261d3fe46…424f · Hugging Face

Download file

ff-c23-preprocessed/Deepfakes/055_147.pt

pt · 6.89 MB · SHA-256 f517e055bb10…16f2 · Hugging Face

Download file

ff-c23-preprocessed/Deepfakes/056_996.pt

pt · 6.89 MB · SHA-256 eeb4b3dfd266…4cb0 · Hugging Face

Download file

ff-c23-preprocessed/Deepfakes/057_070.pt

pt · 6.89 MB · SHA-256 04355d06ab26…ec42 · Hugging Face

Download file

ff-c23-preprocessed/Deepfakes/058_039.pt

pt · 6.89 MB · SHA-256 0d99c74bfecc…b0b1 · Hugging Face

Download file

ff-c23-preprocessed/Deepfakes/059_050.pt

pt · 6.89 MB · SHA-256 1fc86b7deefd…bde0 · Hugging Face

Download file

ff-c23-preprocessed/Deepfakes/060_088.pt

pt · 6.89 MB · SHA-256 e56a26b6c9f5…fa2a · Hugging Face

Download file

ff-c23-preprocessed/Deepfakes/061_080.pt

pt · 6.89 MB · SHA-256 89cc6204dcaf…9a76 · Hugging Face

Download file

Source excerpts

3 excerpts

ReadmeAug 2huggingface.co/MUmairAB/deepfake-detection-ff-cn-transformer

--- license: mit tags: - deepfake-detection - video-classification - computer-vision - face-forgery-detection - pytorch - cnn - transformer - faceforensics datasets: - faceforensics language: - en library_name: pytorch pipeline_tag: video-classification metrics: - accuracy - f1 - precision - recall - roc_auc model-index: - name: deepfake-detection-ff-cnn-transformer results: - task: type: video-classification name: In-domain deepfake detection dataset: type: faceforensics name: FaceForensics++ (C23, in-domain test split) metrics: - type: roc_auc value: 0.9458 name: AUC (Hybrid, in-domain) - type: accuracy value: 0.8462 name: Accuracy (Hybrid, in-domain) - type: f1 value: 0.8966 name: F1 (Hybrid, in-domain) - task: type: video-classification name: Cross-manipulation generalization (held-out FaceShifter) dataset: type: faceforensics name: FaceForensics++ (C23, held-out FaceShifter) metrics: - type: roc_auc value: 0.6273 name: AUC (Hybrid, FaceShifter) --- # Deepfake Detection on FaceForensics++ — CNN, Transformer, and Hybrid Models This repository contains the trained checkpoints for a controlled study of video-level deepfake detection on **FaceForensics++ (C23)**. It hosts five models: four established baselines and one proposed **Hybrid CNN–Transformer**. The study is a scientific investigation with a **negative result**: adding a temporal Transformer on top of a strong per-frame CNN backbone does **not** improve in-domain detection or cross-manipulation generalization under the limited data and compute budget used here. These weights are released for reproducibility and educational purposes, not as a production-ready detector. --- ## Models The five models form a 2×2 design matrix over {pure-CNN, has-Transformer} × {spatial-only, spatiotemporal}: | Checkpoint | Architecture | Params | Temporal? | |---|---|---|---| | `xception.pt` | Xception (frame-averaged logits) | 20.8M | No (spatial-only) | | `efficientnet_b3.pt` | EfficientNet-B3 (frame-averaged logits) | 10.7M | No (spatial-only) | | `cnn_lstm.pt` | ResNet-18 + 2-layer BiLSTM | 13.7M | Yes (recurrent) | | `vit_base.pt` | ViT-Base/16 (frame-averaged logits) | 85.8M | No (spatial-only) | | `hybrid_cnn_transformer.pt` | EfficientNet-B3 + 4-layer Transformer encoder | 74.9M | Yes (attention) | The proposed hybrid uses the **same EfficientNet-B3 backbone...

NoteOct 11huggingface.co/MUmairAB/deepfake-detection-ff-cn-transformer

Source context: 0 downloads · 2 likes · Pipeline video-classification · Library pytorch · Repo MUmairAB/deepfake-detection-ff-cn-transformer

NoteAug 2huggingface.co/MUmairAB/deepfake-detection-ff-cn-transformer

Source context: 0 downloads · 0 likes · Pipeline video-classification · Library pytorch · Repo MUmairAB/deepfake-detection-ff-cn-transformer