Wav2vec2

Improved version available: caa-speech-detection-asvspoof2019/wav2vec2-v2-unfrozen — wav2vec2_v2 (top-4 transformer blocks unfrozen) achieves 1.55% eval EER (vs 7.53% here) and 0.3966 tandem min t-DCF.

wav2vec2 — ASVspoof 2019 LA Binary Anti-Spoofing Detector

Binary bonafide-vs-spoof classifier built on facebook/wav2vec2-base with a frozen encoder and a lightweight classification head, trained on the ASVspoof 2019 Logical Access scenario.

Model Details

Property	Value
Base model	facebook/wav2vec2-base
Encoder	Frozen during training
Classification head	Linear(768→256) → GELU → Dropout(0.1) → Linear(256→2)
Input	Raw waveform, 16 kHz, padded/truncated to 64 000 samples (4 s)
Parameters	~95 M total (~300 k trainable head only)
Checkpoint size	362 MB (best.pt)
Local version tag	wav2vec2

The encoder masking used during wav2vec 2.0 self-supervised pretraining (mask_time_prob, mask_feature_prob) is disabled at load time, following Tak et al. (2022) who showed masking hurts countermeasure performance.

Training Data

ASVspoof 2019 Logical Access

Split	Utterances	Attacks
Train	~25 000	A01–A06 (known)
Dev	~25 000	A01–A06 (known)
Eval	~71 000	A07–A19 (unseen)

• No data augmentation was applied (baseline experiment).
• Class-weighted cross-entropy ([8.837, 1.0] for bonafide/spoof) compensates for the ~8.84:1 spoof-heavy imbalance in the training split.
• Hyperparameters: seed=17245921, lr=1e-5, batch_size=8, 20 epochs, no early stopping.

Results

Baseline to beat: EER 8.09% (LFCC+GMM system from the ASVspoof 2019 paper).

Split	EER	tandem min t-DCF	In-the-Wild EER
Dev (baseline → improved)	4.199% → 0.197%	—	—
Eval	7.53%	0.9994	27.68%
Eval (improved: wav2vec2_v2)	1.55%	0.3966	27.68%

Note on t-DCF: The eval t-DCF of ≈ 1 for this baseline indicates a calibration failure — the model's score distribution is not well aligned with the cost function's operating point despite achieving reasonable EER. t-DCF values use the normalized [0, 1] convention with tandem ASV scores.

For comparison, lcnn_v7_cqt achieves 3.26% eval EER with a t-DCF of 0.4930, making it the recommended model for applications that weight calibration alongside raw detection rate. The improved wav2vec2_v2 achieves the best eval EER (1.55%) and the best t-DCF (0.3966) across all models.

Usage

Download

huggingface-cli download caa-speech-detection-asvspoof2019/wav2vec2

Load

Clone the training repository and install dependencies, then:

import torch
from src.models.wav2vec2.model import Wav2Vec2Model

config = {
    "pretrained_model": "facebook/wav2vec2-base",
    "hidden_dim": 256,
    "dropout": 0.1,
    "freeze_encoder": True,
}

model = Wav2Vec2Model(config)
state = torch.load("best.pt", map_location="cpu")
model.load_state_dict(state)
model.eval()

Inference

from transformers import Wav2Vec2FeatureExtractor
from src.data.audio import load_audio, pad_or_trim
import torch

extractor = Wav2Vec2FeatureExtractor.from_pretrained("facebook/wav2vec2-base")
waveform = load_audio("audio.flac", target_sr=16000)
waveform = pad_or_trim(waveform, 64000)

inputs = extractor(waveform, sampling_rate=16000, return_tensors="pt")
frames = inputs.input_values.float()

with torch.no_grad():
    logits = model({"frames": frames})["logits"]
    pred = logits.argmax(dim=-1).item()
    # 0 = bonafide, 1 = spoof

Intended Use & Limitations

• Intended use: detecting AI-generated / synthesised speech in the ASVspoof 2019 LA scenario as a countermeasure (CM) module.
• Domain: Trained and evaluated on the LA (logical access) partition only. Not validated on physical access (PA) or in-the-wild recordings.
• Unseen attacks: The eval set (A07–A19) contains attack types not seen during training. The 7.53% eval EER demonstrates reasonable generalisation, though the t-DCF ≈ 1 signals that score calibration is poor and the model should not be used directly as a calibrated likelihood ratio.
• Deployment caveat: The 362 MB checkpoint size makes this unsuitable for edge or real-time deployment. For resource-constrained scenarios, lcnn_v4_cqt (3.5 MB, 3.03% eval EER) is recommended.

Citation

ASVspoof 2019 dataset and challenge:

@inproceedings{wang2020asvspoof,
  title     = {{ASVspoof} 2019: A Large-Scale Public Database of Synthesized,
               Converted and Replayed Speech},
  author    = {Wang, Xin and Yamagishi, Junichi and Todisco, Massimiliano and
               Delgado, H{\'e}ctor and Nautsch, Andreas and Evans, Nicholas and
               Shim, Md Sahidullah Hector and Kinnunen, Tomi and Lee, Kong Aik and
               Patino, Jose and others},
  booktitle = {Computer Speech \& Language},
  volume    = {64},
  year      = {2021},
}

wav2vec 2.0:

@inproceedings{baevski2020wav2vec,
  title     = {wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations},
  author    = {Baevski, Alexei and Zhou, Henry and Mohamed, Abdelrahman and Auli, Michael},
  booktitle = {Advances in Neural Information Processing Systems (NeurIPS)},
  year      = {2020},
}