Hervé Bredin, Ruiqing Yin, Juan Manuel Coria, Gregory Gelly, Pavel Korshunov, Marvin Lavechin, Diego Fustes, Hadrien Titeux, Wassim Bouaziz, Marie-Philippe Gill
We introduce pyannote.audio, an open-source toolkit written in Python for speaker diarization. Based on PyTorch machine learning framework, it provides a set of trainable end-to-end neural building blocks that can be combined and jointly optimized to build speaker diarization pipelines. pyannote.audio also comes with pre-trained models covering a wide range of domains for voice activity detection, speaker change detection, overlapped speech detection, and speaker embedding -- reaching state-of-the-art performance for most of them.
| Task | Dataset | Metric | Value | Model |
|---|---|---|---|---|
| Speaker Diarization | ETAPE | DER(%) | 4.9 | pyannote (waveform) |
| Speaker Diarization | ETAPE | FA | 4.2 | pyannote (waveform) |
| Speaker Diarization | ETAPE | Miss | 0.7 | pyannote (waveform) |
| Speaker Diarization | ETAPE | DER(%) | 5.6 | pyannote (MFCC) |
| Speaker Diarization | ETAPE | FA | 5.2 | pyannote (MFCC) |
| Speaker Diarization | ETAPE | Miss | 0.4 | pyannote (MFCC) |
| Speaker Diarization | ETAPE | DER(%) | 7.7 | Baseline |
| Speaker Diarization | ETAPE | FA | 7.5 | Baseline |
| Speaker Diarization | ETAPE | Miss | 0.2 | Baseline |
| Speaker Diarization | DIHARD | DER(%) | 9.9 | pyannote (waveform) |
| Speaker Diarization | DIHARD | FA | 5.7 | pyannote (waveform) |
| Speaker Diarization | DIHARD | Miss | 4.2 | pyannote (waveform) |
| Speaker Diarization | DIHARD | DER(%) | 10.5 | pyannote (MFCC) |
| Speaker Diarization | DIHARD | FA | 6.8 | pyannote (MFCC) |
| Speaker Diarization | DIHARD | Miss | 3.7 | pyannote (MFCC) |
| Speaker Diarization | DIHARD | DER(%) | 11.2 | Baseline (the best result in the literature as of Oct.2019) |
| Speaker Diarization | DIHARD | FA | 6.5 | Baseline (the best result in the literature as of Oct.2019) |
| Speaker Diarization | DIHARD | Miss | 4.7 | Baseline (the best result in the literature as of Oct.2019) |
| Speaker Diarization | AMI | DER(%) | 6 | pyannote (waveform) |
| Speaker Diarization | AMI | FA | 3.6 | pyannote (waveform) |
| Speaker Diarization | AMI | Miss | 2.4 | pyannote (waveform) |
| Speaker Diarization | AMI | DER(%) | 6.3 | pyannote (MFCC) |
| Speaker Diarization | AMI | FA | 3.5 | pyannote (MFCC) |
| Speaker Diarization | AMI | Miss | 2.7 | pyannote (MFCC) |
| Multi-Label Classification | CheXpert | NUM RADS BELOW CURVE | 0.2 | Baseline |