Beyond the Illusion: Ensemble Learning for Effective Voice Deepfake Detection

Deepfake synthetic media, manipulated using artificial intelligence to mimic authenticity, has become more dangerous in the modern digital era. Despite significant progress in video deepfake detection, audio deepfake detection relies on specific datasets and machine learning algorithms. This study addresses this limitation by developing a deep ensemble learning approach. The proposed deep ensemble approach for audio deepfake detection is called the Ensemble Convolutional Neural Network-Mel Frequency Cepstral Coefficient (ECN-MF). The ECN-MF model comprises a recurrent neural network, a 1D convolutional neural network, long short-term memory networks, and a convolutional long short-term memory network to extract a wider range of audio features, including mel-frequency cepstral coefficient, chroma features, and zero crossing rate. The data processing step was carried out through the creation of a preprocessing pipeline, inclusive of feature extraction, dimensionality standardization, and data normalization. The summarized features were incorporated into the feature vector, which was then normalized before being standardized in order to enhance consistency and stability across the audios. The investigation of the suggested ECN-MF model was carried out using the Fake-or-Real dataset. The dataset comprises four sub-datasets ('for-original', 'for-norm', 'for-2sec', and 'for-rerec'), categorized by audio duration and bit rate. To evaluate the performance of the proposed ensemble model, we utilized all sub-datasets with fake and real audio. The proposed ensemble approach achieved state-of-the-art accuracies of 99.5% on the 'for-original' sub-dataset, closely matching the CNN model at 99.6%. It also achieved accuracies of 98% on 'for-norm,' 96.9% on 'for-2sec,' and 92.8% on 'for-rerec' sub-datasets. By applying the proposed ensemble model to the 'for-merged' dataset, which comprises all sub-datasets, we obtained a state-of-the-art accuracy of 98%. These results demonstrate the effectiveness of the proposed approach, which outperforms the results of the individual models.