Significance of phonetic information in speaker verification systems

Abstract

The Automatic speaker verification (ASV) system validates whether the same or newlinedifferent speakers speak the two given utterances. The speech signal is a complex function newlineof the speaker s intrinsic characteristics (like pitch and accent), acoustic factors newline(like channel and noise), and phonetic information. These factors influence the performance newlineof ASV systems. The first two factors are more comprehensively investigated newlinethan the third factor. In this thesis, we analyze the influence of phonetic information newlineon ASV systems. Especially, ASV systems blended with explicit phonetic information newlinecan generate more speaker discriminative representations. The ASV system uses newlinephonetic information as auxiliary information to extract fixed-length phonetic-aware newlinespeaker embeddings. Typically, phonetic information is obtained from ASR systems newlinetrained using manual transcriptions in a supervised manner. However, the generation newlineof speech transcriptions is a laborious and expensive process. Hence, we pursue unsupervised newlinealternatives for obtaining phonetic information from the given speech utterance newlinewithout requiring explicit phonetic information. Specifically, we propose unsupervised newlineclustering and self-supervised approaches to acoustic modeling. A mixture of newlineautoencoders is employed for unsupervised clustering to group the acoustically similar newlinespeech segments. These groups are called virtual phonemes; hence, this network acts newlineas an unsupervised alternative to a supervised ASR system. Our experiments demonstrate newlinethat phonetic information extracted in an unsupervised approach performs on newlinepar with the supervised counterpart in extracting phonetic aware speaker embedding. newlineThe clustering techniques have limitations, like choosing the number of clusters newlineand their interpretability. To overcome this limitation, a self-supervised approach is newlinechosen as an alternative. In this approach, long-term contextual information can be newlinecaptured without transcriptions by training the network with contrastive estimation newlineloss. The phonetically rich self-sup