基于自监督语音表示的语音增强模型位置不变微调

Integrating front-end speech enhancement (SE) models with self-supervised learning (SSL)-based speech models proves effective for downstream tasks in noisy conditions. Conventionally, SE models are fine-tuned using SSL representations with a mean squared error (MSE) loss between enhanced and clean speech. However, MSE is prone to over-exploiting positional embeddings within SSL models, allowing the objective to be minimized primarily through positional correlations rather than genuine speech feature relationships. This over-reliance on positional information can restrict the model's generalization capability when encountering speech with varying temporal alignments or speaking rates. To address this limitation, a novel position-invariant fine-tuning method is proposed. This approach aims to compel the model to learn more robust representations of speech content, thereby reducing its dependence on specific positional cues. A new loss function is introduced, which considers temporal offset robustness when computing the similarity between enhanced and clean speech. Specifically, techniques such as dynamic time warping (DTW) or cross-correlation might be employed to permit a certain degree of temporal misalignment between the enhanced and clean speech, preventing the model from merely reducing loss by aligning positional information. This encourages the model to focus on intrinsic speech features and semantic content rather than absolute positions within the sequence. Experimental results demonstrate that, compared to traditional MSE-based fine-tuning methods, position-invariant fine-tuning significantly improves the performance of speech enhancement models across various noisy conditions. It exhibits stronger robustness and generalization, particularly when processing speech with temporal shifts or variability.