This PhD project aims to develop novel methods for conversational speech processing in noisy and multispeaker environments, with a particular focus on integrating Speech LLMs into speech understanding pipelines. While Speech LLMs can capture rich linguistic and conversational information, their performance in real-world conversations is still limited by noise, speaker overlap, and complex multiparty interactions.

The research will investigate how self-supervised representations and Speech LLMs can improve robustness to adverse acoustic conditions while supporting deeper conversational understanding. A key objective is the development of unified frameworks combining speech enhancement, speech separation, speaker diarization, and automatic speech recognition. Beyond transcription, the project will explore conversational tasks such as turn-taking prediction, speaker intent recognition, meeting understanding, and context-aware dialogue.

Expected outcomes include novel algorithms, robust multimodal speech representations, and new insights into how Speech LLMs can bridge speech perception and conversational reasoning. Ultimately, the project seeks to advance conversational AI and enable reliable speech technologies for noisy, dynamic, and multispeaker real-world environments.