Hossein Damavandi, Hasan Jalali, Boshra Pishgoo
This paper introduces a novel method for automated server provisioning by integrating Transformerbased Named Entity Recognition models with Automated Speech Detection using OpenAI's Whisper. Leveraging advanced Transformer architectures-BERT, RoBERTa, and DeBERTa-combined with robust speech-to-text capabilities, our approach enables IT professionals to provision cloud servers efficiently via natural spoken commands. A customannotated dataset containing real-world and AI-generated provisioning requests is presented, meticulously labeled using the BIO tagging scheme across fourteen critical entity categories relevant to cloud infrastructure provisioning. Comprehensive evaluations of model performance and robustness were conducted under realistic conditions, including controlled transcription noise to simulate practical speech recognition errors. While all tested models achieved high performance on clean test data, results from noisy test scenarios revealed notable disparities in model generalization capabilities. Specifically, DeBERTa exhibited exceptional resilience, maintaining an F1 score of 96.23 % under adverse conditions. These findings highlight the practicality and robustness of combining speech-to-text processing with advanced Named Entity Recognition models, significantly advancing real-time, voice-driven IT automation workflows.