AI-powered chatbot assistant for course organization

In this paper we present the design and evaluation of an AI-powered chatbot tool to support course-related information for an undergraduate class at the University of Florence. Using the Landbot platform integrated with OpenAI’s gpt4 model, the chatbot provided students with real-time responses to course-related frequent questions. We assessed the chatbot’s performance through a post-course survey administered to 35 students. The ease of use, speed, and reliability were highly appreciated, while usefulness and response adequacy received mixed feedback. The reported main limitation was its limited scope, only answering questions about few selected topics. A k-means clustering analysis revealed two distinct groups of students, with a different satisfaction level; however, both groups agreed on preferring to use the chatbot over the official website of the University of Florence.