Investigating AI Chatbot Integration in Academic Libraries: A Case Study

Abstract

AI chatbots have recently received great attention due to the advancement of large language models, such as OpenAI's GPT series and Google's Gemini, which have shown excellent performance and applicability. Therefore, many academic libraries endeavor to integrate AI chatbots into their services. The University of Oklahoma (OU) Libraries started an AI chatbot project in 2017, ahead of most other academic libraries, and developed an original AI chatbot. In July 2019, OU Libraries launched an AI chatbot service using Ivy.ai, which utilized cutting-edge technology at the time. The chatbot service, launched before COVID-19, continued to operate even when the OU Libraries was closed due to the pandemic and remained operational until recently, providing valuable log data recording interactions between users and AI chatbots over the past few years. This study used mixed methods to investigates (a) how the AI chatbot has been implemented at the OU Libraries, (b) the usage patterns of OU Libraries’ AI chatbot, and (c) how users of OU Libraries’ AI chatbot evaluate its performance by exploring the important factors influencing users’ chatbot evaluation and questions the chatbot was unable to provide answers. The study provides practical recommendations for institutions planning to implement AI chatbots, and also lays the foundation for future research on AI chatbot technology to improve library services.