It was late July 2019 when Loyola University of Chicago reached out to us about launching a digital assistant by Labor Day just 6 short weeks away. Loyola had established a five-year plan to advance the Loyola Digital Experience (LDE) strategy in part by leveraging artificial intelligence (AI). A digital assistant would be a perfect blend of digitizing previously analog services, like student support, by deploying AI. Loyola, with their long tradition in basketball, would call that an “and 1”!
The digital assistant would be piloted during the fall semester and eventually rolled out to 16,000+ undergraduate students. They named it LUie, and it would provide hundreds of answers to common questions while leveraging information sourced from bookstores, google maps, and PeopleSoft Campus. This posting will take you through the pilot, a sample of use cases, the data collected, and next steps for Loyola.
LUie is a clever play on the acronym Loyola University, or LU for short. Giving their digital assistant a personality while paying homage to its parent, the university.
The end result of this journey was an award-winning project receiving an Innovator’s Award from Oracle. The accuracy of LUie has even surpassed Alexa and on October 7, 2020 you can see LUie for yourself at Reconnect during the session, “Succeed with Chatbots in PeopleSoft”.
The Pilot
In a mere six weeks, LUie was configured with a mix of Ida’s pre-built campus answers and skills, plus Loyola’s own questions. It was then launched to students who were just arriving on campus. The only way to meet this timeline was to leverage the Ida platform and the IntraSee implementation methodology. Three departments took part in the pilot: the IT service desk, the bursar’s office and advising. Those departments would identify the most common questions fielded, which would then initiate automated testing and NLP learning sessions. Equipped with years of knowledge, LUie was then able to service hundreds of different undergraduate student and advisor requests.
The pilot focused on a fully-authenticated digital assistant because the point of a pilot is to prove LUie can handle the sophisticated features contained in authenticated campus systems like PeopleSoft. LUie would look up secure enterprise data and perform transactions on behalf of the student and advisor, so it needed to be fully authenticated and aware of permissions and roles. Upon login, LUie provided personalized responses applicable to the person’s role at the university. The pilot started with 50 users, but gradually built to around 1,000 users by the end of the semester. The goals were to show LUie in action with real users, prove the integration of enterprise information was successful, and to collect a ton of usage data to prove its usefulness.
Pilot Results
During the pilot, LUie hit an accuracy rate of 86%. That is much better than most humans, and also added the benefit of being available 24 hours a day and responding almost instantly. LUie never even had to take a lunch. Interestingly, the top categories of user questions were ones asking about the student’s information and academic data (44% of all questions asked). This observation showed that students didn’t want a basic chatbot nor a link to go somewhere else. They wanted a digital assistant with personalized responses right within the conversation flow, and returned in an instant.
During the pilot, responses were provided by LUie at an accuracy rate of 86%.
Another interesting observation was that 28% of all questions asked during the pilot were on subjects LUie was never trained to answer. An example of one of these was, “how do I get my broken dorm window fixed?” AI is much like a young student in that it needs to be taught to understand a topic. However, it is unlike a student in that LUie only needs to be trained once; as it never forgets, it maintains its accuracy, and never leaves for greener pastures. The type of questions asked were so broad it showed that students expected their digital assistant to do it all. Even if a user was on the Bursar’s page when they launched LUie, it didn’t mean they kept their questions limited to student financials. This is an important fact to consider when launching a digital assistant vs. a chatbot.
On occasion, LUie asked users for feedback about how he performed. 91% of that feedback was positive. It’s hard to imagine having a 91% satisfaction rate on any support experience, but LUie’s speed, availability and accuracy made for a happy user.
Full Launch and COVID
In the Spring, LUie launched to all undergraduates and a portion of advisors for a total of over 16,000 users. During the full roll out, the COVID pandemic hit. Coincidentally, we realized how key a digital assistant would be during this time. There has never been a moment in recent history with so many questions from students, faculty and staff. And the answers to those questions seem to change daily. Speed and accuracy is king in this environment. There is no room for false information and LUie excels at just this task.
LUie took in almost a hundred new questions from Ida’s library concerning institutional policy related to COVID. Topics such as symptoms, living on campus, essential employee status, zoom tech support, drop deadline changes, grading basis, building access and so much more.
As with any good AI, Loyola now monitors how LUie is doing and provides it with weekly feedback (just as a teacher would for a student). That feedback automatically evolves LUie’s understanding of the campus such that it is getting better at its job every week.
Not Just Students
There is a lot of focus with chatbot providers on the student, but LUie went beyond students to help advisors too. After all, advisors are key to student success, the ultimate goal. If we make the advisors life easier and save them time, they can serve the students better. LUie’s benefit to advisors can be illustrated by examining one innovative use case at Loyola.
Students are limited to taking 18 credit hours a semester. Loyola doesn’t want them to take on too much and impact the student’s success. However, there are certainly cases where a student can handle taking more, or they simply need to do so in order to graduate in four years. Traditionally, the student would approach their advisor and the advisor had to:
- Look up their GPA to ensure they were around a 3.0 or higher and then…
- Update two different pages in the student system including about 10 different fields.
This process could easily take 15-20 mins, and it was easy for the advisor to miss a step, or a field, which would create further problems.
LUie took a 15-20 minute advising process and automated it down to under 30 seconds
LUie now automates this whole process. An advisor simply asks LUie, “can you approve John Doe’s overloaded schedule?” LUie then automatically knows to look up their GPA and ask the advisor for approval to proceed. Once approved, LUie will automate the filling in of those 10 fields across two pages. The whole dialogue takes less than 30 seconds! All that time goes back to the Advisor to spend on strategic work.
Next Steps
LUie will not stand still. Different departments at Loyola see the potential and a line is starting to form to get LUie serving those departmental needs. LUie has a developing roadmap, with the following next phases being planned:
- Non-authenticated LUie: the ability for users to ask LUie for help and not have to sign in. As long as LUie’s answers are not sensitive or secured, it will provide the help. This includes parents, prospects and community members who can now use LUie. As of September 2020, this feature is now live!
- Graduate Students: the audience will expand to include graduate students which will have their own questions and even graduate-specific answers to current questions.
- Employees: the HR department wants to serve employee specific questions, including integration into HR applications and COVID related staff questions.
Conclusion
Industry surveys of Higher Education CIOs have highlighted the two top priorities in recent years: digital transformation and artificial intelligence. LUie has helped Loyola address both of these high priority areas while improving student service and helping to navigate a pandemic.
See LUie for yourself this October at the Reconnect conference on Wednesday, October 7 at 5pm ET. Just add Session ID 102210 to your schedule. If you can’t make Reconnect, just contact us below and we can setup a personal demo and show you how digital assistants can help your students, faculty and staff.
