Did you know that NewDay's contact center handles a lot of calls every single day? Around 2.5 million calls every single year, to be more precise. The agents often spend a lot of time looking for answers to simple customers questions, which in turn, slows them and the entire system down. This led to a hackathon competition with a simple question and objective: how could generative AI improve customer and agent satisfaction and solve issues quicker?
During the hackathon, a NewAssist system was created that can process speech into text in real-time.
Think small at first
At one point, the team realized that a spoken assistant was a bit too ambitious, so they scaled back and created a chatbot and decided to prove that it can be a useful tool. They gave it more than 100 questions together with the right answers to test its capabilities and reliability.
At the start, the system wasn't that good, scoring below 60%. But they didn't give up and eventually figured out that by slightly tweaking the way they were handling knowledge articles, the accuracy could skyrocket. This quickly demonstrated how important data quality and structure are since the accuracy jumped from 60% all the way to 73%.
Want to learn more about how Generative AI can help financial services companies? You can find more details check here!
Testing the system the good old way
After reaching 80% accuracy, NewDay rolled out the experiment to 10 experienced customer service agents. Unexpectedly, agents used acronyms such as “dd” for direct debit and “cst” for customer. This dropped the accuracy down to 70%.
The team quickly noticed that NewAssist became increasingly unstable with every day passing by, so they quickly decided to integrate more data, especially those acronyms and abbreviations that agents frequently used. Almost like a brain, the system started getting smarter and agents even began asking it how to explain processes to customers using easier-to-grasp language.
NewDay decided to go through responses that weren't so good and used that information to quickly fix them or delete them if the mistake was too grave, thus leading to improvements in their knowledge base. NewDay eventually pushed its accuracy to 90%, and after extensive testing, a system for better customer service was rolled out to more than 150 agents. Early results showed that answers which previously took ninety seconds to find could now be retrieved in only four seconds. Talk about progress.
