As a result of the ongoing lockdown, I’ve had a lot more time to explore the chatbot community and read about the many chatbot solutions that have been created around COVID-19.
It’s amazing to see how chatbot developers and designers have built great solutions so quickly. But I’ve also been sceptical of chatbot providers offering quick relief and triage for call centres. It’s still a huge effort to have a fully trained and up-to-date chatbot around a topic that’s extremely fluid and ever-changing.
A recent article explored some of the strains on existing chatbot platforms and explained:
The solutions aren’t perfect — there’s always going to be a need for human teams, even where chatbots are deployed — but COVID-19 has accelerated the need for AI-powered contact center messaging.
I also wanted to explore how the current situation has been a struggle for existing chatbots, especially within the heavily affected travel industry. At Caravelo we build chatbots for travel and we have seen how the fluid situation has strained our existing workflows; trying to keep our chatbots up-to-date, relevant and useful to users.
Here are some of the main pain points we struggled with early on in the COVID-19 crisis.
Agent Support
I know, I know — a chatbot is there to relieve customer care agents. However, a great chatbot is one that augments existing support channels. They need to work hand in hand, offering human hand-offs when needed for complex scenarios.
Due to lockdowns we faced a situation where contact centres were closed completely, and the chatbot became the sole communication channel for customers. This ended up frustrating users with a genuine need for human help: complex questions, out-of-scope queries, and other disgruntled users desperate for the latest information.
Using customers’ booking information helped us prioritise those with immediate travel plans. We added qualifiers (such as bookings within 5 or 7 days) while communicating to other users to have patience. This allowed our chatbots to at least acknowledge that users will be helped, although at a later date.
Platform Quotas
The influx of users strained our platform. We experienced a number of outages. Not great during a crisis situation.
Although our infrastructure has always had limitations such as server storage, Facebook APIs and Dialogflow quotas, we had never come close to reaching these in the past.
For example, our main NLU platform, Dialogflow, offers 180 requests per minute, plenty for even our most high volume chatbots. These levels were exceeded, delaying message responses or providing no response at all. There’s nothing worse than not receiving a response in a time of crisis.
We’re definitely more mindful of these restrictions now, and we’ve spread the load on Dialogflow by duplicating our AI training sets (agents). This worked well with our multi-app Facebook chatbots, as we had parallel apps installed on multiple Facebook pages for a client. We linked these apps to different Dialogflow agents.
Also having a strong sys-admin team helped us resolve our server issues quickly.
AI Training
Our fallback error rates increased due to the influx of unknown COVID-19 related terms. Our traveller user base entered a wide variety of queries that were out of the normal scope of our trained chatbots.
We regularly trained new COVID-19 terminology to keep up.
Examples:
COVID
Virus
Coronavirus
Lockdown
Epidemic
Quarantine
Isolation
And more.
The list evolved as the situation changed. Coronavirus to 2019-nCoV, travel restriction evolved to quarantine, quarantine to lockdown, and in some cases repatriation. We also added common typos e.g. qauranteen.
It’s certainly been a learning experience. I never knew what ‘force majeure’ meant until this crisis, and how applicable it’s been to the travel industry. Understanding that COVID-19 is an act of God, nullifying travellers ticket rules and insurance policies. This inevitably led to complaints increasing in our chatbots from users trying to understand their rights.
Furthermore, the strain on our platform was more challenging as the team had to train the intents in 6 different languages. Communication and co-ordination were key within our team and with clients.
Failing to train may result in more human hand-off requests, further straining customer support agents.
Content & Flows
We’ve had to re-think our main conversational flows, temporarily changing welcome messages, error fallbacks and even the persistent menus. We found that a pro-active approach worked best in directing people to the latest information quickly, which also reduced the reliance on AI training.
Earlier on we found it best to guide users to COVID-19 content rather than relying on the NLU (Natural Language Understanding) to catch the queries, which was likely to cause a fallback. This gave us some time to catch up on training.
But its been a struggle. Clients have often asked for urgent content changes, sometimes after-hours. We modified many intents temporarily and made hasty ad-hoc content edits. Some have resulted in less than ideal chatbot responses.
We definitely added a bit too much information on our chatbots. But with the urgency to convey the latest advice and client pressure, we decided it better to have the information available, than spend time designing new conversational flows.
👉 I highly recommend keeping backups of all original content, as in the rush of the crisis, we permanently replaced some content accidentally.
I’ve also seen similar patterns on other travel chatbots such as Iberia’s IBot above. An urgency to communicate all the information at once leading to lengthy text responses.
Qantas’ Facebook chatbot took it to the extreme:
Communication across the company was equally important. We worked hard to keep the team in the loop on changes across our platform, from our chatbot designers, AI trainers, content specialists and developers to our managers. This sharing of information helped focus our efforts and redirect resources from other non-essential products at Caravelo.
Conclusion
To really assist businesses such as airlines and travel agencies, a dedicated and focused team to manage a chatbot is essential. That team might qualify to be truly essential workers in these uncertain times, keeping businesses afloat and really assisting customers while relieving already strained customer support agents.
Travel Chatbots with COVID-19 content:
Also on Chatbots Life
