Design your chatbot to retrain itself
Users are going to submit unexpected input. Updating your chatbot afterwards using log data is slow. Build retraining right into your chatbot.
Download the complete example here: https://github.com/spackows/chatbot-auto-retrain
Intents
When a chatbot receives user input, a Natural Language Classifier model classifies the input according to the user’s intent. For example, if someone intends to compete in the horse show at the fall fair, they might ask the fall fair chatbot: “When are horse show entries due?” On the other hand, if they intend to watch the horse show, they might ask: “Do you need tickets to see the horse show?”
Anticipating users’ intents is tough
When building a new chatbot, trying to guess what your chatbot users will ask can be surprisingly difficult. Even with well-established chatbots, users can start asking new questions when their needs change over time.
Plan to retrain
When deploying a new chatbot, plan to continue investing a lot of time reviewing chatbot performance and retraining your chatbot. For example: systematically review chatbot logs, collect sample user input, label it, and then retrain your chatbot.
The easiest way to create a truly successful chatbot is to deploy a live chat solution first and then use the logs from the live chat as training data. You can slowly add chatbot automation to handle the most frequent, easiest issues and then gradually reduce staffing your live chat as the chatbot gets better. If you maintain a live chat option even when your chatbot is performing well, you can meet users’ changing needs while collecting new data to keep your chatbot up to speed.
Your users know what they mean
Reviewing chatbot logs to collect and label sample user input is time-consuming. That causes a delay between users asking unexpected questions and your chatbot being able to handle them. For some applications, a delay of a day or ever a few hours could make the chatbot useless right when people need it. And after having had a bad experience with your chatbot once, people might not use it again.
But you know who can help you? Your chatbot users!
If you design your chatbot dialog to ask users to label their own input, you can use that labelled input to retrain your chatbot automatically with little or no delay.
Example
Using IBM Watson Assistant, here’s just one way you could go about building retraining into your chatbot:
- Ask users to confirm their intent:
2. When the user confirms their intent, send the confirmed intent and the original input to a webhook:
3. In the webhook application, call the Watson Assistant API to add the input as an example to the specified intent:
g_app.post( "/confirmedintent", function( request, response )
{
response.json( { "Success" : "Success" } );
var intent = request.body.intent;
var user_input = request.body.user_input;
updateIntent( intent, user_input );
} );function updateIntent( intent, user_input )
{
var url = g_wa_instance_url +
"/v1/workspaces/" +
g_wa_skill_id +
"/intents/" +
intent +
"?append=true&version=2021-11-27";
var data = { "examples" : [ { "text" : user_input } ] };
g_axios.post( url, data, ... ).then( function( data )
{
console.log( "Intent updated successfully" ); } );}
See: https://cloud.ibm.com/apidocs/assistant/assistant-v1#updateintent
Watch this video for a full explanation:
There are many ways to implement this idea. And there are many details to work out, such as: what options to present; how to handle the “none of the above” case; and whether to send confirmed intents in batches instead of one at a time. You get the idea.
Conclusion
After you deploy your chatbot, that’s when the real work begins. Maintaining a chatbot — monitoring its performance, retraining it — is a lot of work. Save some time and set yourself and your chatbot users up for success by designing your chatbot to retrain itself.
