Chatbot analytics provide a chatbot developer with valuable insights. Most bot-builder platforms track every chat session. The information is then summarized in a table, graph, or both. The statistics detail people’s interactions with the chatbot, where they are using it, and if there are problems. This data can be used to enhance performance.
If you are teaching a chatbot development course, analytics should be part of the curriculum. Once the class has completed bot building, have them share their bots with classmates. These interactions, will not only be an excellent way to celebrate students’ accomplishments, but it will provide valuable data to analyse later.
Data Analytics and Student Learning Outcomes
A data analysis assignment using chatbots develops logical thinking skills. It targets several learning objectives:
- demonstrate an understanding of chatbot terminology
- formulate questions about chatbot usage and bot performance
- interpret the meaning of chatbot statistics and describe how it relates to chatbot usage
- recommend a solution for improving bot performance based on data analysis
- effectively communicate findings
- simulate the job duties of a chatbot developer to gain an appreciation for this STEM career
Terminology and Chatbot Analytics
You don’t need your students to analyze every data point. Instead, focus on the information that can help them improve their chatbot. Every platform has their own labels to report chatbot analytics. However, below are some common terms and their meaning.
Conversation: A conversations is a chat session. A chat session begins when a user opens the chatbot and ends when the user closes the chatbot window. This data tells a chatbot developer how often people are using the chatbot.
Message: A message is a pair group that includes a user’s question and a bot’s answer. For example, if the user asks, “can you tell me what time you open?” and then the bot replies “9:00 a.m.”, this exchange equals one message.
Engaged Conversation: An engaged conversation is a chat session with at least three messages. This data measures customer engagement. When a customer asks the chatbot several questions, this indicates, they are finding the conversation useful. Whereas, if a customer only types one or two messages and then exits the chatbot, it can indicate that the person did not find the exchange meaningful.
Default Responses: A default response is a reply a bot outputs when it does not understand the user’s input. It might say, “I would like to help. Could you rephrase your question?” The more default responses a chatbot gives, the less helpful it is to customers. This data lets a developer know if the chatbot requires training to improve its ability to accurately provide answers. To fix the problem, the developer will often view the conversation log to discover what the customer wants to know and then add a correct response.
Confidence: Confidence is the percent a bot believes its response is correct. 100% is very confident because the bot is using a known answer in its response list. 0% is not confident because the bot does not understand and uses a default response. If a chatbot has a low confidence level, a chatbot developer needs to view the conversation log and edit the response to improve the bot’s confidence.
Questions About Chatbot Analytics to Ask Your Students
- Which day had the greatest number of conversations?
- What is the greatest number of messages you had in one day?
- What is the greatest number of engaged conversations you had in one day?
- Why might a high level of engagement result in a person becoming a new customer?
- If a bot has very few engaged conversations what should a chatbot developer do to fix the problem?
- What is the greatest number of default responses you had in one day?
- Did your bot’s default responses decline over time? If yes, what does that tell you about the chatbot?
- If a bot has many default responses what should a chatbot developer do to fix the problem?
- What is the greatest percentage of confidence you had in one day?
- Did your bot’s confidence level increase over time? If yes, what does that tell you about the chatbot?
- If a bot has a low confidence level what should a chatbot developer do to fix the problem?
Which Chatbot Platform Should You Use to Teach Students?
If you are unsure about which bot-builder to use for your course, Bot Libre is ideal for a high school course. It includes simple tools for making a chatbot, as well as an Analytics feature. However, when teaching older students in college or university you should introduce professional tools such as Google Cloud Dialog Flow or IBM WatsonX. Both provide statistics on chatbot use.
TechnoChatbot Course
TechnoKids has a chatbot course ideal for highschool students. It includes step-by-step instructions that explain how to build a chatbot and train it to improve responses. The chatbot is an animated avatar that looks like a teenager. The assignments are ideal for students in Grades 8-12. The tasks provide a foundational knowledge of chatbots.