In this article, let us discuss the following:
The Bot Dashboard is the home page of your Bot and displays the usage, user engagement, training opportunities, troubleshooting information, and popular Skills as well as key navigation and configuration information. The Dashboard helps you understand key performance metrics for your bot. You can view the metrics for a selected period, the maximum period being one year. The Bot Dashboard metrics help you understand how your users are leveraging the bot. The dashboard allows you to navigate to the related pages to view more details. Click the Training Opportunities tiles to navigate to the Training page; likewise, click the Troubleshooting tiles to navigate to the Troubleshooting page, and the Most Popular Skills tiles to navigate to the respective skills page.
The following is the Bot Dashboard which shows metrics and the various options available on this page.
Filter Dashboard Metrics
All the dashboard metrics can be filtered using any of the following values from the list.
- Last 7 days
- Last 30 days
- Last 90 days
- Last 180 days
Custom Filter: To view the dashboard metrics for your own defined date range, select the Custom option from the list. You can select the From Date and To Date from the respective calendars. Click Apply to view the metrics for the period.
Usage helps you to understand the bot usage, which includes the Total Messages, Conversation Transactions, Automated Workflow Transactions and Agent Transfers metrics.
The total number of messages between the bot and end-users across all channels during the selected period. You can compare the total number of interactions between the bot and your users across different channels over different periods and determine the most popular channel of interaction. This also helps you to analyze how effectively your bots are being leveraged by users for various skills.
Used Conversation Transactions
This metric shows the number of used conversation transactions during the selected period. It helps the administrator to ensure compliance with license agreements and purchase more transactions if necessary. Click to view the total purchased, used and available transactions in Subscription Information tab of the Settings menu. Conversation transactions relate to triggering any fulfillment type (text, web service, automation, email) on a custom or ITSM skill. To learn about what are conversation transactions, refer Subscription Information#ConversationTransactionRules.
|Used Automated Workflow Transactions|
This metric shows the currently used automated workflow transactions. Click to view the total purchased, used and available transactions in Subscription Information tab of the Settings menu. For more details, see Subscription Information.
|Agent Transfers||The number of times users triggered the Transfer to Agent skill during the selected period. Analyzing this metric helps you to understand the skills which need to be built, or modify the existing skills to avoid the agent transfer process.|
Users include the New Users, Self-service Users, and Analyst users’ metrics. This metrics helps you to understand how effectively the bot is being leveraged by different types of users over a selected period.
The number of users that interacted with this bot for the first time during the selected period.
The number of unique self-service users that interacted with this bot during the selected period.
The number of unique Analyst users that interacted with this bot during the selected period.
View Training Opportunities
Training Opportunities include the Unmatched Phrases, Aborted Skills, Average Accuracy of the Bot, Total Phrases Trained and Skills Fall Below Required Threshold metrics. Note that clicking the Training metrics navigates you to the Training page, but does not apply the selected filter.
The percentage of phrases entered by users that did not match a skill. Calculated as unmatched phrases divided by total phrases for the selected time period.
Analyzing this metric helps you determine whether the user entered any phrases other than the ones defined in Luma for a skill, or whether a new skill needs to be built to cater to the users' request. For well training bots, this percentage should decrease over time.
Click the metric to navigate to the Unmatched Phrases tab on the Training page to view the details.
The percentage of skills that were exited early by users (i.e. aborted skills). Calculated as aborted skills divided by total executed and aborted skills during the selected period.
Analyzing this metric helps you determine the skills which need to be built or enhanced in order to avoid the users from exiting the chat conversation. Analyzing the user's interaction also helps you to understand why the users aborted the conversations.
Click the metrics to navigate to the Aborted Skills tab on the Training page to view the details.
|Average Accuracy of the Bot|
This metric indicates how accurate the bot is at matching user phrases to skills. A lower number indicates that more skills, training, and phrases may be required. Calculated as the average NLP score of all entered phrases.
Analyzing this metric helps you monitor if accuracy is improving over time. For new bots, you should aim to have a score of greater than 70%. Well trained bots should target being above 80%.
The average bot accuracy for a given period is calculated as follows:
Average accuracy = Sum of Accuracy of the skills / number of skills * 100.
|Total Phrases Trained|
The number of phrases added to skills by type of training during the selected time period. User Assisted training is based on phrases added when Automated Training is enabled. Administrator Assisted is the number of phrases that were manually assigned through Training.
Clicking this tile navigates you to the Suggested Training tab, where you can assign or un-assign phrases to a suggested skill, existing skill or a new skill.
To enable Automated Training see Manage Bot Configuration
|Skills below the Required Threshold|
The percentage of phrases that are not meeting the configured NLP threshold. When a phrase doesn’t meet the threshold a user will be prompted with the Validation message.
To view and edit your Bot's configured NLP threshold see Manage Bot Configuration
View Troubleshooting Information
Training Opportunities include the Errors and Warnings metrics. Note that clicking the Troubleshooting metrics navigates you to the Troubleshooting page, but does not apply the selected filter. Analyzing this metrics helps you to train your bot in order to effectively interact with the users.
The number of errors that occurred during interactions with this Bot in the selected period. Click the metric to navigate to the errors log on the Troubleshooting page to view the details. Note that this will filter the Troubleshooting page to only show Errors. To review the context and history surrounding a specific error you should filter by the time the error occurred.
The number of warning messages that occurred during interactions with this Bot in the selected period. Click the metrics to navigate to the warnings log on the Troubleshooting page to view the details. Note that this will filter the Troubleshooting page to only show Warnings. To review the context and history surrounding a specific warning you should filter by the time the warning occurred.
View Most Popular Skills
Most Popular Skills include the Custom Self-service Skill, ITSM Self-service Skill, Custom Analyst Skill and Skills which Contributed to 80% of Highest Bot Transactions metrics. Analyzing these metrics helps you to understand which skills the users are using effectively. The metrics indicate the most popular skills out of the complete list of skills, and the number of times the skills were executed.
Custom Self-service Skill
The custom skill with permission to self-service users that was executed the most times during the selected period. Click the skill to edit it.
ITSM Self-service Skill
The ITSM skill with permission to Self-service users that was executed the most times during the selected period. Click the skill to edit it.
Custom Analyst Skill
The custom skill with permission to Analyst users that was executed the most times during the selected period. Click the skill to edit it.
|Skills Contributed to 80% of Highest Bot Transactions|
The skills that together contribute to 80% of the bots conversation transactions. Click to download and view the list of skills which contribute to the highest bot transactions.
Satisfaction Survey Skills include the Skills Average and Transfer to Agent Average metrics. Analyzing this metrics helps you to understand the responses received to the satisfaction surveys prompted to the users. The satisfaction survey metrics can be downloaded by clicking .
The skills average is calculated based on the sum of all skill survey rating responses divided by the number of responses. The response rate is determined by the total responses divided by the number of surveys shown to the users. Users are surveyed after the ITSM and Custom skills are completed.
Transfer to Agent Average
The Transfer to Agent Average is calculated based on the sum of all Transfer to Agent survey rating responses divided by the number of responses. The response rate is determined by the total responses divided by the number of surveys shown to users.
Perform Bot Actions
On the Bot Dashboard page, you can perform the following actions.
|FAQs||Click FAQs to navigate to the FAQs page, where you can view and upload the FAQ documents. For more details, see the FAQs page.|
|Skills||Click Skills to navigate to the Skills list page, which displays the existing skills. You can also add new skills or add from the catalog and perform additional actions. For more details, see the Skills page.|
|Training||Click Training to navigate to the Training page, which displays the Unmatched Utterances and Aborted Skills details. For more details, visit the Training page.|
|Troubleshooting||Click Troubleshooting to navigate to the Troubleshooting page, which displays the errors and warnings, as well as future bot health metrics. For more details, see the Troubleshooting page.|
|Bot Configuration||Click Bot Configuration to navigate to the Configuration page, which displays the Bot Information and Follow-up tabs. For more details, see the Manage Bot Configuration page.|