In this article, let us discuss the following:
...
Metric | Details |
|---|---|
Total Messages | The total number of messages between the bot and end users across all channels during the selected period. You can compare the total number of interactions between the bot and your users across different channels over different periods and determine the most popular channel of interaction. This also helps you to analyze how effectively your bots are being leveraged by users for various skills. |
Conversation Transactions | This metric shows the currently used, owned and available conversation transactions. It helps the administrator to ensure compliance with license agreements and purchase more transactions if necessary. Conversation transactions relate to triggering any fulfilment type (text, web service, automation, email) on a custom or ITSM skill. To learn about what are conversation transactions, refer Subscription Information#ConversationTransactionRules. |
| Automated Workflow Transactions | This metric shows the currently used, owned and available automated workflow transactions. It helps the administrator to ensure compliance with license agreements and purchase more transactions if necessary. Automation transactions are in addition to conversation transactions and are used when an automation workflow is triggered. To learn about what are automated workflow, refer Subscription Information#AutomatedWorkflowTransactionRules. |
| Agent Transfers | The number of times users triggered the Transfer to Agent skill during the selected period. Analyzing this metric helps you to understand the skills which need to be built, or modify the existing skills to avoid the agent transfer process. |
View Bot User Details
Users include the New Users, Self-service Users and Analyst users’ metrics. This metrics helps you to understand how effectively the bot is being leveraged by different type of users over a selected period.
...
Training Opportunities include the Unmatched Phrases, Aborted Skills, Average Accuracy of the Bot, Total Phrases Trained and Skills below Fall Below Required Thresholdmetrics. Note that clicking the Training metrics navigates you to the Training page, but does not apply the selected filter.
Metric | Details |
|---|---|
Unmatched Phrases | The percentage of unmatched phrases over the total phrases sent to the bot during the selected period. Analyzing this metrics helps you to determine whether the user entered any phrases other than the ones defined in Luma for a skill, or whether a new skill needs to be built to cater to the users' request. Click the metrics to navigate to the Unmatched Phrases tab on the Training page to view the details. |
Aborted Skills | The percentage of skills that were aborted by the users over the total skills executed during the selected period. Analyzing this metrics helps you to determine the skills which need to be built or enhanced in order to avoid the users from exiting the chat conversations. Analyzing the user's interaction also helps you to understand why the users aborted the conversations. Click the metrics to navigate to the Aborted Skills tab on the Training page to view the details. |
| Average Accuracy of the Bot | Average accuracy of the bot (combined percentage of accuracy of each identified or matched skills over all the phrases sent to the virtual agent). This metric indicates how accurate the bot is at matching user phrases to skills. A lower number indicates that more skills, training and phrases are required. Calculated as the average NLP score of all entered phrases. Analyzing this metrics helps you to start focusing onknow the percentage ofeffort required to improve the bot accuracy. The average bot accuracy for a given period is to be calculated as follows: Average accuracy = Sum of Accuracy of the skills / number of skills * 100. |
| Total Phrases Trained | Percentage of training done manually (by Administrator) and automated (from automated training feature). Clicking this tile navigates you to the Suggested Training tab, where you can assign or unassign phrases to a suggested skill, existing skill or a new skill. |
| Skills below Required Threshold | Percentage of skills that do not meet the configured accuracy. |
View Troubleshooting Information
...
Most Popular Skills include the Custom Self-service Skill, ITSM Self-service Skill, Custom Analyst Skill and Skills which Contributed to 80% of Highest Bot Transactions metrics. Analyzing this metrics helps you to understand which skills the users are using effectively. The metrics indicates the most popular skills out of the complete list of skills, and the number of times the skills were executed.
Metric | Details |
|---|---|
Custom Self-service Skill | The custom skill with permission to self-service users that was executed the most times during the selected period. Click the skill to edit it. |
ITSM Self-service Skill | The ITSM skill with permission to Self-service users that was executed the most times during the selected period. Click the skill to edit it. |
Custom Analyst Skill | The custom skill with permission to Analyst users that was executed the most times during the selected period. Click the skill to edit it. |
| Skills Contributed to 80% of Highest Bot Transactions | The skills which that together contribute to 80% of the total bots conversation transactions. Click |
...