Once the Bot is deployed and Skills are built, End-users can start interacting with the Virtual Agent. You can get insight into the effectiveness of Luma Virtual Agent in your organization by tracking and measuring user interactions with the Bot, Skill utilization, Training opportunities.
Luma VA provides the following Dashboards to monitor the performance of the Bot.
KPI Dashboard provides Organizational Key Performance Metrics in terms of Money, Time saved, and the value that Luma VA brings to the organization. It enables you to view the key metrics related to Return On Investment with Luma VA and helps you understand how your users are leveraging the Bot. For more information, refer to KPI Dashboard.
Usage Dashboard provides an insight into how the bot is being utilized in your organization. It provides details on License consumption, User Adoption/Engagement, Skills adoption, and Training opportunities. These metrics aid you to understand and monitor Skills utilization, Unmatched user phrases that help to identify training opportunities for your Bot. For more information, refer to Usage Dashboard.
Following are a few monitoring parameters that you can check on these Dashboards:
How to view Key Performance Metrics for your Organization?
To view KPI metrics for your bot, follow the below steps:
Navigate to Bots Menu → Dashboard → KPI Dashboard.
Select the Date Range from the predefined list. You may select custom to set a specific date and time range.
Click Apply.
The metrics will be recalculated for the selected date range.
How to view and download the License consumption report?
Navigate to Bots Menu → Dashboard → Usage Dashboard.
In License Consumption metrics, view the Total Purchased License, Consumed Licenses (to date), and Available Licenses.
Click on the download button to download the License Consumption Report in a CSV file.
Alternatively, you may also view the report:
Navigate to Tenants Menu → Settings → Subscription.
In the License details section, click on the View Consumption button on the Consumed field.
You can now view the License Consumption report. Click on conversation count to view the conversation details in Conversation History.
Click on the download button to download the License Consumption Report in a CSV file.
How to view the identified Training Opportunities?
Navigate to Bots Menu → Dashboard → Usage Dashboard.
Select the Date Range from the predefined list. You may select custom to set a specific date and time range.
Click Apply.
The Training Opportunities metrics will be recalculated for the selected date range. You can view the training opportunities i.e. count of unmatched phrases identified by the bot, Average NLP skill matching score for executed skills, and number of Ambiguities identified and resolved by the Bot during the selected date range.
How to measure Skill utilization in your bot?
Navigate to Bots Menu → Dashboard → Usage Dashboard.
Select the Date Range from the predefined list. You may select custom to set a specific date and time range.
Click Apply.
The most popular or executed skills during the selected date range are populated in the Skill Adoption section.
Select Include System Skills to include out-of-the-box system skills to the calculation.
Click on the download button to download the Skill Utilization Report in a CSV file.
Alternatively, you can also view the skill utilization details on Skill Builder:
Navigate to Bots Menu → Skill Builder → Skills page.
Sort the list (descending order) on the Total Executions column.
The Custom Skills in your Bot will be sorted based on the total number of executions to date.