Once the Bot is deployed and is being used you might want to Track and Measure Skills are built, End-users can start interacting with the Virtual Agent. You can get insight into the effectiveness of Luma Virtual Agent in your organization by tracking and measuring user interactions with the Bot, how Skills are being utilized, identify Training opportunities, Measure the Performance of Support AgentsSkill utilization, Training opportunities.
Luma VA provides the following Dashboards to monitor the performance of the Bot.
KPI Dashboard
Usage Dashboard
Agent Chat Dashboard
KPI Dashboard
KPI Dashboard provides Organizational Key Performance Metrics in terms of Money, Time saved, and the value that Luma VA brings to the organization. It enables you to view the key metrics related to Return On Investment with the Luma VA and helps you understand how your users are leveraging the Bot.
The following metrics are calculated based on the KPI Settings and the KPI Metrics defined at the Tenant level and each skill level.
Actionable Support Requests
Automated Resolutions
Deflected Support Requests
Follow-ups
Usage Dashboard
Usage Dashboard For more information, refer to KPI Dashboard.
Usage Dashboard provides an insight into how the bot is being utilized in your organization. It provides details on License consumption, User Adoption/Engagement, Skills adoption, and Training opportunities. These metrics aid you to understand and monitor Skills utilization, Unmatched user phrases that help to identify training opportunities for your Bot.
Key metrics related to the following categories are given in the usage Dashboard:
License Consumption
User Adoption
Training Opportunities
Skill Adoption
Agent Chat Dashboard
Agent Chat Dashboard provides the Administrators with real-time Insights related to the performance of Support Agents Groups. It provides details about the Live and Historical conversation. These metrics help you to understand and monitor the current usage and get the metrics related to current and historical chat conversations between support agent groups and users. Helps to plan for staffing levels based on volumes.
Live Metrics
It provides the status of the logged-in Support Agents and details of the User requests received for the Support groups.
Historical Metrics
The Historical Metrics focuses and displays consolidated data of the past information:
Total User Requests
Average Wait Time, Average Chat Time
Agent Accepted Chats, Agents Unaccepted Chats
For more information, refer to Usage Dashboard.
Following are the monitoring parameters that you can check on these Dashboards:
| Table of Contents | ||
|---|---|---|
|
How to view Key Performance Metrics for your Organization?
Key Performance Metricsprovides a view on Return On Investment with the Luma Virtual Agent. Here you can view the cost and time saved to manage Support Requests and automate Follow-ups by using your bot .
To view KPI metricsfor your bot, follow the below steps:
Navigate to Bots Menu → Dashboard → KPI Dashboard.
Select the Date Range from the predefined list. You may select custom to choose for a specific date and time range.
Click Apply.
The metrics will be calculated for the selected date range.
How to view and download the License consumption report?
The License consumption report provides the details on your Subscription. It helps you to understand how the Sessions or User Licenses are being used and plan for subscription renewals.
Navigate to Bots Menu → Dashboard → Usage Dashboard.
In License Consumptionmetrics, view the Total Purchased License, Consumed Licenses (to date), and Available Licenses.
Click on the download button to download the License Consumption Report in a CSV file.
Alternatively, you may also view the report:
Navigate to Tenants Menu → Settings → Subscription.
In the License details section, click on the View Consumption button on the Consumed field.
You can now view the License Consumption report. Click on conversation count to view the conversation details in Conversation History.
Click on the download button to download the License Consumption Report in a CSV file.
How to view the identified Training Opportunities?
The metrics helps you to understand the training needs of your Bot. You can view the training opportunities i.e. count of unmatched phrases identified by the bot, Average NLP skill matching score for executed skills, and number of Ambiguities identified and resolved by the Bot .
Navigate to Bots Menu → Dashboard → Usage Dashboard.
Select the Date Range from the predefined list. You may select custom to set a specific date and time range.
Click Apply.
The Training Opportunities metrics will be calculated for the selected date range.
How to measure Skill utilization in your bot?
Skill Utilization report highlights the most popular skills of your Bot based on the number of skill executions by the end-users. To view the report:
Navigate to Bots Menu → Dashboard → Usage Dashboard.
Select the Date Range from the predefined list. You may select custom to set a specific date and time range.
Click Apply.
The most popular or executed skills during the selected date range are populated in the Skill Adoption section.
Select Include System Skills to include out-of-the-box system skills to the calculation.
Click on the download button to download the Skill Utilization Report in a CSV file.
Alternatively, you can also view the skill utilization details on Skill Builder:
Navigate to Bots Menu → Skill Builder → Skills page.
Sort the list (descending order) on the Total Executions column.
The Custom Skills in your Bot will be sorted based on the total number of executions to date.
How to measure User Adoption for your bot?
User Adoption indicates how users are interacting with your bot. It represents how many registered users are using Luma Virtual Agent and the average time spent interacting with the Bot.
Navigate to Bots Menu → Dashboard → Usage Dashboard.
Select the Date Range from the predefined list. You may select custom to set a specific date and time range.
Click Apply.
In the User Adoptionsection, you can view the count of Registered Users who interacted with Luma Virtual Agent during the selected date range. You can view the count of existing and new Registered who interacted with your bot during the selected date range and the Average time spent by registered users in a conversation with the bot.