Monitoring the Bot

Owned by Bhavani.tvr (Unlicensed)
Last updated: Feb 09, 2021
4 min read

Once the Bot is deployed and Skills are built, End-users can start interacting with the Virtual Agent. You can get insight into the effectiveness of Luma Virtual Agent in your organization by tracking and measuring user interactions with the Bot, Skill utilization, Training opportunities.

Luma VA provides the following Dashboards to monitor the performance of the Bot.

KPI Dashboard provides Organizational Key Performance Metrics in terms of Money, Time saved, and the value that Luma VA brings to the organization.  It enables you to view the key metrics related to Return On Investment with Luma VA and helps you understand how your users are leveraging the Bot. For more information, refer to KPI Dashboard.

Usage Dashboard provides an insight into how the bot is being utilized in your organization. It provides details on License consumption, User Adoption/Engagement, Skills adoption, and Training opportunities. These metrics aid you to understand and monitor Skills utilization, Unmatched user phrases that help to identify training opportunities for your Bot. For more information, refer to Usage Dashboard.
Following are the monitoring parameters that you can check on these Dashboards:

How to view Key Performance Metrics for your Organization?

Key Performance Metrics provides a view on Return On Investment with the Luma Virtual Agent. Here you can view the cost and time saved to manage Support Requests and automate Follow-ups by using your bot .

To view KPI metrics for your bot, follow the below steps:

  1. Navigate to Bots Menu → Dashboard → KPI Dashboard.

  2. Select the Date Range from the predefined list. You may select custom to choose for a specific date and time range.

  3. Click Apply.

The metrics will be calculated for the selected date range.

How to view and download the License consumption report?

The License consumption report provides the details on your Subscription. It helps you to understand how the Sessions or User Licenses are being used and plan for subscription renewals.

  1. Navigate to Bots Menu → Dashboard → Usage Dashboard.

  2. In License Consumption metrics, view the Total Purchased License, Consumed Licenses (to date), and Available Licenses.

  3. Click on the download button to download the License Consumption Report in a CSV file.

Alternatively, you may also view the report:

  1. Navigate to Tenants Menu → Settings → Subscription.

  2. In the License details section, click on the View Consumption button on the Consumed field.

  3. You can now view the License Consumption report. Click on conversation count to view the conversation details in Conversation History.

  4. Click on the download button to download the License Consumption Report in a CSV file.

How to view the identified Training Opportunities?

The metrics helps you to understand the training needs of your Bot. You can view the training opportunities i.e. count of unmatched phrases identified by the bot, Average NLP skill matching score for executed skills, and number of Ambiguities identified and resolved by the Bot .

  1. Navigate to Bots Menu → Dashboard → Usage Dashboard.

  2. Select the Date Range from the predefined list. You may select custom to set a specific date and time range.

  3. Click Apply.

The Training Opportunities metrics will be calculated for the selected date range.

How to measure Skill utilization in your bot?

Skill Utilization report highlights the most popular skills of your Bot based on the number of skill executions by the end-users. To view the report:

  1. Navigate to Bots Menu → Dashboard → Usage Dashboard.

  2. Select the Date Range from the predefined list. You may select custom to set a specific date and time range.

  3. Click Apply.

  4. The most popular or executed skills during the selected date range are populated in the Skill Adoption section.

  5. Select Include System Skills to include out-of-the-box system skills to the calculation.

  6. Click on the download button to download the Skill Utilization Report in a CSV file.

Alternatively, you can also view the skill utilization details on Skill Builder:

  1. Navigate to Bots Menu → Skill Builder → Skills page.

  2. Sort the list (descending order) on the Total Executions column.

  3. The Custom Skills in your Bot will be sorted based on the total number of executions to date.

How to measure User Adoption for your bot?

User Adoption indicates how users are interacting with your bot. It represents how many registered users are using Luma Virtual Agent and the average time spent interacting with the Bot.

  1. Navigate to Bots Menu → Dashboard → Usage Dashboard.

  2. Select the Date Range from the predefined list. You may select custom to set a specific date and time range.

  3. Click Apply.

In the User Adoption section, you can view the count of Registered Users who interacted with Luma Virtual Agent during the selected date range. You can view the count of existing and new Registered who interacted with your bot during the selected date range and the Average time spent by registered users in a conversation with the bot.