This article details how you can train Luma periodically to improve the accuracy in matching user phrases to skills in your Tenant. Training Luma means training the NLP engine on how to understand the user's language better. The more phrases are fed to the NLP engine, the better it matches skills to the phrases and even matches to phrases that do not exist. This helps to improve the User experience with Luma.

Overview

A Luma Skill is triggered when user utterance matches user phrases associated with the skill. While building the skill, Developers and Administrators can provide a list of user phrases by which the user can request the skill in different ways. When Luma does not find any matching skill for an utterance above the configured fallback threshold, it is marked as 'Unmatched Phrase'.

Unmatched phrases assist Skill Developers and Administrators in two areas of continuous improvement:

• Train the NLP engine by adding unmatched user phrases to existing skills

• Identify unresolved user requests by the Bot so that these requests could be developed as skills later.

There are two ways to train Luma on these unmatched phrases:

1. Supervised Training

2. Automated Training

Supervised Training

As part of Supervised training, a Tenant administrator or Developer can identify and associate unmatched phrases to System Skills and Custom Skills available in Luma.

Unmatched Phrases

Navigate to the Unmatched Phrases tab on the Supervised Training Page under Bot.

Open

All Unmatched Phrases i.e. phrases entered by users that did not match to any skill as per NLP Settings Configuration are available on this tab. You may review and validate these phrases before training Luma to identify the phrases.

Below information is available for each phrase in the list:

• Phrase: Displays the phrase entered by the user that did not match any skill available in the system.

• Conversations: Displays the total number of conversations (including conversations through Test Widget) in which the unmatched phrase was used by the specific user during the conversations with Luma.

• Last Used Channel: Displays Channel name where the unmatched phrase was used the last time.

• Last Used On: Displays the date and time details when the unmatched phrase was last entered.

• Last Used By: Displays user details who has used the unmatched phrase.

• Status: Displays the current status of the unmatched phrase

  • Unresolved: Default status of Unmatched Phrases. This indicates that the phrase is pending for review or waiting to be assigned to a skill by the tenant administrator or developer.

  • Reviewed: The status indicates that the phrase is reviewed, but has not been assigned or associated with any skill.

To view more details on the phrase, user’s request, and its context, click on the Conversations count in the record. You will be redirected to the Conversation History page with conversation details. Refer to Conversation History for more details.

Assign Unmatched Phrase to Skill

After the review of any user phrases from the ‘Unmatched Phrases’ list, if the user phrase is related to any of the existing System skill or Custom skill, you may either assign the phrase to the matching skill or create a new skill to associate the phrase.

To assign a phrase to skills in Luma, do the following:

1. Select the required phrases from the list and click on Assign to Skill.

   Open

   

    

2. All existing System Skills and Custom skills in Luma will be listed on the screen.

   1. Below information for each skill will be available on the screen:

      1. Name: Displays the existing skill name.

      2. Identifier: Displays skill identifier that is used internally to identify the skill.

      3. Last Updated On: Displays the date and time when the skill was last updated

      4. Suggested Skill: Displays if the skill is marked as Suggested Skill

      5. Category: Displays the category the skill belongs to.

      6. Published Status: Displays the current Build and Publish status of the Skill.

3. Assign to Skill: If the selected phrase matches an existing skill, associate the skill to an existing skill in Luma

   1. Search for a specific skill using Name or Identifier in the Search field and press Enter

   2. Select the required skill from the list, then click Assign to Skill.

   3. Success message ‘Phrase Assigned to Skill’ will appear on the screen.

      
      Note: If the phrase already exists in the selected skill, an error message would appear and Luma will not associate the phrase to the skill. In this case, the phrase will continue to appear in the Unmatched Phrase list. You may mark the phrase as Reviewed. Refer to Mark as Review section below for more details.

       

4. Once the phrases are associated with the Skill, the phrases are removed from the Unmatched phrases list.

Mark as Reviewed

An Administrator can mark an unmatched phrase as Reviewed in the below scenarios:

• The phrase does not match any existing skill

• User’s request can not be fulfilled through Luma or Administrator does not wish to create a new skill for the request

• The matching skill has already been trained with the phrase, i.e. the phrase has been manually added to the skill.

To review a phrase, do the following:

1. Select one or more phrases from the list.

2. Click Mark as Reviewed.

3. Phrase status will change from Unsolved to Reviewed.

Automated Training

Luma can also be enabled to train itself automatically, based on the phrases used by the End-user to execute a skill. This ensures that Luma automatically associates an unmatched phrase to an existing skill and no manual review/intervention is required from the Administrator or Developer. This reduces the number of Unmatched Phrases in the system.

When Automated training is enabled for your Tenant, a user phase is automatically associated with skill, if the following conditions are met:

1. The confidence score for a user phrase against the skill is below the Confirmation Threshold set for your Tenant.

2. No other skill falls within the proximity range set for your tenant,

3. The user’s response to the confirmation prompt is Yes.

For example:

Consider the below configuration for your Tenant:

• The confirmation Threshold is set to 0.75.

• The proximity range is 10%

• Automated Training is Enabled.

A user requests “Password Required“ in Luma. A skill “Password reset for Outlook“ exists in the system and the requested phrase calculates a confidence score of 0.70 against the skill. If no other skill is available within the proximity range (between 0.65 to 0.75) and the user confirms the skill for execution, the phrase will be associated with the “Password reset for Outlook“ skill automatically.

Refer to NLP Settings for more details on configurations.

To enable Automated training, do the following:

1. Navigate to Bot → Configurations → NLP Settings → Disambiguation tab.

2. Scroll to Confirmation Settings.

3. Toggle the Automate Training With User Responses and set it to Active.

4. Click Save to save the changes.