Tuning NLP Thresholds

Natural Language Processing (NLP) enables your Bot to understand user messages and derive meaning from user text inputs. It helps provide context and meaning to user inputs so that the Bot can respond appropriately. With NLP, you can “train” your Bot on the various possible questions which end-users may ask, and suitable responses.

Luma Virtual Agent uses NLP to understand the user's intent and match it to the most relevant skill. It calculates the matching probability or confidence score of each Skill and executes the best match. The Virtual Agent can also parse the user input to extract information and store it in attributes.

Following NLP configurations in Luma Virtual Agent govern how your bot responds to the end-users requests. These settings are out-of-the-box and can be fine-tuned based on your Organization’s need.

Confirmation Threshold

This is the minimum confidence score required to execute a skill without prompting the end-user for confirmation.

1. Navigate to Bot → Settings → NLP Settings.

2. On the Disambiguation tab, scroll to Confirmation Settings.

3. Set Confirmation Threshold between 0.0 to 1.0.

For a user’s request, if the bot identifies a skill with a confidence score above the defined Threshold, the Skill is directly executed. If one or more skills are identified, the skill with the highest confidence score is automatically executed by the bot.

Fallback

The Fallback process occurs when the bot is not able to identify any high confidence matching skills based on the user's phrase. If the confidence score for all the skills falls below this threshold then the Fallback is triggered and the user is encouraged to rephrase their request. Refer to Customize System Skills behavior for more information on the Fallback process.

On Fallback section:

1. Set Skill Fallback Threshold between 0.0 to 1.0. The threshold should be less than the Confirmation Threshold in the above step.

2. Set Small Talk Fallback Threshold between 0.0 to 1.0. The threshold is used in matching user phrases with Small Talk and aids the Virtual agent to avoid matching the custom-defined service requests with Small Talk in the bot. It is recommended to set the Small Talk threshold value high.

3. Add Bot response to be shown to the end-user after repeated fallback attempts

4. Click on Save.

Disambiguation

The Virtual Agent uses Disambiguation settings to resolve ambiguity in case the bot matches the user's phrase to one or more skills above the Skill Fallback Threshold, but is not highly confident to directly execute it. In such a case Bot prompts the user to select the most viable skill to execute.

Follow the below steps to configure the settings:

1. Navigate to Bot → Settings → NLP Settings → Disambiguation section.

2. Set Proximity Range. It is used to calculate the confidence range for the skills that are viable for disambiguation. In case of ambiguity, the bot prompts the end-user to choose from the skills falling within this range. For more information, refer to Disambiguation.

3. Under Suggestion Settings, set the Number of skills to show. It is the maximum number of skills to be presented to the user as an option, during the disambiguation process.

4. Add Message to be displayed to the user in case of ambiguity.

5. Verify the Confirmation Threshold.

6. Set Automate Training With User Responses to Active to enable Automated training for your bot.

7. Click Save to apply the configuration settings.