Natural Language Processing (NLP) is a component of Artificial Intelligence (AI) that helps computers analyze and understand human language. In the Luma Virtual Agent platform, NLP helps the bot understand a user's phrase and match it to a skill or helps parse user input to populate an attribute using synonyms. During the process of matching the user phrase, the NLP module calculates the probability or confidence score of each Skill and Attribute value.
The following configurations are available in the NLP Settings tab to tailor the NLP engine for your bot and how to respond when the user's input does not strongly match one or any skills.
The Fallback process occurs when the bot is not able to identify high confidence matching skills based on the user's phrase. The Fallback process is triggered based on the Fallback Threshold parameter defined by the Administrator. This parameter defines the minimum confidence score required that must be returned before a skill is executed. If all matching skills fall below this threshold then the Fallback process is triggered and the user is encouraged to rephrase their request. To learn how the Fallback process works in practice, see the Examples section below.
Using the Fallback section, an Administrator can define the bot responses that inform the user when the bot is not able to understand their request.
Define the Smalltalk Threshold: Small Talk Threshold aids in avoiding the Virtual agent to match the custom-defined service requests with Small Talk in the bot. This ensures that the Small talk skills are matched only when no custom skills are identified and the confidence score is above the Small Talk threshold set. The Smalltalk Threshold must be in the range from 0.0 to 1.0. It is recommended to set the Small Talk threshold value high.
The following image shows the conversation based on the Fallback behavior configuration. For the user's phrase, "reset password in oracle", the fallback process triggered and one of the Bot Responses that was configured from the list, "Sorry, could you say that again?" is shown to the user.
Disambiguation is the group of settings that defines what happens when all returned skills are above the Fallback Threshold but there isn't a single high confidence score. Using the NLP Settings, the administrator can define what the bot should do in this case. You may want your bot to take the best result and execute the skill or prompt the user to let them select a result.
There are two main settings under Disambiguation:
The reason why the Disambiguation process exists is the NLP engine may match the user's phrase to a skill which is not a highly confident match and directly execute it. If this is a wrong skill, it may create an unsatisfactory user experience. Hence, prompting the user for confirmation is always a viable option. However, it's important to adjust these settings during user acceptance testing as you may not want users to be prompted for every phrase they enter.
The following image depicts the Disambiguation section in NLP Settings.
Proximity Range: Define the required confidence gap between skills. A value from 1 to 10 is ideal but the field supports any integer from 0 to 100.
Suggestion Settings: In this section, configure the settings when a phrase results in multiple skills are within the defined proximity range.
Proximity Range is considered for all the matched skills with the confidence scores that are below the Confidence Threshold and above the confidence Threshold. |
Confirmation Settings: Configure the settings to reaffirm the user decision when there only one matching skill with the confidence score that is below the Confirmation threshold:
The Confirmation Threshold Message is only shown when the best matching skill is above the Fallback Threshold, below the Confirmation Threshold and there are no other matching skills within the Proximity Range. |
Save: Click Save to apply the configuration settings.
In the below example, for the user phrase, "update ticket", since the confidence is below the Affirmation Threshold, 0.4, the configured message is shown to the user asking for the affirmation, Yes and No. If the affirmation is set to Yes, the bot continues to execute the identified skill. If the affirmation is set to No, the Rephrase Text configured is shown to the user.
During the initial rollout of a new bot, it is recommended to have the Proximity Range set high (20%-30%). This ensures that users are presented with the right skill, however, this hampers the user experience and makes the bot look unintelligent because the user has to confirm everything. As the training data evolves over time, reduce the Proximity Range to a lower value (5-10%) to reduce the number of confirmations required by users.
Let's consider the following scenarios and configuration to illustrate how the Disambiguation settings work. The configured Fallback Threshold is 0.3, the Confirmation Threshold is 0.5, and the Proximity Range is 5%.
