Following is an example of a simple search through Search Widget and Luma Virtual agent (Luma Supported Channel). In this example, the end-user is looking for information on resetting their Gmail account password. The search results in a list of relevant knowledge artifacts.
Search Widget
Knowledge Search in Search Widget is based on a Relevancy score. In the below example:
...
Knowledge Search in Luma Knowledge is based on the following:
| Table of Contents | ||||
|---|---|---|---|---|
|
Tenant Configurations
Luma Knowledge uses the following tenant configurations to identify Artifacts for a user query. For example let us consider the following configurations:
Search Configurations | Presentation Settings | ||
|---|---|---|---|
Configuration | Value | Configuration | Value |
Maximum Artifacts per Query | 10 | Maximum decision levels | 1 |
Score Range Percentage | 60 | Maximum Topics per Decision Level | 10 |
Upper Confidence Score Threshold value for LOW Confidence level | 0.75 | Maximum Artifacts per Topic: | 10 |
Upper Confidence Score Threshold value for MODERATE Confidence level | 0.50 | Exclude Articles with Low confidence from search results | No |
Exclude Articles with Moderate confidence from search results | Yes | ||
Relevancy Score
The Relevancy score of an Artifact determines how relevant is the Knowledge available in the Artifact to the user’s question. The relevancy score is automatically calculated by the system based on the availability of terms in the search phrase, term frequency, and other parameters. Luma Knowledge uses the score to filter artifacts that are not relevant to the user’s query.
In our example, the user searches for “How to reset account password.“
Luma Knowledge identifies 3 matching articles and calculates respective relevancy scores:
A1: “Change or reset your Gmail password” with a Relevancy score of 12.78
A2: “Reset a user's password using Azure Active Directory” and a Relevancy score of 11.42
A3: “Send password reset email for an Enterprise ID user” and a Relevancy score of 11.14
A4: “Create a new account in Azure Active Directory” and a Relevancy score of 5
Based on the following tenant configurations, results are displayed on the Search Widget:
...
Search Configurations
...
Presentation Settings
...
Configuration
...
Value
...
Configuration
...
Value
...
Maximum Artifacts per Query
...
10
...
Maximum decision levels
...
1
...
Score Range Percentage
...
60
...
Maximum Topics per Decision Level
...
10
...
Maximum Artifacts per Topic:
...
10
tenant configuration Score Range Percentage, Luma Knowledge filters the Artifacts. All the Artifacts that fall in the Relevancy Score range, i.e., between the highest relevancy score for the matching artifacts (12.78) and the calculated Relevancy Threshold (7.66), are
...
considered relevant Artifacts.
Relevancy Threshold is calculated as the Score Range Percentage (tenant configuration) of the highest relevancy score (60% of 12.78 = 7.66).
...
The search result is displayed as:
...
Luma Virtual Agent
When the Artifact is searched through Luma virtual Agent or any of the Luma support Channels, Knowledge is displayed based on the following factors:
| Table of Contents | ||||
|---|---|---|---|---|
|
Confidence Band
...
Confidence Band
The Confidence Score of an Artifact indicates how confident the system is on in the identified artifact to help the user with their inquiry. The This is calculated based on the metadata generated from the user query user’s query and metadata of the Artifact. The query’s metadata is matched against the Artifact's metadata of the Artifacts available in Knowledge Base. If at least one phrase/value in the metadata type is matched, the respective score is awarded to the Artifact as the Confidence score. For more details on calculating Confidence score, refer to /wiki/spaces/KNOWLEDGEMANAGEMENT/pages/26049413514 Calculating Confidence score.
In the current example,
User searches for “How to reset account password.“
Luma Knowledge calculates the confidence score for the matched Artifacts (identified as per the relevancy score):
Artifact | Relevancy score | Metadata from User query | Matched Metadata | Confidence Score |
|---|---|---|---|---|
A1: “Change or reset your Gmail password.” | 12.78 | TBC | TBC | 0.6 |
A2: “Reset a user's password using Azure Active Directory.” | 11.42 | TBC | TBC | 0.6 |
A3: “Send password reset email for an Enterprise ID user.” | 11.14 | TBC | TBC | 0.55 |
3. Now, based on the following tenant configurations , the matched Artifacts are classified in Confidence Bands. Consider that confidence thresholds for your tenant are as following:Upper Confidence Score Threshold value for LOW Confidence level - 0.75
and Upper Confidence Score Threshold value for MODERATE Confidence level - 0.58, the matched Artifacts are classified into Confidence Bands. The Artifacts are classified as:
Artifact | Relevancy score | Confidence Score | Confidence Band |
|---|---|---|---|
A1: “Change or reset your Gmail password.” | 12.78 | 0.6 | Moderate |
A2: “Reset a user's password using Azure Active Directory.” | 11.42 | 0.6 | Moderate |
A3: “Send password reset email for an Enterprise ID user.” | 11.14 | 0.55 | Low |
Now based on the Confidence Band, Artifacts are displayed to the end-user. As per our configuration, the Artifacts in High and Moderate bands should be displayed in the search result and Artifacts in Low Confidence Score should be ignored.
Exclude Articles with Low confidence from search results is set to No
Exclude Articles with Moderate confidence from search results is set to Yes
...
| Info |
|---|
When searching artifacts in Luma Virtual Agent, the over all Knowledge Confidence Band is used to decide the presentation. The Highest Confidence Band of the identified |
...
Artifacts is considered. For our search, the Knowledge Confidence Band is MODERATE. |
...
The final Artifacts displayed on the Search Widget are:
...
Additional Factors for search through Luma Virtual Agent
When the Artifact is searched through the Luma Virtual Agent or any of the Luma support Channels, the following factors play a role in the presentation of Artifacts:
User Intent
User intent is derived from the search phrase. It indicates if the user wants to view Knowledge or execute the predefined service.
...
Based on the above factors, the system recommends if the end-user should be presented with Knowledge, predefined Service, or both. For our search, the system recommends displaying Knowledge Followed by Skills. For more information on rules for Knowledge and Services display, refer to Integration with Luma Knowledge.
User Intent/ Pre-classification | Knowledge Confidence Band | Luma Skill Confidence Band | DE Recommendation | Notes |
Knowledge | Medium | Low | Knowledge followed by Skill | This indicates that all the Knowledge articles identified are presented before predefined Skills. In this case, Services are not presented upfront and displayed only after the user explicitly requests to see the relevant Services. |
However, based on your organization’s representation preference Presentation Preference, the Luma VA administrator can customize the way Knowledge and Services are delivered to the end-user. You can configure Luma VA to always deliver Knowledge followed by SkillSkills, Knowledge & Service together or honor the DE recommendation. The configuration overrides the DE recommendation. For more information, Knowledge & Services Settings.
For In the current example, let us consider that the Presentation preference for your Luma tenant is Default. This means the DE recommendation (Knowledge followed by Skill) is followedis honored.
Based on the above factors, Luma VA will should present Knowledge Artifacts followed by Identified Skills to the end-user. Since there are no matching skills, so only Knowledge Artifacts will be presented.
...
In the Luma Virtual Agent, the Artifacts are displayed as shown below:
...
| Info |
|---|
Luma Virtual Agent presents the Knowledge Artifacts as identified by Luma Knowledge. Artifact presentation, count, or order is not changed by the Virtual Agent. |