Measure Customer Satisfaction

There are many different factors to consider when judging the success of your bot, but one of the most important ones is customer satisfaction, often referred to as CSAT. It is essential to understand if users are satisfied with their interactions with your bot.

To measure customer satisfaction, a satisfaction survey can be triggered directly in the chat conversation. The survey is related to the last executed skill or their last transfer to agent interaction. This provides insight into overall satisfaction with the bot as well as specific communications with agents and skills. Based on survey results, you can analyze which skills are working well and which skills need to be improved.

In general, surveying ITSM users is difficult as they do not want to spend the time to fill out an online form or reply to an email. Luma makes this much easier by asking users a single question with quick replies (buttons) directly in the chat conversation. This should yield much better response rates than a typical online survey.

View and Edit the Satisfaction Survey

An Administrator can view and edit the satisfaction survey details. To do so:

On the bot dashboard, from more options , click Bot Configuration.
On the Bot Configuration page, click the Satisfaction Survey tab as shown below.
Select the toggle button to enable the satisfaction survey. After saving the survey, the satisfaction survey will be triggered to users after they execute the skills for the configured number of times in the above settings and complete the transfer to agent conversations.
Note

An Administrator can view the satisfaction survey results on the bot dashboard only if the Enable Satisfaction Survey option is selected.
Skill Survey Frequency denotes the number of skills executed after which the survey is triggered. A value of 1 means that users will be surveyed after every custom or ITSM skill they execute. The best practice is to survey users after every 5 or more executed skills as to not annoy them with constant surveys.
Agent Survey Frequency denotes the frequency to survey users after a transfer to agent chat conversation completions . This is separate from the Skill Survey Frequency setting as many organizations want to measure every Agent Chat interaction. The results for this survey are also tracked separately from Skill surveys.
The Skill Prompt is the message displayed to the user during the skill survey. This prompt should be short and simple and make it easy for the user to understand that they are responding to a survey.
The Agent Chat Prompt is the message displayed to the user during the agent chat survey.
An Administrator can edit the Response Values details. A higher value indicates a better experience. Display labels will be shown to the user from left to right for rich text channels and top to bottom for text only channels.
Note

Any input that does not match a rating or synonym is considered a skipped or ignored survey response and triggers a new request.
View and edit the User Rating Label and Synonyms details if required.
Type the commands for the end user to type to ignore the survey in the Ignore Survey Commands field.
Specify the Post Survey Rating Message, which appears once the user submits the survey response.
Specify the Post Survey Ignore Message, which appears when the user ignores the survey.

Once the user executes the configured number of skills or agent transfers in a chat channel, the satisfaction survey is triggered. User is prompted to share their feedback by replying to the quick reply options as shown below.

The user's response to the Survey is recorded and used to determine the Survey results for your bot. In case, user types a phrase other than the survey options, the user phrase is recognized as a new request and is processed as a new conversation.

View Survey Results

An administrator can view and download the survey results if the Enable Satisfaction Survey option is enabled. To view the survey results, navigate to the bot dashboard. The Satisfaction Survey tiles appear as shown below, which consists of the skills average and the agent transfers.

Skills: This tile shows your customer skill satisfaction rating, the number of responses and the response rate during the selected time period.

Skills Average: The Skills Average is calculated based on the sum of all skill survey responses divided by the number of responses in the selected time period. Any ignores or skips are not included in the calculation.
Responses: The Responses is simply the total number of responses to the skill survey during the selected time period.
Response Rate: The Response Rate is determined by the total responses divided by the number of surveys shown to users in the selected time period. Users are only surveyed after ITSM and Custom skills.

Transfer to Agent: This tile depicts your agent chat satisfaction rating, the number of responses and the response rate during the selected time period.

Transfer to Agent Average: The Transfer to Agent Average is calculated based on the sum of all transfer to agent survey responses divided by the number of responses in the selected time period. Any ignores or skips are not included in the calculation.
Responses: The Responses is simply the total number of responses to the transfer to agent survey during the selected time period.
Response Rate: The Response Rate is determined by the total responses divided by the number of surveys shown to users in the selected time period.

Click to download the survey results in CSV format as shown below. This includes all skill and transfer to agent surveys including where the user ignored or skipped the survey.