Use the Sidekick Assists by Classification Topic report to understand how well Sidekick automates responses and helps Customers. It also shows you which types of Conversations are not being automated well by Sidekick, providing insights into areas where you could build new Threads or enhance existing ones.
The Sidekick Assists by Classification Topic report shows you how Sidekick is classifying the Conversations that it assists. Use the table in the report to understand why end Customers are reaching out, and how Sidekick is handling those Conversations.
Note that the “Classification Topics” applied by Sidekick are distinct from the Topics applied in typical, non-Sidekick Conversations.
How data in the report is measured #
The report is measured by the following:
Grouping
- Grouped by Conversations.
Time Anchor
- Data is time anchored using ‘Conversation Created At’ in the UI and API.
Metrics used for the Sidekick Assist by Classification Topic report #
Review the definition of each metric and how it’s used to calculate data. These are fields you’ll see in the CSV download. The columns in the report fill dynamically when the information is available and may change as the Tasks are updated.
| Metric | Description | Comments |
|---|---|---|
| Classification Topic Hierarchy | The full classification hierarchy for the Conversation, as determined by Sidekick. For example, “Returns > Return Shoes > Return Heeled Shoes.” | Note that the “Classification Topics” applied by Sidekick are distinct from the Topics applied in typical, non-Sidekick Conversations. |
| Count of Automated Resolutions | Assisted Conversations where Sidekick completed the steps specified in the Thread without handing off to an Agent. | |
| Count of Hero-Assisted Handoffs | Number of Assisted Conversations that were deliberately handed off to an Agent in the thread configuration. | |
| Count of Insufficient Message Handoffs | Conversations in which Sidekick began assisting the Customer but then handed off to an Agent to Resolve the inquiry because the Conversation lacked sufficient information for Sidekick to complete the steps specified in the Thread. | Only applies to Legacy Flows, not Threads. |
| Count of No Response Auto-Closed Waiting Conversations | Number of Long Waiting Conversations that have been waiting longer than 24 hours since the start of the first Thread and have been auto-closed. | Sidekick Conversations waiting on a Customer’s response for longer than 24 hours are considered long waiting. If an Agent closes the Conversation in Gladly, it will remain a Long-Waiting Conversation in reporting. We auto-close a Conversation in Gladly when a Customer fails to respond to Sidekick within two hours in Automation API Conversations. If a Conversation uses Messaging API, we never auto close, and it remains Waiting indefinitely. |
| Count of Waiting Conversations | Number of Sidekick Conversations waiting for a response from the Customer. | |
| Count of Unexpected Handoffs | Number of Conversations that Sidekick unexpectedly handed off to an Agent. | Anything that is not an intentional transfer in Threads or an automated resolution is considered an Unexpected Handoff. Any error in Threads will be considered an Unexpected Handoff. Threads can throw errors at any point, even when they have provided sufficient value to a Customer. Threads throw errors in situations we could conceptually categorize as expected and in situations we would consider Insufficient Message Handoffs. |
| Count of Unassisted Conversations | These are Conversations where Sidekick was able to classify the Conversation (to understand the Customer’s intent), but where no Threads exist. | These are Conversations where Sidekick could classify the Conversation (to understand the Customer’s intent), but where no Threads exist. |
How the Sidekick Assist by Classification Topic report is filtered #
Use these filters to modify the data in your report. Using a combination of filters will also provide more targeted results. You can learn more about filters here.
Date
- Lists all Sidekick Conversations within the specified date or date range.
Channel
- The Channel Sidekick evaluated or communicated in Conversations.
FAQs #
What are ‘Long-Waiting’ Conversations and what can we do to reduce this number?
Long-Waiting Conversations are Conversations that have been waiting for a response from the Customer for more than 24 hours. It’s important to note that even after an Agent closes the Conversation within Gladly, these Conversations will continue to count as Long-Waiting within reporting.
Here are some potential case scenarios leading to a high count of Long-Waiting Conversations:
- The Customer no longer needs support or has gotten help elsewhere. As they don’t need further assistance from Sidekick, they do not follow up with the requested information.
- The Customer does not understand that they will need to provide additional information to move forward with their inquiry.
- In this case, reviewing the phrasing of the question or reply by Sidekick may improve the chances of the Customer responding. Rephrasing questions or replies with clear directions such as “Please reply ‘yes’ or ‘no’” or “Once I receive a response from you, I’ll be able to help you with this request,” etc., may help mitigate the number of long-waiting Conversations.
What can we do if there are too many unexpected handoffs?
A high count of unexpected handoffs indicates that one or more Threads may be hitting an error when executed. Please contact Gladly Support to help you investigate.
How do we reduce the number of unassisted Conversations?
Unassisted Conversations is a helpful metric to consider new areas of opportunity for Sidekick. If there are Conversations within this category that you’re interested in building a Thread to address, please contact Gladly Support.
