Reports

Dashboard

The Dashboard in Knovvu Virtual Agent offers a quick snapshot and a deeper understanding of how the virtual agent is performing. It provides key metrics and visualizations for understanding user behavior, measuring effectiveness, and identifying areas for improvement.

Filters

Virtual Agent: This will filter results for specific Virtual Agents that are part of the project. If no project has been filtered, the Virtual Agent filter will be inactive as well as ‘Filter’ button down below. If no VA filter is selected, the data will be retrieved for all the Virtual Agents that are part of the filtered project.
Channel: Channel refers to the integrated channel being used for communication in a project. This includes IVR, Facebook, Webchat etc.
Date: The date filter allows data retrieval for a desired time range. The time range can either be chosen from standard options that are provided, “today”, “yesterday”, “last 7 days”, “last 30 days”, or it can be a custom date range between any two dates of choice.

Filters works with ‘and’. Which means the information displayed on the dashboard to come from data that satisfies all the constraints set by the filters.

Key Metrics

On the top of the dashboard page, there are four boxes placed side by side, displaying the following information:

• Total Sessions: It displays the total number of user sessions that occurred within the selected filter range. A session represents a continuous interaction between a user and the virtual agent.

• Average Session Duration: It represents the average length of the sessions.

• Average Message per Session: It denotes the average number of messages of customers during one session.

• Self-service rate: It measures the percentage of sessions resolved without human agent intervention.

• Fallback Messages:

This graph lists the utterances that resulted in a fallback, that is unmatched intent detection. Fallback messages are displayed when an intent fails to be detected for an utterance. There are two scenarios where a fallback message is triggered:

1. If the VA cannot match any intent over a confidence threshold.
2. If the intent with the highest value match on the confidence threshold is the fallback intent.
   The width of the corresponding bar of the utterance is proportional to the frequency with which the utterance was used and resulted in a fallback. The Fallback Messages widget also has an option that allows the user to download this list.
   Note that fallback utterances can be added to an intent by clicking on them, allowing users to remove them from the fallback list or alternatively, users can create a new intent by clicking on the fallback utterance.

• Intent Overview:

This chart shows the ratios between the intents that were matched and the ones that couldn’t be detected, labeled as “fallback”. This helps in analyzing the performance of the VA’s detection abilities.

• Top Intent:

It lists the intents according to how frequently they have been used by customers. The top 7 intents are displayed here, and if the cursor is hovered on any of them, the number of times the intent was used is shown.

• Intent Trends:

It is a graph that displays the trend of usage over time. The x-axis displays the dates, and the overall graph shows the trend of how frequently the corresponding intent was used over a certain range of time. Trends display in here are the Top Intents.

• Sentiment Trends:

Sentiment trends are depicted using a line graph, with the vertical axis representing sentiment values ranging from -1 to 1, and the horizontal axis indicating dates. Placing the cursor on the trend line reveals the total number and percentage of positive, negative, and neutral conversations within the chosen date range. Additionally, the tooltip displays the average sentiment of all conversations, without categorizing them or considering the total number of conversations. It is important to note that the sentiment averages solely reflect the sentiment of user inputs.

• Average Sentiment:

The average sentiment is divided into five segments: extremely negative, negative, neutral, positive, and extremely positive. The proportion of each segment represents the average score of session scores falling within that particular segment. The value displayed at the center of the graph represents the average score of all sentiment data within the selected date range.

Insights

The Insights tab in Knovvu Virtual Agent provides a detailed view into individual conversations and AI-driven responses, enabling a deeper understanding of how the virtual agent interacts with users.

Conversations

The Conversation page provides full visibility into end-to-end user interactions. It is designed to review how the virtual agent processes inputs and navigates through the dialogue.

Key Features

• Full Conversation History: View entire user sessions, including both user messages and bot responses, in chronological order. If speech services are enabled, it is also possible to listen to the conversation recordings.

• Matched Intents and Entities: View the recognized intents and extracted entities for each user message, along with clear indicators of fallback messages or errors where the agent failed to understand or respond correctly.

• User Journey Flow: It shows a visual timeline of the conversation, highlighting which flows the user passed through and when, with message timestamps for each transition. It helps identify how users navigate the virtual agent, including any fallback points or repeated flows.

Generative Q&A

The Generative Q&A page provides visibility into the performance of LLM-powered question answering. This feature is especially useful if the virtual agent uses generative models to answer knowledge-based user queries.

Displayed Fields:

• User Question: The original question asked by the user.
• Generated Response: The response provided by the generative model.
• Status: Indicates whether the response was successfully found or not.
• Average Confidence: Shows the average confidence score of the response, which helps evaluate the reliability of the generated answer.