Reports

Dashboard

The Dashboard in Knovvu Virtual Agent offers a quick snapshot and a deeper understanding of how the virtual agent is performing. It provides key metrics and visualizations for understanding user behavior, measuring
effectiveness, and identifying areas for improvement.

The Dashboard consists of two main tabs:

• Business Metrics: Provides overall performance indicators and conversation quality metrics.

• Intents: Focuses on intent detection performance and user behavior patterns.

Filters

Virtual Agent: This will filter results for specific Virtual Agents that are part of the project. If no project has been filtered, the Virtual Agent filter will be inactive as well as ‘Filter’ button down below. If no VA filter is selected, the data will be retrieved for all the Virtual Agents that are part of the filtered project.

Channel: Channel refers to the integrated channel being used for communication in a project. This includes IVR, Facebook, Webchat etc.

Date: The date filter allows data retrieval for a desired time range. The time range can either be chosen from standard options that are provided, “today”, “yesterday”, “last 7 days”, “last 30 days”, or it can be a custom date range between any two dates of choice.

Filters works with ‘and’. Which means the information displayed on the dashboard to come from data that satisfies all the constraints set by the filters.

Business Metrics Tab

The Business Metrics tab provides a comprehensive overview of virtual agent performance with focus on session-level metrics, resolution outcomes, customer sentiment and ratings.

Key Metrics

On the top of the dashboard page, there are four boxes placed side by side, displaying the following information:

• Total Sessions: It displays the total number of user sessions that occurred within the selected filter range. A session represents a continuous interaction between a user and the virtual agent.

• Average Session Duration: It represents the average length of the sessions.

• Average Message per Session: It denotes the average number of messages of customers during one session.

• Self-service rate: It measures the percentage of sessions resolved without human agent intervention.

Resolution Type Metrics

This section provides comprehensive insights into how customer sessions are resolved.

• Resolution Type Trends: This line chart displays how different resolution types evolved over time.

  • Self-service: Sessions where the customer's inquiry was successfully resolved by the virtual agent without human intervention.

  • No Customer Interaction: Sessions where no meaningful customer engagement occurred.

  • Live Agent Handover: Sessions that were transferred to a human agent due to complexity or customer request.

• Resolution Type: A pie chart that shows the distribution of different resolution types.

Session Trends Metrics

This combined visualization uses bar charts to display total session volumes for specific dates, while a line chart overlay shows the average duration trend (measured in seconds). This dual representation helps identify peak usage times and correlate session volume with average duration, revealing patterns in user behavior and system load.

Sentiment Trend Metrics

This section analyzes customer sentiment throughout interactions, helping track emotional responses and overall satisfaction trends over time.

• Sentiment Trends:

Sentiment trends are depicted using a line graph, with the vertical axis representing sentiment values ranging from -1 to 1, and the horizontal axis indicating dates. Placing the cursor on the trend line reveals the total number and percentage of positive, negative, and neutral conversations within the chosen date range. Additionally, the tooltip displays the average sentiment of all conversations, without categorizing them or considering the total number of conversations. It is important to note that the sentiment averages solely reflect the sentiment of user inputs.

• Average Sentiment:

The average sentiment is divided into three segments: negative, neutral and positive. The proportion of each segment represents the average score of session scores falling within that particular segment. The value displayed at the center of the graph represents the average score of all sentiment data within the selected date range.

Customer Trend Metrics

This section tracks customer satisfaction ratings and their trends over time, providing insights into service quality and customer experience.

• Customer Rating Trends: This line chart displays how customer ratings evolved over time. The vertical axis represents rating values, and the horizontal axis shows dates. The trend line reveals patterns in customer satisfaction, helping identify periods of improvement or decline in service quality.

• Average Customer Rating: This circular gauge displays the overall average customer rating across all sessions. The value at the center (e.g., "4.06") represents the average rating within the selected date range. The gauge is divided into five color-coded segments (1 through 5 stars), with each segment's size representing the proportion of ratings in that category, providing a visual breakdown of the rating distribution.

Intents Tab

The Intents tab provides detailed insights into intent detection performance, fallback occurrences, and intent usage patterns.

Fallback Messages

This graph lists the utterances that resulted in a fallback, that is unmatched intent detection. Fallback messages are displayed when an intent fails to be detected for an utterance. There are two scenarios where a fallback message is triggered:

1. If the VA cannot match any intent over a confidence threshold.
2. If the intent with the highest value match on the confidence threshold is the fallback intent.
   The width of the corresponding bar of the utterance is proportional to the frequency with which the utterance was used and resulted in a fallback. The Fallback Messages widget also has an option that allows the user to download this list.
   Note that fallback utterances can be added to an intent by clicking on them, allowing users to remove them from the fallback list or alternatively, users can create a new intent by clicking on the fallback utterance.

Intent Metrics

• Intent Overview:

This chart shows the ratios between the intents that were matched and the ones that couldn’t be detected, labeled as “fallback”. This helps in analyzing the performance of the VA’s detection abilities.

• Top Intent:

It lists the intents according to how frequently they have been used by customers. The top 7 intents are displayed here, and if the cursor is hovered on any of them, the number of times the intent was used is shown.

• Intent Trends:

It is a graph that displays the trend of usage over time. The x-axis displays the dates, and the overall graph shows the trend of how frequently the corresponding intent was used over a certain range of time. Trends display in here are the Top Intents.

Insights

The Insights tab in Knovvu Virtual Agent provides a detailed view into individual conversations and AI-driven responses, enabling a deeper understanding of how the virtual agent interacts with users.

Conversations

The Conversation page provides full visibility into end-to-end user interactions. It is designed to review how the virtual agent processes inputs and navigates through the dialogue.

Key Features

• Full Conversation History: View entire user sessions, including both user messages and bot responses, in chronological order. If speech services are enabled, it is also possible to listen to the conversation recordings.

• Matched Intents and Entities: View the recognized intents and extracted entities for each user message, along with clear indicators of fallback messages or errors where the agent failed to understand or respond correctly.

• User Journey Flow: It shows a visual timeline of the conversation, highlighting which flows the user passed through and when, with message timestamps for each transition. It helps identify how users navigate the virtual agent, including any fallback points or repeated flows.

• Customer Sentiment: Displays the overall sentiment detected for the customer during the conversation (e.g., Positive, Neutral, Negative), helping track emotional tone and satisfaction throughout the interaction.

Generative Q&A

The Generative Q&A page provides visibility into the performance of LLM-powered question answering. This feature is especially useful if the virtual agent uses generative models to answer knowledge-based user queries.

Displayed Fields:

• User Question: The original question asked by the user.
• Generated Response: The response provided by the generative model.
• Status: Indicates whether the response was successfully found or not.
• Average Confidence: Shows the average confidence score of the response, which helps evaluate the reliability of the generated answer.