Conversation Analyzer

Learn how Conversation Analyzer automatically scores your AI Agent’s conversations based on configurable criteria across Topic Discovery, Basic Analysis, and Custom Evaluation dashboards.

Added in 2026.12 (beta)

This feature is in beta. We encourage you to try it out and provide us with feedback.

Conversation Analyzer is a quality evaluation tool in Cognigy Insights that automatically scores your AI Agent’s conversations based on predefined and custom criteria. Instead of reviewing conversation transcripts manually, you define an evaluation topic and a set of metrics, and the analyzer processes your conversations and surfaces results across dashboards.

When to Use it

• Understand what topics your AI Agent handles most frequently and how that changes over time.
• Measure whether your AI Agent is resolving issues, following instructions, and communicating with quality.
• Define custom evaluation criteria specific to your business, compliance requirements, or product domain.
• Replace manual QA spot-checks with consistent, automated scoring at scale.

Key Benefits

• Reduce manual QA effort. Define criteria once and the analyzer scores conversations automatically at scale, replacing time-consuming manual transcript reviews.
• Catch issues earlier. Scheduled runs surface declining sentiment trends, rising escalation risk, or drops in instruction-following before they become visible in support volumes.
• Consistent evaluation. Unlike manual reviews, the analyzer applies the same criteria to every conversation, every run, removing reviewer bias and inconsistency.
• Identify coverage gaps. The Topic Discovery dashboard highlights conversations your current topic doesn’t cover, so you can expand your AI Agent’s scope based on real data.

Prerequisites

Set Up your Model

Conversation Analyzer uses a large language model to evaluate conversations. Before running your first analysis, configure which model to use in your Project settings.

Reasoning models can provide more accurate results, but they typically come with higher costs and longer processing times. For brief analyses, non-reasoning models may be sufficient.

1. In Cognigy.AI, open your Project.
2. Go to Manage > Settings > Generative AI Settings.
3. Under Conversation Analyzer, select a model from the list. Save changes.

If you don’t select a model, the default model is used.

Working with Conversation Analyzer

1. Configure Evaluation Settings

In the left-side menu of the Insights interface, go to Configuration and set up the evaluation topic, schedule, and criteria:

• Topic Discovery — select an evaluation topic that reflects the main subject area of your AI Agent’s conversations.
• Basic Analysis — activate or deactivate criteria sets in Basic Analysis based on what you want to measure.
• Custom Evaluation — define custom evaluation criteria specific to your business needs.

2. Run Analysis

You can run analysis on conversations in the following ways:

On a Schedule

To schedule an analysis, follow these steps:

1. In the left-side menu of the Insights interface, go to Configuration.
2. In the Topic & Scheduler section, go to Analysis Scheduler.
3. By default, the scheduler is off. Select how often you want the analysis to run:
   • Daily — analysis run once a day, starting the following day.
   • Weekly — analysis runs once a week.
4. In the Filter — Only Analyze Transcripts Matching section, configure which conversations to include:
   • Endpoint — limit analysis to a specific Endpoint.
   • Snapshot — limit to a specific Snapshot.
   • Channel — limit to a specific channel.
5. Save changes.

The Analysis Scheduler section shows when the next run is scheduled. For example, Next run: May 15, 2026, 2:00 AM.

Manually

To run an analysis manually, follow these steps:

1. In the left-side menu of the Insights interface, go to Conversations.

2. In the filter bar, select the scope of your analysis. You can filter by time span, Endpoint, and more.

3. Click Run Analysis. Select one of the following options:

   Option	Description
   Reanalyze all	Analyzes all conversations, including those already analyzed. Overwrites previous LLM analysis results.
   Skip already analyzed	Analyzes only conversations that haven’t been analyzed yet. Adds new data to the dashboards without overwriting existing results.

The Analyzing conversations progress bar appears at the top of the Conversations page. 4. When the analysis is complete, click View Analysis in the upper-right corner. The Analysis page opens.

3. Check Results

You can check the results for:

All Conversations

In the left-side menu of the Insights interface, go to Analysis to view topics, performance metrics, and evaluation results.

A Specific Conversation

In the left-side menu of the Insights interface, go to Conversations. Select the entry from the table you want to review, then click > to open Evaluation Details.