# Analysis of Unresolved Conversations

When the **Request Analyzer** service auto-categorizes the bot user **Requests**, it creates the following visualizations and analytics:

1. **Top Flow Gaps** - This visualization identifies the flows that were served in requests, which were subsequently tagged as either "Flow Gap Terminal Node" or "Flow Gap Non-Terminal Node."
2. **Top KB Gaps** - This visualization identifies the knowledge documents that were served in requests, which were subsequently tagged as “KB Gap”.
3. **Top Recommended Actions** - This visualization identifies top recommended/ suggested actions that a user can take to address the top gaps affecting the highest number of requests. Each of these recommended actions will correspond to a possible impact that the user can make upon addressing the gaps.

The Auto-Categorization algorithm identifies and tags all requests where the application (bot) served a correct RAG response / Knowledge Base Article fulfillment, but still received negative feedback from the end user.

**Possible impact = Number of requests with a given gap (auto-tag) / Total number of unresolved requests**

<figure><img src="https://3281977978-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FvBFXjH9S1CAy9f5hzg5Q%2Fuploads%2FBkUvBfCYZY0lnAjxZrl5%2FScreenshot%202024-06-04%20at%204.19.00%E2%80%AFPM.png?alt=media&#x26;token=e5bbff43-1016-4ed6-809f-759d399dd58c" alt=""><figcaption></figcaption></figure>

The goal of Unresolved Analytics is to:

* Get a holistic view of Unresolved Requests:
  * Type of Unresolved Requests
  * Track Top Unresolved Intents
  * Track Not Understood and Not Helpful Rate - Not Understood Requests are requests that have no intent identified by intent identifying engine of Aisera’s bot. However, a correct fulfillment could have been served from a different engine. On the other hand, Not Helpful Requests are requests where a fulfillment was served but the user clicked thumbs down
* Find appropriate Intents, KBs and Flows for Unresolved Requests

### Unresolved Analytics has four major components:

* Not Understood Request Clusters
* Not Helpful Request Intents
* Top Unresolved KPIs
* Auto-Categorization

<figure><img src="https://3281977978-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FvBFXjH9S1CAy9f5hzg5Q%2Fuploads%2FL0FwwrcD0tJKG06oAIMg%2FScreenshot%202024-06-04%20at%204.36.25%E2%80%AFPM.png?alt=media&#x26;token=ffed0395-ee1d-42fc-9e4b-f00054fec693" alt=""><figcaption></figcaption></figure>

## **How does the Aisera platform handle Clusters that are Not Understood?**

**Step 1:** View Top Cluster that are Not Understood&#x20;

Aisera automatically clusters all “Not Understood” conversation requests by looking at existing entities from ontology. Clustering job is executed periodically every day automatically. Each phrase cluster can contain multiple unresolved conversation requests.&#x20;

<figure><img src="https://3281977978-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FvBFXjH9S1CAy9f5hzg5Q%2Fuploads%2FB8rB052W2OX9A5piQQE8%2FScreenshot%202024-06-04%20at%204.37.15%E2%80%AFPM.png?alt=media&#x26;token=144aff79-2fbe-4caf-a7bc-b9a85279b68e" alt=""><figcaption></figcaption></figure>

**Step 2:** Select Cluster

Select a cluster that has a large bucket of unresolved conversations. Each phrase cluster contains information on cluster name, number of Requests and date/time stamp when it was last updated.

<figure><img src="https://3281977978-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FvBFXjH9S1CAy9f5hzg5Q%2Fuploads%2Fd2kCQht5aRY11smnLjYK%2FScreenshot%202024-06-04%20at%204.37.45%E2%80%AFPM.png?alt=media&#x26;token=07aede91-2fec-4c37-b34d-94c51696ace5" alt=""><figcaption></figcaption></figure>

**Step 3:** Take on of the following Actions:

Once a cluster has been identified and scanned, admin needs to click on the action menu. The two possible options are:

1. Add to an existing intent
2. Create a new Intent&#x20;

<figure><img src="https://3281977978-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FvBFXjH9S1CAy9f5hzg5Q%2Fuploads%2F4Y5tGZPqCV04eT6rhYyz%2FScreenshot%202024-06-04%20at%204.39.54%E2%80%AFPM.png?alt=media&#x26;token=6dd316f5-cd88-4f82-a807-fa3774fff46e" alt=""><figcaption></figcaption></figure>

**Step 4:** Select Relevant Intent&#x20;

Select all or most applicable phrase clusters. Click on Action 🡪 Add to existing intent and then select the relevant intent that phrase applies to.

<figure><img src="https://3281977978-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FvBFXjH9S1CAy9f5hzg5Q%2Fuploads%2FhGHFSwH9pJpzMZMrP8Fi%2FScreenshot%202024-06-04%20at%204.40.58%E2%80%AFPM.png?alt=media&#x26;token=7b7df96f-bc02-48c7-a4bd-07f8c0cecaad" alt=""><figcaption></figcaption></figure>

**Step 5:** Preview Intent Modifications

1. Review all phrases to add to existing intent
2. Add/Remove/make modifications to phrases inline
3. Review existing phrases of intent as well to get context and click on
4. This action also removes these phrases from existing cluster in real-time

<figure><img src="https://3281977978-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FvBFXjH9S1CAy9f5hzg5Q%2Fuploads%2F1833MgmkfO3sgXYgbU7z%2FScreenshot%202024-06-04%20at%204.42.56%E2%80%AFPM.png?alt=media&#x26;token=83f33047-3629-4f29-aa8f-ec8cc5e66b8f" alt=""><figcaption></figcaption></figure>

**Step 6:** Intent Review

1. Go to Intent Review Page
2. View the newly annotated intent with phrases
3. View the Review Status of the annotation as “Needs review”, “Approved – Validated” and “Rejected”
4. Admin has option to either approve or reject the proposed changes

**Note:** The Annotator role can propose changes to **Intents** but cannot approve or reject changes.&#x20;

<figure><img src="https://3281977978-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FvBFXjH9S1CAy9f5hzg5Q%2Fuploads%2FENeAhZCXPyOCiXHEtyht%2FScreenshot%202024-06-04%20at%204.51.06%E2%80%AFPM.png?alt=media&#x26;token=1973328c-bf4a-417e-8326-38a7e2bc8b0a" alt=""><figcaption></figcaption></figure>

**Step 7:** Run Validation job

1. Once the intent changes have been approved, proceed to run the “Validate” job
2. This is equivalent to taking all newly added phrases and running it through AI lens

<figure><img src="https://3281977978-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FvBFXjH9S1CAy9f5hzg5Q%2Fuploads%2F6Q3PlKARH1cHsE1MieCJ%2FScreenshot%202024-06-04%20at%204.51.35%E2%80%AFPM.png?alt=media&#x26;token=10196da1-1dcb-4260-acc2-211aa942f825" alt=""><figcaption></figcaption></figure>

**Step 8:** View Validation Success

1. Once validation is successful, you can go back to AI lens and try to do a spot check&#x20;
2. Spot check should include if the newly added phrases are associated with expected intent&#x20;

## **How does the Aisera platform handle Requests that are Not Helpful?**

**Step 1:** View Top Requests that are Not Helpful

1. Aisera automatically clusters all “Not Helpful” conversation requests by looking at existing entities from ontology
2. Clustering job is executed periodically every day automatically
3. Each phrase cluster can contain multiple unresolved conversation requests&#x20;

<figure><img src="https://3281977978-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FvBFXjH9S1CAy9f5hzg5Q%2Fuploads%2FPw8aeaS89YPDeOBJzvDj%2FScreenshot%202024-06-04%20at%204.53.08%E2%80%AFPM.png?alt=media&#x26;token=9aac7501-49ca-47df-8ddf-3896642f029a" alt=""><figcaption></figcaption></figure>

**Step 2:** Select a cluster or multiple requests

1. Identify and Select a cluster that has a large bucket of Not Helpful conversations
2. Each phrase cluster contains information on cluster name, number of Requests and date/time stamp when it was last updated

<figure><img src="https://3281977978-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FvBFXjH9S1CAy9f5hzg5Q%2Fuploads%2Fp4XmPgyUZT95eR1IKZ9x%2FScreenshot%202024-06-04%20at%204.53.57%E2%80%AFPM.png?alt=media&#x26;token=09a20bda-0bb8-4f23-973b-b603d84fb95f" alt=""><figcaption></figcaption></figure>

**Step 3:** Analyze Missing Information in  Fulfillment Attached

* For Workflow >> Optimize the Flow
* For Knowledge >> Enrich the KB&#x20;

<br>