Request Analyzer - Conversation Status

The Aisera Gen AI platform includes a service that analyzes all conversations, including unresolved conversations, and applies auto-tags to all Requests so the models can sort them as related to Intents, Workflows, Knowledge Documents, or Internal Server/ Flow Execution Errors categories. This conversation service is known as the Request Analyzer.

The resulting Conversation Status can be:

• Resolved

• Unresolved

• Casual

• Assisted

• Abandoned

• Unhandled

Automatically and accurately analyzing all conversations especially unresolved conversations and applying tags to all conservation requests helps better with representation and analysis of conversations when reviewing a tenant on a daily/ weekly cadence.

The Request Analyzer supports conversations with or without Fulfillments and also provides coverage for Requests from non-intent based fulfillment sources like RAG and Public Knowledge Base Articles.

The list of the conversation tags and definitions are as follows:

Tag Name	Tag Description
Correct Answer	Request was correctly answered by the Fulfillment. What constitutes “correctly answered”? Positive Feedback from User OR LLMs confirming that KB is relevant to the request OR Action Flow Finished without Negative Feedback
Incorrect Answer	The KB served from Neural/Search/Neural+ was Irrelevant (As decided by LLM) OR Received Negative Feedback on a KB (from Neural/Search/Neural+) (even if LLM states it’s correct answer) OR Incorrect Intent (by ICM) and Irrelevant KB (As decided by LLM) OR Incorrect Intent (by ICM) and Abandoned Flow OR Abandoned Flow from Any Intentless (RAG) Fulfillment Source
Incorrect Answer- Update Annotation	This tag satisfies the conditions for Incorrect Answer. Additionally, Correct Intent exists but was not identified by ICM.
Incorrect Answer- Update Ontology	This tag satisfies the conditions for Incorrect Answer. Additionally, the request contains some entities (as determined by LLMs) which were not identified by KGNER.
Incorrect Answer- Update Annotation and Ontology	This tag satisfies the conditions for Incorrect Answer. Additionally, Correct Intent exists but was not identified by ICM AND The request contains some entities (as determined by LLMs) which were not identified by KGNER.
KB Gap	The KB served (KB can be from the Intent or the Fallback) was Irrelevant (As decided by LLM) but the Intent was correct OR Received Negative Feedback for a KB (KB can be from the Intent or the Fallback) but the Intent was correct
Flow Gap - Terminal Node	Negative Feedback at the end of the flow OR If the KB served at the end of the KB Flow (from ICM or Flow Search) is irrelevant (As decided by LLM)
Flow Gap - Non- Terminal Node	Abandonment of the flow (But intent is correct, as decided by LLM).
Flow Execution Error	Flow Execution Failure. - All errors that are ERR-007 Examples: ERR-007 : Flow_Execution_Failure : Fail to execute FlowNode:649512065 ERR-007 : Flow_Execution_Failure : java.lang.NullPointerException
Something Else	Internal Error Message from the Conversation Server Example: ERR-002, ERR-006 Any error code other than ERR-007
Not Understood	No Fulfillment was Served

The Request Analyzer performs the auto-categorization (application of auto-tags) once or twice per week depending on the volume of requests for a given tenant instance. Each tenant has its own schedule.

These tenant jobs are staggered automatically and run on the Aisera Gen AI server every few days depending on the average time it takes to run the job. For example, tenants with a high volume of requests will have the job auto-triggered once a week, lower volumes will run every few days.