Part 2: system diagram and thoughts
This isn't the optimal or most advanced design yet, but it's functioning effectively as a proof of concept. I've already revamped much of the logic to enhance the AI component, making it more human-like and less dependent on deterministic decisions.
Below is a highly simplified diagram and only the account branch is complete, the other topics are similar and the flow depends on the requirements:
Ai native understanding of natural language and detail extraction:
The plan is to achieve:
- Deliver generative responses to users based on documentation and instructions integrated into the knowledge base.
- Capability to verify user accounts for basic troubleshooting (e.g., account status, expiration, licensing).
- If an account is identified as a shared mailbox, direct the user to follow the process for managing it.
- To perform basic troubleshooting for a device, checking Entra join type, Intune enrollment, and similar configurations.
- A feature to set reminders for email notifications when a reported issue is resolved, reducing the need to repeatedly check with the AI or ICT staff.
- Integration with the Freshservice ticketing system (create tickets, view your tickets).
I designed it in Copilot Studio with specific topics and generative responses drawn from the knowledge base. For account and device checks, I use Azure Functions, triggered via Power Automate flows, as Copilot Studio has limited options for conditional logic and string manipulation.
For Freshservice integration, I utilize Power Automate to invoke the API.
Logic flow (example for the account troubleshooting branch):
- The Copilot can be deployed in SharePoint or Teams, or it can function as a standalone tool within Copilot itself or on a website.
- The user communicates with the agent by utilizing one of the entry points mentioned above.
- The agent will determine how to handle the request based on the topic triggers and initial instructions, deciding whether to provide generative responses from the knowledge base or to activate one of the topics according to their descriptions.
- Should the agent opt to activate one of the topics, it will adhere to the predefined flow illustrated in the topic diagram.
- Optionally, a "create search query" node may be incorporated to reformulate the user's message for improved alignment with the AI's expectations. For instance, the user message "Alex is unable to access his account" was reformulated as "Alex is unable to access his account. Can you help troubleshoot this issue?"
- An AI prompt node is utilized to extract information such as email, requester, issue type, and resource, and then store them in variables.
- A "create generative answers" node utilizes the "create search query" query to produce a response, which is then stored in a variable for subsequent confidence score assessment to determine if it should be presented to the user.
- Subsequently, another AI node is implemented for confidence score, receiving the AI response from the preceding step along with the user's query, and is prompted to rate, on a scale from 0 to 10, the extent to which the AI reply aligns with the user's request.
- Following this, a conditional branch is implemented. In my configuration, I employed a condition of less than 5 to reroute the flow to account troubleshooting; otherwise, the AI response from the preceding generative answer node is delivered to the user as a message.
- When the confidence score is below 5, we proceed to assess the requester type. If the request pertains to the current user, we utilize the email address of the logged-in user. If it is for another individual, we rely on the extractor from the preceding step; in cases where no email is provided, we prompt for the other person's email.
- Using the email address, we utilize an HTTP request node to initiate a Power Automate flow, which subsequently invokes an Azure Function to retrieve the required data for evaluating the account's health.
- We deliver the results to the user, inquire about raising a ticket, ask whether to set a reminder, and check if the agent can offer any additional assistance.
- Should the user request a ticket and/or a reminder, their response will activate the corresponding dedicated topics.
At this point, the agent is prepared to deliver another generative response or activate any other topic.
Diagrams made with draw.io and mermaidchart.com.
