Azure AI and Copilot

Copilot on Azure portal

Azure is the cloud platform provided by Microsoft. The Azure ecosystem consists of Azure’s compute services, storage devices, networking services, database services, security services, AI, machine learning, IoT, developer tools, and many more capabilities. Through the Azure UI portal, you can easily create these services and resources.

Microsoft has come up with its own interactive AI assistant, named Copilot, which you might have seen on your Windows desktop or the Edge browser. Copilot is knowledgeable and friendly. Microsoft has released multiple implementations of Copilot - 

• Microsoft 365 Copilot, which is available for all the products within the suite, and can help you with tasks like writing emails, analyzing excel data and much more. Since it is integrated with 365, it can access your suite and provide context-specific answers based on the data collected from emails, reviews and other conversations.
• Azure Copilot, which focuses on cloud management and development tasks once you are logged into Azure portal and helps users understand a service, steps to do a task, automation and guidance for cloud resources, services and applications.
• Copilot in Microsoft Edge, which provides AI-driven assistance directly within the Edge browser, and can help users with web-related tasks such as summarizing pages, answering questions, and providing contextual information. It uses your current page to provide the most relevant response.

Both are tailor-made for their respective environments, catering to the specific needs of users.
In this article, let us learn more about copilot and its architecture, and then dig on its capabilities on the Azure portal.

What is Microsoft Copilot?

Microsoft Copilot is an interactive AI assistant that helps you in performing various tasks. You can simply ask, “Hey copilot, can you help me write an essay on elephants?” or perform more complex tasks like orchestrating large-scale cloud infrastructure deployments. Copilot can also troubleshoot issues, recommend optimizations, and automate parts of any workflow to ensure a smooth and efficient process.

Copilot on edge browser:

Copilot on Windows taskbar (part of Microsoft 365 suite):

Let’s use the Copilot to do a simple task. Suppose we have an SQL database having tables like customers, orders etc. I want to run a query that gets all the customer orders placed between a certain given date range. Let’s ask Copilot to do that for us:

My next concern is whether this query will take a while, or give fast results. So, I ask Copilot:

Notice the way in which Copilot not only gives the tweaked query but also relevant suggestions on the changes that need to be made in order to satisfy the performance criteria.

As a user, you just give a prompt in natural language, and get the response in the same language, with additional details and links if required. Copilot also suggests the next steps for you to take up.

Capabilities of Copilot

One of the most important capabilities of Copilot, as we discussed earlier, is the ability to process natural language queries. Imagine a computer understanding and responding to a question in a human language! Other than its natural language processing capabilities, Copilot is also equipped to do the following (not limited to):

Task Automation

From simple tasks, like setting reminders for your everyday mundane tasks, Copilot can also write scripts to automate routine tasks so that you need not do them over and again. For example, “write a stored procedure to delete log tables where data is older than  2 years from the current date. Can you also set a trigger so that the script runs every night at exactly 10pm?” Copilot quickly gives you the script - all you have to do is replace the database and table name with the right names and you should be good to go! This saves a lot of time for developers and even if they want to further customize it, the script gives them a skeleton to work on.

Code and database query assistance

If you are a business analyst or a non-technical person, but want to perform some quick checks, or know a few details about your business’s performance, you need not wait for a development team to come up with the details. You can ask the same to Copilot and it will give you the necessary queries/code the data you require. For example, let’s say you want answers to questions like:

• How many users visited my website on a daily basis?
• How many users clicked on CTA?
• How many visitors ordered my product in the past 30 days?

You don't need to search the table structure or database schema to get this information. Copilot will guide you on how and which queries you have to run to get the result. Copilot also ensures that you can access only the data that you are allowed to and not get any information outside of your assigned ‘role’ for the project or application.

Information search, retrieval and presentation

Not just technical, you can ask all sorts of different questions to Copilot. For example, you can get help with new recipes, which car to purchase, where to go for next vacation and much more. It presents the information in a structured format for easy comprehension.

Image generation

Copilot can also create images based on text prompts. For example, you can type a prompt “generate an image depicting a cricket match”, and it quickly generates an image like:

Image generation is still in very initial phases and evolving!

Data analysis

Copilot can help with data preprocessing activities like cleaning, filtering missing values, normalizing data, and exploratory data analysis like identifying outliers, or visualizing data trends using matplotlib and seaborn. Copilot can also help in building and tuning machine learning models for finding trends and patterns.

Architecture of Copilot

Copilot is continuously evolving, based on the feedback it receives, and by learning it receives as and when more data is fed to it. However, the basic architecture is based on the principles of Responsible AI, user-friendly and user-centric design, and interoperability amongst others.

Let us discuss each of the components one by one and how Copilot gets the response for a user query.

Copilot frontend

The front end is the layer that the users interact with. It is the Copilot window consisting of chat, menu items and other sub components.

The Copilot frontend has the Chat UX, which is the user experience design for the chat between the user and Copilot, and focuses on intuitive and engaging interactions.

The next frontend component is the adaptive card. You can create them to present rich, interactive, visually engaging content. You can embed adaptive cards in platforms like Microsoft Teams, Azure Bot Service and other supporting platforms. Adaptive cards can help reduce the front end development and yet provide rich content using JSON schema.

Services layer

The service layer is where all the action happens. It has the following components:

Context Action Services: These services provide contextual information and actions based on user input and history, enhancing the relevance of responses.

Orchestration: Orchestration component ensures that Copilot can handle complex queries efficiently and provide accurate, relevant responses. It is the brain that manages the interaction between user input, business logic and AI components. It uses LLMs (Large Language Models) to generate a deep semantic understanding of the user's query and intent. Orchestration uses the Azure Resource Manager (ARM) to discover the best resources to act upon the user query, and performs the necessary compliance and security checks to ensure there are no risks or deviation of permissions in the user request.

Grounding: Grounding preprocesses the input prompt by accessing relevant data sources, such as Microsoft Graph, which includes emails, chats, and documents the user has permission to access. This is done to get accurate and reliable information, ensuring the credibility of the content.

Prompt and Response Filtering: This stage includes mechanisms to filter and refine user prompts and responses for clarity, context relevance, compliance and appropriateness. Filtering process helps maintain the integrity, quality, and safety of interactions within Copilot.

Integration layer

The integration layer consists of AI infrastructure, which is what powers the capabilities of Copilot. It provides the required computational resources, manages storage, processing and retrieval of big data to train and run AI models, and hosts the AI models that Copilot uses to understand and generate responses in natural language. Additionally, it facilitates the integration of AI services with other components of Copilot, such as the Copilot frontend, orchestration, and plugins.

Plugins are also a part of the integration layer and enhance Copilot’s capabilities by adding new functionalities.

Responsible AI layer

The responsible AI layer is responsible for ethical compliance, security, privacy, transparency, inclusiveness, non-bias and accountability mechanisms to ensure that the AI operations strictly follow ethical standards and guidelines.

What's the Azure portal?

Azure is the cloud offering provided by Microsoft. Azure portal is a UI portal that acts as the control center for managing and deploying all of Microsoft Azure's services and resources.  The portal also provides a unified view of all your resources on the Azure ecosystem and makes managing your cloud resources with ease.

Some of the services provided by Azure are:

• Storage services: SQL and NoSQL (CosmosDB) databases, blob storage, disk storage, data lake storage,
• Compute services: Kubernetes, Virtual machines, container instances, app services, functions and logic apps.
• Networking: Virtual Networks, Load Balancers, and Azure DNS.

Azure also provides a host of other services, including Cognitive service, Azure Machine learning, Stream analytics, Azure Synapse, Azure Devops, Azure digital twins, Key Vault and many more. All in all Azure provides more than 200 products and services that can provide basic to advanced resources for your distributed application.

Your first azure portal

You can login to Azure account to see all the services, tools and products provided by Azure. For this, you need a microsoft live account. You can easily create one if you don't have one. Once you sign in to the portal, you can see a list of services. You can simply search or press the + button to create a new service. Before that, you need to get a subscription.

You can start with a free trial for learning purposes or if you are a student, you can get free software and resources for teaching and learning.

Copilot in Azure portal

Earlier, we talked about the capabilities of Copilot on your windows machine. Now, let’s talk a bit about the capabilities of Copilot on Azure.

You can find the ‘Copilot’ button on the top near the search.

Copilot on Azure provides context-based answers to your (user) queries, rather than generalized answers and

Azure Copilot follows the principles of Responsible AI and gives responses only based on whether the user is authorized to ask certain questions or receive certain responses. For example, if you are trying to ask Copilot to write a query to read xyz data from abc database, Copilot will first check if you have the necessary permissions to read data from abc and xyz.

Example workflow using Azure Copilot

Let’s say you want to create a new virtual machine with Windows Server 2019 in the East US region.

So, the user query will be, “Create a new virtual machine with Windows Server 2019 in the East US region.”

Now, let’s use our previous knowledge of the Copilot architecture to understand how Copilot completes this task.

 Prompt filtering - Azure Copilot checks if the query is appropriate, safe, adheres to all the ethical standards and is contextually correct. For example, it checks the availability of the resource you are requesting, verifies the resource group, and reviews permissions. If it needs any information from the user, it will prompt the user to add the same. In this case, the user might be asked to enter his username, password, or confirm the VM name.

Semantic understanding - The orchestration layer now uses the copy of the latest trained LLM to understand the intent of the user, and instructs the ARM to find resources and tools that can fulfill the task.

Context analysis: Meanwhile, the context action services identify additional information about the context, for example, the subscriptions and roles of the user, his existing configurations, resource groups and any other relevant information to complete the creation of a new virtual machine.

Grounding: Based on the search from MS Graph, which uses RAG (Retrieval Augmented Generation) to enhance the query with additional context and information from Azure services, to render a more accurate and relevant response.

ARM Service discovery and Resource allocation: ARM identifies the resources and configurations to cater to the user request. In this case, the Azure Virtual Machines service.

Task Execution: The ARM then provisions the new Virtual Machine as per the criteria in the user query.

Responsible AI - The orchestration works with Responsible AI to ensure policy enforcement and compliance.

Response generation and filtering: Copilot collates the response, whether the task was a success and additional details about the task. In our case, since the VM was created, it would be the details of the VM. The response is then filtered for correctness and compliance.

Response is then shown to the user as plain text.

Why is Copilot in Azure considered a breakthrough?

Azure Copilot not only simplifies the management of Azure resources, but also unifies knowledge and data across hundreds of Azure services. Azure Copilot’s AI powered assistance uses the latest LLM models to provide intelligent assistance even to non-technical users. You can simply type your tasks or queries in natural language.  Additionally, it helps troubleshoot issues by orchestrating data across Azure services. Copilot  ensures that all actions are performed safely and in compliance with organizational policies within the roles and permissions of a user.

All the above reasons make Azure Copilot an important breakthrough for a more efficient and secure cloud management.

Azure Copilot doesn’t stop there. It is ever-evolving, with some new capabilities planned in the upcoming days:

• Easier troubleshooting by providing AI powered diagnostics and recommendations to quickly resolve issues that would otherwise require deep technical know-how.
• Embedded YAML editing for Kubernetes configurations to provide real-time validations and suggestions within the Copilot interface.
• Managing Azure Kubernetes Service (AKS) clusters using natural language commands, wherein users can describe what they want to do, and Copilot will translate these instructions into the appropriate API calls or CLI commands.

Summary

In this blog, we have learnt the basics of what Copilot is, and the various ways you can use Copilot to solve your queries. We also understood the basic architecture of Copilot, with an example workflow of a simple user task.