Azure AI Foundry Tutorials - AI Agents Hello World Example

What are AI Agents?

We can say that the AI movement started with release of ChatGPT. After that there were many LLM's released. Training and deploying these LLM's is a costly affair. It costs billions of dollars to keep them running and give the responses. Initially we could only chat with these models which is quite useful but does not justify the Return on Investment(ROI).

This led to advent of compound AI systems. Compound AI systems are systems that handle AI tasks by combining multiple interacting components, including multiple calls to models, retrievers, or external tools, rather than relying on a single AI model. Example of Compound AI will be combining chatgpt with websearch for better results. Compound AI becomes AI agents when we give the LLM to independently take actions, make decisions, and work toward goals over multiple steps.

Example of AI agent will be a travel planning compound AI system. Here we just tell the LLM the destination, dates, preferences and some other details. LLM on its own makes use of multiple components like Flight Booking API's, Hotel Booking API's, past experiences of other travellers and so on to either book or present you with your travel plan. So AI Agents autonomously create plans and execute tasks based on their understanding of the problem. They make use of the tools but it is up to the agent to decide which tool to use and when. They have more autonomy and decision making capabilities to function independently.

Azure AI Agents - Table of Contents
Azure AI Foundry Hello World Example
Azure AI Foundry - Azure AI Agent Hello World Example
Azure AI Foundry - Foundry vs Hub Based Projects
Azure AI Foundry - Build Agent with Azure AI Foundry SDK
Azure AI Foundry - Bing Web Search Agent

Azure AI Agents -

AI Agents can be defined as 2 types -

• Declarative Azure AI Agent - An agent where you define the goal, instructions, and allowed tools, and the LLM autonomously decides the reasoning steps and tool usage to produce results.
  LLM is in charge of thinking + deciding steps. For example "Travel Recommendation Assistant" (No booking). It helps users plan a trip and explore options - no real-world action.
  User says - "Plan a 5-day Goa trip in December under 50,000 for a family"
• Custom Azure AI Agent - An agent where your application code controls the workflow and execution order, and the LLM is used as a bounded reasoning component for understanding, ranking, or generating responses.
  Your application is in charge; LLM is only a helper. For example "Corporate Travel Booking System" (REAL booking). It helps book official company travel safely and compliantly.
  Employee says - "Book me a flight to Goa next Monday under company policy"

Implementation

In previous tutorial we had seen what is azure ai foundry and implemented azure hub and project. We had also deployed an azure ai model. We will be making of the same project to create an azure ai agent. In this tutorial we will be implementing a very simple AI agent which just answers questions similar to LLM deployment. Only difference with LLM deployment will be this AI agent has long term memory. Later we will be modifying this ai agent to use bing websearcher, function, RAG, MCP etc.
Also in this tutorial we will be creating and interacting with the agent using azure portal. In the next tutorial we will make use of the azure ai foundry sdk code.

Also in previous tutorial we had deployed o3-mini. For AI agents we need atleast gpt-4o model. So we will first deploy gpt-4o model and later use this for creating AI Agents.

Go to the AI Agents section. Here select the model we are going to use to create the agent. The model we will be using is gpt-4o model.



Click next and a new agent gets created.

Open this AI agent in playground.

In the playground we can see that a new thread has been started. A thread system in azure ai agent services is used to maintain the conversation history and associated state of a user.
So all the coversations between the user and agent get stored in this thread. For a different user, another thread will be created which will store this data.
When the user asks anything to the agent, the user message and the response gets stored in the thread. So we have a conversation history. Also the agent can make use of this conversation history to better answer user questions as it has the previous context. Previously implementing this thread management for azure foundry was a tedious task but now it is provided out of the box.
In the next tutorial we will look at how to connect the azure ai agent using the azure ai foundry sdk.

Difference between deploying Azure Model and Azure AI Agent.

Feature	Raw Model Deployment	Azure Foundry Agent
Core Purpose	Just inference - We just ask some question and get the response. Some action is to be taken programatically in the calling code and not be llm.	Full AI workflow orchestration - The AI agent can perform tasks or take actions on its own.
Memory	You implement it - When we connect to the llm, the llm is stateless. In each request we need to send the previous responses like below "messages": [ {"role": "system", "content": "..."}, {"role": "user", "content": "Hi"}, {"role": "assistant", "content": "Hello!"}, {"role": "user", "content": "What did I ask before?"} ]	Built-in long-term memory - We already have thread to maintain conversation history
Tool Calling	Manual - If any API, database or function is to be called, llm cannot do it on its own, but must be done by the backend/calling code.	Automatic, built-in - Based on llm understanding and instructions, llm can call on its own any API, database or function.
Multi-step Plans	No	Yes (planner included)
RAG / knowledge grounding	You code it	Native capability
Safety & governance	Manual	Automatic
Ease of integration	Medium	Very easy
Flexibility & control	Maximum	High but less low-level
Use Cases	Simple prompts, low-level control	Agents, copilots, end-to-end apps