Foundry Models in Microsoft Foundry
AI-103: Developing AI Apps and Agents on Azure
Microsoft Foundry provides several ways to work with AI models, ranging from ready-to-use AI capabilities to highly specialized custom models.
Choosing the right approach depends on your use case, the level of customization you need, the quality of your data, and the amount of control required.
This article explains the main model approaches available in Microsoft Foundry and how to choose between them.
What Are Foundry Models?
Foundry Models provide the model ecosystem within Microsoft Foundry. They allow you to select models and AI capabilities that can be used to build applications and AI agents.
There are four important approaches to understand:
- Pre-built tools
- Foundation models
- Retrieval-Augmented Generation (RAG)
- Fine-tuned or custom models
These approaches can be viewed as a progression from simple, well-defined AI tasks toward increasingly specialized solutions.
1. Pre-built Tools
The simplest option is to use Microsoft's pre-built AI tools.
These capabilities were previously associated with Azure AI services and are now consolidated within the Microsoft Foundry ecosystem.
Examples include:
- Vision
- Language
- Search
- Speech
- Translation
- Document Intelligence
These tools are designed for well-defined and common AI tasks.
Example
Suppose your application needs to translate text from English to French. Instead of building and training your own model, you can use Microsoft's translation capability.
Advantages
Pre-built tools provide:
- Fast time to value
- Built-in safety capabilities
- Predictable performance
- Relatively predictable costs
- Minimal model development effort
Consideration
The main limitation is flexibility. Because these tools are designed for specific tasks, they cannot be customized as extensively as foundation or custom models.
When should you use them?
Use pre-built tools when:
For example: "Translate this text from English to German."
2. Foundation Models
The next level is foundation models. Foundation models are more appropriate when you need generative AI capabilities such as:
- Creating content
- Summarizing documents
- Answering questions
- Reasoning
- Generating code
- Creating multilingual responses
Unlike a specialized translation or vision API, foundation models can handle a much wider variety of tasks.
Example
Instead of asking a specialized service to translate a sentence, you could ask a foundation model:
The model can reason about the information and generate a response.
Advantages
Foundation models provide:
- Powerful generative capabilities
- Greater flexibility
- Support for reasoning
- Support for creative tasks
- A broad range of application scenarios
Considerations
They also introduce additional considerations:
- Higher cost than many specialized tools
- Token-based consumption
- Need for monitoring
- Need for appropriate safety controls
For enterprise applications, simply selecting a powerful model isn't enough. You should also consider how the model is monitored and governed.
3. Retrieval-Augmented Generation (RAG)
The third approach is Retrieval-Augmented Generation, commonly abbreviated as RAG.
RAG is useful when you want an AI model to generate responses based on your organization's data.
Instead of relying solely on the model's existing knowledge, relevant information is retrieved from an external data source and provided to the model.
Example
Imagine a company has documentation stored in SharePoint. A user asks:
Rather than expecting the model to know the company's internal policy, the application can retrieve the relevant information from the organization's data and use it to generate the response.
Why use RAG?
The primary goal is to:
- Improve accuracy
- Ground responses in organizational data
- Reduce hallucinations
- Provide responses based on current or private information
Important consideration
RAG is only as good as the data behind it. You need well-curated and reliable data sources.
If the underlying documents are outdated, incomplete, or incorrect, RAG cannot magically make them accurate.
4. Fine-Tuned and Custom Models
At the most specialized end are fine-tuned or custom models. These approaches make sense when you have highly specialized requirements, such as:
- Domain-specific terminology
- Strict output formatting
- Highly specialized workflows
- Large volumes of repetitive tasks
- Very precise behavior requirements
Example: Automotive Industry
Imagine a company wants to build an AI assistant specifically for automobile mechanics. The system might need to understand specialized terminology relating to:
- Engines
- Vehicle components
- Diagnostic codes
- Mechanical failures
- Repair procedures
A specialized model may provide better results for this particular domain than a general-purpose model.
Advantages
Fine-tuned or custom models can provide:
- Greater precision for a specific task
- Specialized domain knowledge
- More predictable behavior for particular workflows
- Potential cost efficiencies at scale
Considerations
They also require considerably more effort. You need:
- Curated training data
- Appropriate governance
- Monitoring
- Drift management
- Specialized expertise
You cannot simply tell a custom model to "look at the entire internet" and expect it to become a reliable domain expert.
Comparing the Four Approaches
| Approach | When to Use | Strengths | Key Consideration |
|---|---|---|---|
| Pre-built tools | Well-defined, common tasks | Fast, predictable, built-in capabilities | Limited flexibility |
| Foundation models | Generative AI and reasoning | Powerful and flexible | Higher cost and monitoring requirements |
| RAG | Responses grounded in organizational data | Better accuracy, fewer hallucinations | Requires good source data |
| Fine-tuned / custom models | Highly specialized domains | Precise and specialized | Requires curated data and governance |
Choosing the Right Approach
There isn't one model approach that is always the best. A practical way to approach an AI project is to start as simple as possible.
Stage 1 - Start with a pre-built tool
If your requirement is simply "Translate customer messages", a specialized translation capability may be sufficient.
Stage 2 - Move to a foundation model
If your requirements become more complex - "Read the customer's message, summarize the issue, determine its sentiment, and suggest a response" - a foundation model may be more appropriate.
Stage 3 - Add RAG
If the response also needs to reference company policies - "Generate a response based on our internal refund and warranty policies" - you can introduce RAG to ground the model's response in organizational data.
Stage 4 - Fine-tune or customize
If the application requires highly specialized behavior at significant scale, you may eventually consider fine-tuning or a custom model.
The important point is that you don't necessarily need to jump directly to the most complex solution.
Chatting with a Foundry Model Using Python
Once a model has been deployed in Microsoft Foundry, you can interact with it programmatically. The Python workflow follows a relatively simple pattern.
1. Install the required packages
Install the required Python SDK packages for interacting with Microsoft Foundry.
2. Load the configuration
Configuration such as the Foundry project endpoint and model deployment name can be stored in environment variables. This keeps sensitive configuration separate from the application code.
3. Connect to the Foundry project
The application creates the appropriate client and connects to the Microsoft Foundry project.
4. Open a chat client
A chat client can then be used to send requests to the deployed model.
5. Send a prompt
For example, a system instruction such as:
The application can then send a question such as:
The model generates a response that can be printed by the Python application. This workflow can be demonstrated using a deployed GPT-4.1 mini model.
Token Consumption
When interacting with a model in Foundry, token consumption is an important consideration.
Different prompts can consume significantly different numbers of tokens. A simple question may consume relatively few tokens, while a more detailed prompt and longer response can consume substantially more.
For example, asking "Hi. How are you?" is considerably simpler than asking the model to explain Microsoft's responsible AI principles in language suitable for a five-year-old.
Therefore:
This is particularly important when deploying AI applications at scale.
Key Takeaways
- Pre-built tools are best for well-defined and common AI tasks.
- Foundation models are suitable for generative AI, creativity, and reasoning.
- RAG grounds model responses in organizational data and can reduce hallucinations.
- Fine-tuning and custom models are useful for highly specialized requirements.
- Start with the simplest approach that satisfies the business requirement.
- RAG depends heavily on the quality of the underlying data.
- Fine-tuning and custom models require curated data, governance, and monitoring.
- Model usage is also affected by token consumption, which becomes important for cost management.
- Microsoft Foundry allows developers to interact with deployed models programmatically using Python.
Exam Tip: Remember the progression: Pre-built tools → Foundation models → RAG → Fine-tuned/Custom models. The further you move toward custom models, the greater the specialization and control - but also the greater the complexity and operational requirements.
AI-103 - Table of Contents
Develop AI apps and agents on Azure with study guides covering every exam domain.