The 5 Azure AI Foundry Mistakes I See Every AI-103 Candidate Make

Update (September 2026): AI-900 and AI-102 retired on June 30, 2026. Their replacements, AI-901 (Azure AI Fundamentals) and AI-103 (Azure AI App and Agent Developer Associate), are the current exams. For the current path, see the AI-103 Study Guide.

I've watched enough AI-103 candidates work through hands-on Azure AI Foundry labs to know what's going to go wrong before it goes wrong. Five mistakes account for the vast majority of the problems, and the interesting thing isn't that they happen. It's that they're the same five mistakes, again and again, across candidates with wildly different backgrounds.

They're not dumb mistakes. They're the natural mistakes of someone treating a prototype as an experiment rather than as a step toward production. Foundry is a platform designed to reward production-thinking from the first line of code. These mistakes are what happens when you don't apply that thinking yet.

If you're preparing for AI-103, this post is worth more to you than another study guide. The exam tests whether you can reason like an Azure AI engineer, which means recognizing these patterns before you make them, or catching yourself when you do. Here's what to look for, why it happens, and exactly how to fix each one.

Mistake 1: Treating Every Foundry Project Like a Standalone Workspace

The most common mistake I see on day one is candidates opening Foundry and building everything inside a single Project (deployments, connections, compute, permissions, storage) without realizing that Foundry's architecture is deliberately split between two scopes. Some things belong at the Hub level. Others belong at the Project level. When you don't understand which is which, you end up with duplicated deployments, wasted quota, permission errors that don't make sense, and a Foundry setup that gets messier every week.

This happens because the Hub/Project distinction is genuinely subtle, and the portal UI doesn't always make it obvious which scope you're operating on. Most tutorials show single-project scenarios, so the pattern never gets taught properly. Candidates learn Foundry the way they learn Visual Studio Code, as a single workspace, when they should be learning it the way they'd learn an Azure landing zone: as a hierarchy with shared services above and per-team workloads below.

Here's the rule to internalize. Hub is where shared, organization-level concerns live: connections to Azure Storage and Azure AI Search, compute resources, networking and private endpoint configuration, cost centers, and identity boundaries. Project is where specific workloads live: agent deployments, playgrounds, evaluation runs, and per-team RBAC. One Hub per organizational unit, typically a business unit or major product line, with multiple Projects under it for different applications or teams. Think of Hub as the "platform team's job" and Project as the "application team's job", even if the same person is wearing both hats.

The fix is architectural, not tactical. Before you build anything else, sketch out your Hub structure explicitly and decide what goes where. Document it. If you're preparing for AI-103, the exam will test this. Questions in the "Plan and manage" domain routinely present resource-organization scenarios and ask which scope a given resource belongs to.

Mistake 2: Starting With API Keys and Never Migrating to Managed Identity

Everyone starts with API keys. They're the fastest way to make the first API call work: copy the endpoint, paste the key, and you're getting responses within minutes. The mistake isn't starting with them. The mistake is never leaving them.

I've watched candidates build a complete AI-103 lab project on API keys, ship the pattern into their real work, and end up with keys sitting in appsettings.json, keys in environment variables committed to Git by accident, keys in CI/CD variables shared across teams, and eventually keys leaked in a code snippet pasted into a Teams channel. Then the emergency rotation. Then the next incident three months later.

Managed Identity solves this cleanly, and Foundry supports it as a first-class pattern. But it requires more upfront setup. You need to assign an identity (system-assigned or user-assigned), grant the right role (for Azure OpenAI, the "Cognitive Services OpenAI User" role at minimum, or the equivalent Foundry roles for other services), and change your SDK code to use DefaultAzureCredential instead of an API key. It feels like overhead when you're just prototyping. It stops feeling like overhead the first time you'd have had to rotate a leaked key.

Do this instead, from day one. Use DefaultAzureCredential in your code even during development. It works locally with az login, works in Azure App Service or AKS with system-assigned or user-assigned managed identity, works in containers with workload identity, and falls back to environment variables when nothing else is available. Grant your identity the specific RBAC role it needs (no more) at the smallest scope that works. Reserve API keys for genuinely narrow scenarios like specific third-party integrations that don't yet support Entra ID.

AI-103 tests this explicitly. Scenario questions in the "Plan and manage" domain will present an application architecture and ask you to choose between key-based and identity-based authentication. Managed Identity is almost always the correct answer for anything production-shaped, and knowing why matters more than knowing the SDK call.

Mistake 3: Ignoring Quotas Until Production Breaks

In development, you will never hit a quota. So it's tempting to defer quota planning entirely. This is the mistake that hurts the most in production, because by the time you hit it, you're already in an incident.

Azure OpenAI Service enforces two consumption limits: tokens per minute (TPM) and requests per minute (RPM). When you deploy a model, you're allocated a share of the region's quota. Under real load, if your application exceeds either limit, you'll start getting HTTP 429 responses, throttling errors. Without a retry-and-backoff pattern, a single 429 becomes a single failed user request. Without a fallback deployment, a throttled region takes down your app.

The candidates who don't plan for this hit the wall the same way. Traffic scales, latency degrades under load, they see 429s in logs, and they scramble to request a quota increase, which takes days of Microsoft approval that they don't have. Then they discover that if they wanted guaranteed capacity via Provisioned Throughput Units (PTUs), that's a different commitment model with different pricing, and they can't just switch to it in an afternoon.

Plan for capacity the way you'd plan for a database. Understand which billing model you're on: consumption for spiky, prototype, or variable workloads; PTU for predictable production where you need latency guarantees and cost predictability at scale. Implement exponential backoff for 429s in every client. This is basic hygiene that gets skipped constantly. Build a fallback pattern with a primary deployment in one region and a secondary in a different region for burst capacity, so a regional throttle doesn't take you down. Monitor quota utilization via Azure Monitor before you hit limits, not after.

Cost and quota planning show up across multiple AI-103 domains. Scenario questions frequently describe a workload with specific throughput or cost constraints and ask you to design the right deployment topology. The candidates who understand TPM, RPM, and PTU as design levers, not as trivia, score better here.

Mistake 4: Building RAG With Bad Chunking and Wondering Why It Hallucinates

Retrieval-Augmented Generation is where the "prototype works, production doesn't" gap is widest. Candidates build a RAG pipeline using the default pattern from every tutorial (fixed 512-token chunks with 50-token overlap, pure vector search, top-k retrieval, and a prompt template) and then they wonder why answers are off-topic, hallucinated, or subtly wrong. The instinct is to blame the model. The problem is almost always retrieval.

Fixed 512-token chunking is what tutorials show because it works for simple, uniform content. Real enterprise data isn't uniform. You have long PDFs with headings, tables mixed with prose, multi-language documents, forms with structured fields, transcripts with speaker turns. Chunking the same way for all of it means splitting sentences mid-thought, fragmenting tables across chunks, and destroying the semantic coherence the model needs to answer accurately. Meanwhile, pure vector search means the exact keyword the user asked about (a product code, a policy number, a technical term) may not appear anywhere in the retrieved context because vector similarity is not keyword match.

Match the chunking to the content. Use semantic chunking for continuous prose so chunks break on natural boundaries. Use structural chunking for documents with headings. A chunk per section is often more useful than a chunk per 512 tokens. Use row-based or table-aware chunking for tabular data. Use hybrid search, not pure vector. Azure AI Search supports vector-plus-keyword hybrid retrieval natively, and it dramatically improves recall for queries that hinge on specific terms. Evaluate retrieval separately from generation on a labeled test set before you blame the LLM. Measure recall at k for your queries. Use Foundry's groundedness evaluation to catch regressions when you change chunking or search strategy.

The AI-103 exam tests RAG heavily in the "Implement generative AI" skill area. Questions rarely ask about API syntax. They ask you to diagnose why a RAG system is producing bad answers or to choose between retrieval strategies for a given scenario. Knowing how chunking and search interact with groundedness is the difference between a passing answer and a guess.

Mistake 5: Shipping Agents Without Evaluation From Day One

The last mistake is the most consequential in production and the most invisible in development. Candidates build a demo agent, test it on five example prompts they picked themselves, get it working, and ship. There's no evaluation dataset. There's no baseline. There's no CI check. When the agent's behavior changes because a prompt was updated, a model version was bumped, a tool was added, no one notices until users complain.

This happens because evaluation feels like overhead in early development. Foundry's evaluation tooling has a learning curve. Manual testing on the happy path feels sufficient when the agent works. So the evaluation step gets deferred to "when we have more time", which is a time that never actually arrives.

The failure mode is silent regression. You update the prompt to fix one issue and quietly break two others. You bump from one model version to another and degrade quality on a specific category of query that no one tested. You add a new tool and the agent starts calling it in situations you didn't anticipate. Without an evaluation dataset that runs on every change, none of this surfaces until a user hits it, and users don't file bug reports for AI systems, they just stop trusting them.

Build the evaluation dataset before you build the agent, even if it's twenty examples. Include the boring happy-path cases, the edge cases you're already worried about, and adversarial examples: jailbreak attempts, ambiguous queries, out-of-scope questions. Use Foundry's built-in evaluators for groundedness, relevance, and coherence, and write custom evaluators for domain-specific requirements. Run evaluation in CI when the prompt or model changes, not only before releases. Track a small set of metrics over time so you can see quality drift before your users can.

Evaluation runs through the "Implement generative AI" and "Plan and manage" domains on AI-103. Scenario questions about improving agent quality, detecting regressions, or measuring performance almost always route through the evaluation framework. Candidates who've built an evaluation loop in practice recognize these questions instantly. Candidates who haven't guess.

The Pattern Underneath All Five

If you look at these mistakes together, they share one root cause: treating a prototype as if it isn't a step toward production. Each mistake is what happens when someone gets something working on their laptop and doesn't stop to think about what changes when the workload becomes real.

The Hub/Project split exists because Microsoft assumes your Foundry setup will grow beyond one team. Managed Identity exists because your app will end up in Azure. Quota planning exists because your traffic will scale. Chunking and hybrid search exist because your data will get messier. Evaluation exists because your agent will change and you'll need to know when it degrades.

The engineers who pass AI-103 comfortably, and who go on to build careers on Azure AI, internalize this early. They treat every prototype as if it will become production, because in the work that matters, it usually does. That's not paranoia. That's what senior looks like.

How to Actually Stop Making These Mistakes

Reading about mistakes doesn't fix them. Making them in a low-stakes environment does. That's the whole reason hands-on labs exist, and it's the strongest argument for spending your AI-103 prep time building rather than reading.

In a real production job, each of these mistakes costs somebody money: the outage cost, the rework cost, the incident-response cost, or the "we can't ship this in front of a regulated customer" cost. In a lab, each of these mistakes costs you an hour and a hard-earned intuition you keep for the rest of your career.

The path forward is straightforward. Pick one of the five mistakes above. Deliberately make it in a lab environment. Watch what breaks. Fix it the right way. Do that five times and you'll walk into the AI-103 exam recognizing scenario questions on sight, and you'll walk into your first Foundry job with production instincts that most people take a year of incidents to develop.

Frequently Asked Questions

Are these mistakes covered on the AI-103 exam? Yes, directly. Every one of these five patterns maps to a specific AI-103 skill area: resource organization and identity in "Plan and manage", RAG and evaluation in "Implement generative AI". The exam uses scenario framing rather than trivia, which means recognizing the pattern behind a question matters more than memorizing API signatures.

Is the Hub/Project split still current in 2026 Foundry? Yes. The Hub/Project architecture remains Foundry's core organizing model. If anything, the introduction of the Microsoft Agent Framework and Foundry Agent Service has reinforced the split. The Hub is where shared identity, connections, and governance live, and the Project is where individual agents and workloads run.

Do I need to use Managed Identity in development? You don't strictly need to, but it's the pattern I recommend anyway. DefaultAzureCredential works locally with az login, so there's no real friction to using it from day one, and the code you write is the code that ships. The alternative is writing key-based code first and rewriting it later, which is where the mistake originates.

How much of my AI-103 prep time should be hands-on versus reading? More than you're currently spending. If you're doing more than 30% of your prep as passive reading, tilt the balance toward building. Foundry rewards production-thinking, and production-thinking is a muscle you develop by shipping things, breaking them, and fixing them.

What's the fastest way to build the evaluation habit? Pick your smallest active project and add a twenty-example evaluation dataset to it this week. Not perfect examples, representative ones, including the ones you're already nervous about. Then set up a Foundry evaluation run against them. Once you have the loop, adding more examples is easy. The barrier is starting.

The Bottom Line

Five mistakes trip up more AI-103 candidates than everything else combined, and they're the same five mistakes I see production engineers make in real Azure work. Fix them early and you'll pass the exam more comfortably and build better systems for years afterward. Ignore them and you'll be relearning the same lessons in production at much higher cost.

The right way to internalize this is to make each of these mistakes in a controlled environment and feel what breaks. Explore the GetHandsOn.ai Azure AI Foundry labs and turn these five mistakes into five hard-earned instincts, the kind that show up on exam day and every day after it.

Related reading: Azure AI Foundry Explained · What Is MCP? · Multi-Agent Orchestration in Foundry · Why Hands-On Labs Matter More Than Practice Tests for AI-103 · AI-103 Study Guide