What AI-103 Was Really Testing: Six Lessons From My First Week Running Azure AI Foundry in Production

The Monday morning I opened Foundry as the deployment lead rather than the student was the moment I realized how much of my prep had been recognition rather than reasoning. I had passed the certification. I had worked through the Microsoft Learn paths and read the documentation front to back. What I hadn't done, and hadn't known I needed to do, was practice the reasoning under real constraints.

The gap between "studied for AI-103" and "productive in Foundry" isn't a gap in what the exam covers. It's a gap in how you prepared for it. AI-103's scenario framing tests exactly the kind of reasoning production demands. But if your prep was passive, reading, watching, memorizing, you notice the gap in the exam room and the production room both. Active, scenario-based practice closes it in both places.

What follows is field notes: six things that landed differently in my first week than they had in study. Some are technical. Some are organizational. All of them are the kind of small realizations that make AI-103 scenario questions click on the exam and make the first production week much less expensive.

1. The Cost Model Isn't What the Portal Shows

I knew the pricing page. I understood consumption billing, PTUs, and how tokens cost money. What I didn't fully appreciate until real workloads hit was how many other line items appear on the bill and how quickly they add up around the model.

The model calls were often the smaller number. Underneath them sat Azure AI Search (index size and query volume, both of which climbed faster than expected once we indexed real corporate content), Application Insights ingestion where the instinct to log every request and response met the reality of ingestion pricing, storage for embeddings that never got cleaned up, Content Safety calls that scale with every prompt and completion, and compute for the small number of workloads that couldn't run serverless. None of it is hidden. Every line item lives in Cost Management. But none of it is what candidates focus on during exam prep, so it lands as a surprise.

The lesson: turn on Cost Management with workload-level tagging on day one, not day thirty. Set budget alerts before you build, not after the first invoice. And train yourself to look at total cost of the AI system, not just the model line item, because the model is often the third or fourth most expensive component.

2. The First Real Conversation Isn't Technical, It's About Governance

I expected my first hard conversations to be about architecture. They weren't. They were about data. What could go into the system, who could see the outputs, what the retention policy needed to be, what data protection and legal needed to sign off on before we could go beyond a proof of concept.

The conversation felt slower than it should have, and the reason took me a while to see. Governance appears as a topic in AI-103 study material (RBAC, Content Safety, Purview integration), but topic coverage isn't the same as reasoning through the specific conversations you'll have with a real compliance team. Their questions are about specific data categories, specific regulatory frameworks, specific audit trails, specific exception processes. Foundry provides tools for most of it. But knowing which tool answers which question, and having the answers written down in a form a non-engineer can review and sign off on, takes time you didn't budget for.

The lesson: the governance conversation is a design input, not a compliance checkbox at the end. Involve your data protection and security teams in the first architecture review, not the last. If you're studying for AI-103, weight the "Plan and manage" domain more heavily than its topical share suggests, because that's the domain that maps to these conversations, and those conversations gate whether anything you build actually ships.

3. Content Safety Configuration Isn't Set-and-Forget

In prototype work, I'd used Content Safety with default settings and everything worked. In production, defaults are the wrong answer in both directions. Turn them up and legitimate business queries get blocked, a customer support agent flagged as unsafe because it discussed a medication name, or an internal HR assistant refusing to answer a question because it contained a term that sounded harsher than it was in context. Turn them down and you get outputs you don't want anywhere near an enterprise application.

Getting content safety right required treating it like any other configuration surface: versioned, tested, and iterated. We built a test suite of prompts that should pass and prompts that should be blocked, and we ran it every time we changed the safety configuration. That test suite caught more issues in the first month than user reports did.

The lesson: content safety is a product decision as much as a technical one. The right setting depends on your use case, your user population, and your organization's risk tolerance, not on a default. Build the test suite before you tune the settings.

4. Managed Identity Isn't Just a Security Choice, It's an Operational One

Managed Identity is usually taught as a security best practice, which it is. What I didn't fully appreciate until production was how much it also simplifies operations. When you're using API keys, everything about your system's identity is a shared secret. That means every rotation is a coordinated event across teams, every environment (dev, test, prod) has its own set of secrets to manage, every new team member needs the same secrets provisioned, and every incident where a secret might be compromised triggers a scramble.

With Managed Identity, none of that exists. The identity is the identity. Rotation is a concept that mostly doesn't apply. Onboarding a new engineer becomes granting them the right role ("Cognitive Services OpenAI User" for OpenAI access, "Azure AI Developer" or the more scoped Foundry roles for Foundry itself) at the smallest scope that works, rather than sharing a value. Incidents where credentials might have leaked become "revoke the role" instead of "rotate the key and redeploy everything."

The lesson I already knew intellectually but only understood operationally in that first week: you're not just avoiding a security risk when you use Managed Identity, you're avoiding a class of operational work that grows with the size of your team and the number of environments. For AI-103 candidates: this is why the exam prefers Managed Identity in scenario questions. It isn't ideology. It's an operational judgment.

5. The Tool I Underestimated Was Evaluation

I'd thought of evaluation as a nice-to-have, something to set up once things were working. In practice, evaluation was the tool that saved us from a subtle regression during our first real prompt update in that first week. We changed a system prompt to fix one behavior, deployed to a small user group, and only caught the regression because the evaluation run flagged a drop in groundedness scores on a category of queries we hadn't looked at manually, ambiguously phrased questions where the model had started retrieving the wrong context and answering from it confidently.

Without that evaluation set, we'd have shipped a regression to more users, waited for a complaint, spent time debugging, and eventually rolled back, probably a week of quiet damage to a system that was still building trust. With it, the whole loop was about ninety minutes: detect, diagnose, revert, add a new evaluator for the failure category, redeploy.

The lesson: evaluation is the highest-leverage tool in Foundry for teams that ship changes. Not the flashiest, not the one that ends up in demo videos, but the one that turns "we changed the prompt and hope it's better" into "we changed the prompt and here's the measured impact." If you're preparing for AI-103, evaluation is worth more of your study time than its topical weight suggests. If you're already deploying, treat it as core infrastructure, not overhead.

6. The Question That Actually Matters Isn't "How Do We Build This?", It's "What Are We Willing to Own?"

The subtlest realization of that first week didn't come from the platform. It came from the conversation about what we were signing up to be responsible for. Every AI system in an enterprise sits inside a larger question: when it makes a mistake, because it will, who owns the consequences, and what does the recovery process look like?

The AI-103 curriculum covers this partially through Responsible AI content, but in production it becomes a real conversation with real stakeholders. Are we willing to own a system that occasionally answers wrong on a support query? Yes, if the answer is corrected quickly and users can escalate to a human. Are we willing to own a system that autonomously takes an action, like issuing a refund or updating a customer record? That's a different conversation, with different guardrails, different logging requirements, and different human-in-the-loop patterns.

The lesson that sat with me longest: the architectural question and the accountability question are the same question, expressed in different languages. Answering one without the other produces a system that either technically works but isn't shippable, or is shippable but doesn't do enough. The engineers who eventually become senior in this space are the ones who learn to answer both at the same time.

What This All Means for the AI-103 Path

If you're preparing for the exam, none of what I've described contradicts the curriculum. AI-103 covers exactly the right topics: Foundry architecture, Managed Identity, Content Safety, evaluation, RAG, agents, responsible AI, and its scenario framing tests exactly the reasoning I've described here. The question isn't whether the certification prepares you for this. The question is whether your prep style is preparing you for the certification. Passive reading builds recognition of concepts. Deliberate scenario-based practice builds the reasoning that both the exam and the job require.

That's what the first week teaches, and it's what the second week refines. By the third week, if you were paying attention, you start operating differently, not because you know more technology, but because you know what to protect against, what to invest in, and what conversations to have before you write the first line of code.

For candidates who haven't gotten there yet: the closest thing to that first week you can get before production is deliberate lab practice on realistic scenarios. Not the tutorial paths where everything works, but scenarios where cost matters, governance is enforced, evaluation catches regressions, and content safety is tuned for a specific use case. That kind of practice is what AI-103's scenario questions are designed to reward, and it's the whole reason we built GetHandsOn.ai around scenario-based labs rather than more study guides.

Frequently Asked Questions

Does AI-103 prepare you for real Foundry work?

Yes. Its scenario framing is deliberately designed to test the reasoning production requires. What determines whether you're ready for either the exam or the job is how you prepared. Passive reading builds recognition of concepts. Deliberate scenario-based practice builds the reasoning that AI-103 tests and production demands. If your prep was mostly passive, you'll feel the gap in both places.

What surprised you most in your first week?

The share of time spent in non-technical conversations: governance, cost model design, evaluation setup, and what the team was willing to own if the system made a mistake. Almost none of that was writing code. All of it was necessary before code was worth writing.

Which Foundry feature is underrated?

Evaluation. It's the highest-leverage tool for any team that ships changes to an AI system, and it's the one that's most commonly deferred as "we'll set that up later." Later doesn't arrive on its own.

Is production Foundry work harder than the AI-103 exam?

It's different, not harder. The exam tests whether you understand the platform. Production tests whether you can apply that understanding under organizational constraints. Passing the exam doesn't guarantee productive work, but productive work is much harder without the foundation the exam provides.

How do I bridge the gap between passing AI-103 and being productive?

Two things help. First, deliberate lab practice on realistic scenarios: not tutorials where everything works, but scenarios where cost matters and evaluation catches regressions. Second, if you can, get involved in a small production deployment early, even as a supporting engineer, because the organizational lessons don't transfer through documentation.

The Bottom Line

AI-103 tests the same scenario-reasoning that production requires. Whether you walk into either ready comes down to one thing: how much scenario-based practice you actually did during prep. The engineers who struggle in their first week aren't the ones who lacked certification. They're the ones who prepared for it passively. The engineers who thrive are the ones whose prep looked a lot like the first week itself.

The closest thing to a shortcut is deliberate practice in scenarios that look like real work, not sanitized tutorials, but the messier ones where cost, governance, evaluation, and content safety all interact. Explore the GetHandsOn.ai Azure AI Foundry labs and shorten your own first week by building the scenario-reasoning AI-103 rewards and production demands.