Cloud Services PaaS Clouds

Microsoft adds a continuous-improvement loop to Foundry agents

Microsoft is adding a continuous-improvement workflow to Microsoft Foundry, its platform for building, operating and governing AI agents. Announced at Microsoft Ignite on November 17, the approach uses agent activity, expert feedback and automated evaluations to identify problems, test changes and help teams decide what to deploy.

The workflow is designed to connect agent development with operation and refinement. Developers can build agents with GitHub tools, then run and supervise them in Foundry on Azure. Microsoft says the platform supports more than 11,000 models from providers including OpenAI, Anthropic, Meta and Mistral, as well as open-source models and Microsoft AI models. Its model router can direct simpler requests to smaller models and more complex requests to larger ones; Microsoft says internal testing found potential savings of up to 65% compared with routing every request to a frontier model.

A four-stage improvement cycle

Foundry’s process repeats four stages: establish what good performance means, identify problems, optimize the agent and validate changes. The starting point is an evaluation framework, where teams define criteria for judging agent responses. Microsoft says rubric evaluators can apply business-specific criteria to quality and, in preview, safety.

Foundry records traces of agent runs, including what the agent was asked, the steps it took and its response. In preview, experts can annotate those traces to identify successful behavior, mistakes and better alternatives. The annotations can be turned into evaluation datasets, and Foundry can generate rubrics for individual test cases. Teams can also assemble a “golden dataset” of test cases to benchmark future versions.

For identifying recurring issues, Foundry Insights is generally available. Microsoft says it groups problems found in traces, presents supporting evidence and points to likely causes. The company also says teams can use this visibility with agents built using LangGraph, LangChain and CrewAI; that support is generally available.

Testing proposed changes before deployment

Agent Optimizer uses evaluation results to propose changes to an agent’s instructions, skills or model choice. Microsoft says candidate changes are tested against the team’s evaluations, and production behavior does not change unless the team chooses to apply a recommendation. Existing prompt optimization and model routing form part of the offering, while several additional capabilities are in preview.

One preview feature sends an issue identified in Insights to Agent Optimizer for possible fixes. Another, skill optimization, uses past agent work and expert feedback to suggest new or improved skills. Skills are versioned, allowing teams to reuse or roll back a skill. Cost optimization is also in preview and seeks less expensive ways to run an agent, such as selecting a smaller model while maintaining quality.

Microsoft is also presenting two training options. Managed Training, formerly called managed fine-tuning, lets customers train models on their data. Programmed Training, in preview, allows engineers to write training recipes in Python while Foundry runs them on serverless GPUs. Microsoft cites Docusign’s use of fine-tuned models to reduce AI processing costs by 90%, and says the company is now fine-tuning Microsoft AI models to read agreements. These are customer results reported by Microsoft.

Governance and customer examples

Microsoft says Foundry combines agent identity through Entra Agent ID with guardrails that limit access and can require human involvement for sensitive actions. Agent controls in Foundry Safety are intended to check agent behavior against policies during runtime. The company says these controls apply to agents built in Foundry, with support for agents built elsewhere planned for the future.

Hosted agents in Foundry Agent Service are generally available with support for more concurrent sessions; faster startup is in preview. The Microsoft Copilot harness, which manages the agent framework and includes reusable skills and sub-agents, is also in preview. Voice agents are generally available. Microsoft says Fashable is using voice agents and the Copilot harness for conversational design and sourcing workflows.

Microsoft also shared an Alaska Airlines evaluation in which trained personnel remained responsible for review and final decisions on aircraft maintenance report classification. The company says its test of MAI-Code-2-Flash rose from 81.2% accuracy out of the box to 91.1% after tuning. Microsoft reported that the tuned model outperformed GPT-6 Astra and GPT-6.1 Sol in that evaluation at a fraction of the cost.

For validation, Foundry can compare agent versions using statistical tests, run continuous evaluations on sampled live traffic and schedule tests against golden datasets. Microsoft says monitoring dashboards bring quality, safety, latency and token usage together, with alerts for declining scores. Teams can also add checks to release workflows in GitHub Actions or Azure DevOps.

NTT DATA is applying the improvement cycle to a program-management agent, using observability, evaluations and optimization to identify issues. Microsoft says the company is validating the approach before scaling the agent across more than 1,000 internal projects. The broader premise is that teams can use operational evidence to refine agents over time, while retaining control over which tested changes reach production.