Quickstart: Optimize a hosted agent

• azd CLI (Azure Developer CLI) version 1.37.0 or later.

• Azure CLI for authentication.

• The microsoft.foundry extension for azd. It includes the azd ai agent commands, such as azd ai agent optimize, but not the azd ai eval commands:

  azd ext install microsoft.foundry
  

  If already installed, upgrade:

  azd ext upgrade microsoft.foundry
  

• The azure.ai.evaluations extension for azd, which provides the azd ai eval commands. It requires azd 1.33.0 or later:

  azd ext install azure.ai.evaluations
  

  If already installed, upgrade:

  azd ext upgrade azure.ai.evaluations
  

• Azure CLI for authentication.

• Python 3.10 or later.

• The Python packages used in this path:

  pip install "azure-ai-projects>=2.5.0" azure-ai-agentserver-optimization python-dotenv
  

• An existing Foundry project that already contains the hosted agent, registered dataset, and evaluator you want to use for optimization.

• Azure CLI for authentication.

• .NET 10 SDK or later.

• The .NET packages used in this path.

  dotnet add package Azure.AI.Projects --prerelease
  dotnet add package Azure.Identity
  

• An existing Foundry project that already contains the hosted agent, registered dataset, and evaluator you want to use for optimization.

• Visual Studio Code.

• Microsoft Foundry Toolkit for Visual Studio Code version 1.6.4 or later, signed in to Azure. Foundry Toolkit installs and updates the Microsoft Foundry Skill used by this workflow.

• GitHub Copilot in Visual Studio Code with access to agent mode. Foundry Toolkit sends the Agent Optimizer request to GitHub Copilot after you select the agent workspace.

• Azure CLI and Azure Developer CLI (AZD) version 1.37.0 or later, installed and authenticated:

  az login
  azd auth login
  

• The azure.ai.evaluations extension for AZD, if you want to generate evaluation assets with azd ai eval generate. It requires AZD 1.33.0 or later:

  azd ext install azure.ai.evaluations
  

• A coding agent host with the Microsoft Foundry Skill installed.

• Azure CLI and Azure Developer CLI (AZD) version 1.37.0 or later, installed and authenticated:

  az login
  azd auth login
  

• The microsoft.foundry extension for AZD. It doesn't include the azd ai eval commands. Install it before you start the workflow:

  azd ext install microsoft.foundry
  

  If it's already installed, upgrade it.

  azd ext upgrade microsoft.foundry
  

• The azure.ai.evaluations extension for AZD, which provides the azd ai eval commands that the workflow uses. It requires AZD 1.33.0 or later:

  azd ext install azure.ai.evaluations
  

  If it's already installed, upgrade it.

  azd ext upgrade azure.ai.evaluations
  

Step 1: Create the project

Initialize a new project from the optimization sample template:

mkdir my-agent && cd my-agent
azd ai agent init -t https://github.com/microsoft-foundry/foundry-samples/blob/main/samples/python/hosted-agents/bring-your-own/responses/optimization-customer-support/azure.yaml .

This template imports the Optimization Customer Support sample, an optimization-ready Python hosted agent that uses the bring-your-own approach and the Responses protocol. It represents a consumer electronics support agent that handles order inquiries, returns, warranty claims, troubleshooting, complaints, recommendations, and escalation. The deliberately minimal baseline instruction makes the improvements from instruction optimization and skill discovery easy to compare.

The sample calls load_config() to load baseline or candidate configuration and includes .agent_configs/baseline/, full and quick evaluation datasets, container configuration, and the Foundry deployment manifest. The interactive flow imports these files and prompts for your Azure subscription, region, and model deployment settings.

Step 2: Provision and deploy

Authenticate and provision the Azure resources:

az login
azd auth login
azd provision

Provisioning takes approximately two minutes and creates a Foundry account, project, Azure Container Registry, and model deployments.

Deploy the agent:

azd deploy

Test the deployment:

azd ai agent invoke "What is 2+2?"

Step 3: Generate evaluation assets

The optimizer scores candidates by using a dataset and evaluators that you provide. To create a measure that reflects how you judge your own agent, generate a dataset and a rubric evaluator by using the azd ai eval commands. Use the name of your deployed agent as the target. It's the name of the agent service in azure.yaml.

In the following commands, replace <generation-deployment>, <judge-deployment>, <eval-deployment>, and <optimization-deployment> with names of model deployments in your Foundry project. Set each role explicitly.

azd ai eval generate \
  --target <agent-name> \
  --from prompt \
  --generation-model <generation-deployment> \
  --dataset-name support-tests \
  --evaluator-name support-rubric \
  --max-samples 15 \
  --agent-instruction "A customer support agent for a consumer electronics company. It handles order inquiries, returns and refunds, warranty claims, device troubleshooting, complaints, product recommendations, and escalation to a human agent."

The command generates the dataset and the evaluator in your Foundry project, downloads them to evals/datasets/support-tests.jsonl and evals/evaluators/support-rubric.json, and registers both in evals/azure.eval.yaml.

Open the generated files and review them before you continue. If generate prints an input-quality warning, review the generated files with extra care. Edit rows or criteria that don't match how you want the agent to behave.

Step 4: Measure the baseline

Set up an evaluation that scores the deployed agent, so that you can compare the optimized agent against the same measure later. The generate command prints the matching init command. The optimizer reuses this evaluation in Step 5. Add a judge model and a name:

azd ai eval init \
  --target <agent-name> \
  --source dataset \
  --dataset support-tests \
  --evaluation-level turn \
  --evaluator builtin.task_completion \
  --evaluator support-rubric \
  --path ./evals \
  --judge-model <judge-deployment> \
  --name optimize-check

azd ai eval create optimize-check
azd ai eval run start --eval optimize-check --name baseline --wait

The init command adds an evaluation service to azure.yaml. The create command registers the evaluation in your Foundry project. The run prints the pass rate and a per-evaluator summary:

TEST CASE RESULTS
Total        15
Passed        <n>
Failed        <n>
Errored       <n>
Skipped       <n>
Pass rate  <percent> (<n> passed / 15 total test cases)

EVALUATOR RESULTS
EVALUATOR       PASS  FAIL  SKIP  ERROR   SCORED  PASS RATE  MEAN SCORE
...

Step 5: Run the optimizer

Run the optimizer with the evaluation that you set up in Step 4:

azd ai agent optimize \
  --eval optimize-check \
  --max-candidates 2 \
  --eval-model <eval-deployment> \
  --optimize-model <optimization-deployment>

The --eval option reuses the optimize-check evaluation that azd ai eval init added to your project. The optimizer takes the dataset and the evaluators from that evaluation, so you don't pass --dataset or --evaluator. It reuses only an evaluation that targets an agent and runs at the turn level on a dataset, as the Step 4 evaluation does.

If you omit --eval, the CLI uses the evaluation in your project that targets the agent. If more than one evaluation targets the agent, the CLI prompts you to select one. With --no-prompt, it fails, so pass --eval. To select the evaluation from a configuration file, pass --eval-config with the path to the file, such as evals/azure.eval.yaml.

To score candidates on a different dataset, pass --dataset with the path to a local JSONL file or with a registered dataset as name:version. To add evaluators, pass --evaluator once for each evaluator.

If you omit --optimize-model, the CLI prompts you to select a model. In unattended runs, pass --eval-model and --optimize-model. The CLI detects your agent from azure.yaml. A job with two candidates can take 15 to 20 minutes. The CLI shows progress as the job runs:

  Note: Optimization creates candidate agents as draft versions.
  Your live agent versions are not affected until you explicitly deploy a candidate.

Optimizing agent "customer-support-py"...
  Config: <evaluation-service>/optimize-check
  Baseline saved to .agent_configs/baseline/metadata.yaml
  Job ID: opt_162bd0f09....
  Status: queued
  Portal: <OPTIMIZATION-JOB-URL>

Use the portal URL to monitor your job in the Foundry portal. The Config line shows the source and the name of the evaluation that the optimizer reuses. The source is the evaluation service, or the configuration file when you pass --eval-config. To check the job from the terminal, run azd ai agent optimize status <job-id>.

The eval model scores each response (any chat-completion model works). The optimization model (--optimize-model) generates improved candidates and must be from the supported list (gpt-5 family or DeepSeek).

Step 6: Deploy the winner

The star (★) in the output indicates the best candidate.

azd env get-values

azd env set AGENT_<service-key>_OPTIMIZATION_JOB_ID <job-id>

Apply the optimized config locally, then deploy:

azd ai agent optimize apply --candidate <candidate-id>
azd deploy

The apply command downloads the optimized configuration into .agent_configs/<candidate_id>/, adds OPTIMIZATION_CANDIDATE_ID to your azure.yaml service environment, and prints a diff of the instructions. The deploy command pushes the optimized agent live using code deploy.

Invoke your agent to check the new behavior:

azd ai agent invoke "What is your return policy?"

Step 7: Verify with an independent evaluation

The optimizer scores candidates with its own evaluators on its own run. Confirm that the deployed agent improved by running the same evaluation that you ran in Step 4:

azd ai eval run start --eval optimize-check --name optimized --wait

Compare the pass rate and each evaluator's results with the baseline run. To see why rows failed, run the azd ai eval run output list ... --failed-only command that the run prints under Investigate.

An optimizer score gain doesn't guarantee a gain on an independent evaluation. If the optimized agent doesn't improve on the baseline, don't keep it. Review the failing rows, adjust the dataset or rubric, and run the optimizer again.

To check the pass rate in a pipeline, add --fail-on pass-rate=<0..1> to azd ai eval run start. The command exits with code 1 when the run's pass rate is below that fraction. The check doesn't compare against the baseline run.

Python SDK path

Use the following steps if you want to run the optimizer from Python instead of the Azure Developer CLI workflow described earlier.

This path assumes you already have the following resources in an existing Foundry project:

• A hosted agent to optimize.
• A registered training dataset.
• A registered evaluator.

Unlike the Azure Developer CLI flow described earlier, the Python SDK path doesn't scaffold a project or generate a dataset or evaluators for you. If you want to create those assets by using the Azure Developer CLI, install the azure.ai.evaluations extension with azd ext install azure.ai.evaluations (it requires azd 1.33.0 or later), and then run azd ai eval generate.

1. Create a .env file

Create a working folder, and then add a .env file with these values:

FOUNDRY_PROJECT_ENDPOINT=<your-project-endpoint>
FOUNDRY_AGENT_NAME=<your-hosted-agent-name>
DATASET_NAME=<your-registered-dataset-name>
EVALUATOR_NAME=<your-registered-evaluator-name>
DATASET_VERSION=1
POLL_INTERVAL_SECONDS=10
EVAL_MODEL=<your-eval-model-deployment-name>
OPTIMIZATION_MODEL=<your-optimization-model-deployment-name>

Run the script from this same working folder so load_dotenv() can load the .env file automatically. If you prefer to run it from another directory, set the same values in your shell environment first.

Use the exact project endpoint from your Foundry project's Overview page. The Python script sends its first request immediately. If FOUNDRY_PROJECT_ENDPOINT is only a placeholder or points to the wrong project, the run fails with ResourceNotFound: The project does not exist.

Set EVAL_MODEL and OPTIMIZATION_MODEL to deployment names that already exist in your Foundry project, not just model family names. For example, if your project deployment is named gpt-4.1-mini or DeepSeek-V3.2, use that exact deployment name in .env.

2. Run the optimization job

Create a file named optimize_hosted_agent.py in the same folder as .env:

"""
DESCRIPTION:
    Create an optimization job for a hosted agent using the latest agent
    optimization API, poll the job to completion, and list its candidates.

USAGE:
    python optimize_hosted_agent_v3.py

    Before running the sample:

    pip install azure-ai-projects azure-ai-agentserver-optimization python-dotenv

    Set these environment variables with your own values:
    1) FOUNDRY_PROJECT_ENDPOINT - Required. The Microsoft Foundry project endpoint.
    2) FOUNDRY_AGENT_NAME       - Required. The hosted agent name.
    3) DATASET_NAME             - Required. The registered training dataset name.
    4) EVALUATOR_NAME           - Required. The registered evaluator name.
    5) DATASET_VERSION          - Optional. The dataset version. Defaults to "1".
    6) EVAL_MODEL               - Optional. The evaluation model. Defaults to "gpt-4o".
    7) OPTIMIZATION_MODEL       - Optional. The optimization model. Defaults to "gpt-5".
    8) POLL_INTERVAL_SECONDS    - Optional. Seconds between status polls. Defaults to 10.
    9) OPTIMIZATION_JOB_ID      - Optional. An existing job ID to resume instead of creating a job.
   10) OPTIMIZATION_LOCAL_DIR   - Optional. Path to the local .agent_configs directory.
   11) CREDENTIAL_PROCESS_TIMEOUT_SECONDS
                                - Optional. Credential subprocess timeout. Defaults to 60.
"""

import os
import time
from pathlib import Path

from azure.ai.agentserver.optimization import load_config, load_skills_from_dir
from azure.ai.projects import AIProjectClient
from azure.ai.projects.models import (
    AgentOptimizationBaselineAgentConfiguration,
    AgentOptimizationCandidateExpand,
    AgentOptimizationCandidateSearchConfiguration,
    AgentOptimizationConfiguration,
    AgentOptimizationEvaluationConfiguration,
    AgentOptimizationEvaluator,
    AgentOptimizationFoundryAgentTargetConfiguration,
    AgentOptimizationJob,
    AgentOptimizationModelConfiguration,
    AgentOptimizationSkill,
    AgentOptimizationSpace,
    AgentOptimizationTargetCompletionDatasetReferenceDataSource,
    AgentOptimizationTargetCompletionEvaluationSet,
    EvaluationModelConfiguration,
    JobStatus,
    TargetAttribute,
)
from azure.identity import DefaultAzureCredential
from dotenv import load_dotenv

load_dotenv()

endpoint = os.environ["FOUNDRY_PROJECT_ENDPOINT"]
agent_name = os.environ["FOUNDRY_AGENT_NAME"]
dataset_name = os.environ["DATASET_NAME"]
evaluator_name = os.environ["EVALUATOR_NAME"]
dataset_version = os.environ.get("DATASET_VERSION", "1")
eval_model = os.environ.get("EVAL_MODEL", "gpt-4o")
optimization_model = os.environ.get("OPTIMIZATION_MODEL", "gpt-5")
poll_interval_seconds = int(os.environ.get("POLL_INTERVAL_SECONDS", "10"))
existing_job_id = os.environ.get("OPTIMIZATION_JOB_ID")
credential_process_timeout_seconds = int(
    os.environ.get("CREDENTIAL_PROCESS_TIMEOUT_SECONDS", "60")
)

# Reads the hosted agent's baseline config from .agent_configs/baseline/metadata.yaml.
optimization_config = load_config()

target_attributes = [TargetAttribute.INSTRUCTIONS]
baseline_configuration = None
if optimization_config:
    loaded_skills = optimization_config.skills
    if not loaded_skills and optimization_config.skills_dir:
        loaded_skills = load_skills_from_dir(Path(optimization_config.skills_dir))

    skills = [
        AgentOptimizationSkill(
            name=skill.name,
            description=skill.description,
            body=skill.body or None,
        )
        for skill in loaded_skills
    ]

    if skills:
        target_attributes.append(TargetAttribute.SKILLS)
    if optimization_config.tool_definitions:
        target_attributes.append(TargetAttribute.TOOLS)

    baseline_configuration = AgentOptimizationBaselineAgentConfiguration(
        system_prompt=optimization_config.instructions,
        current_model=optimization_config.model,
        skills=skills or None,
        tools=optimization_config.tool_definitions or None,
    )

with (
    DefaultAzureCredential(
        process_timeout=credential_process_timeout_seconds
    ) as credential,
    AIProjectClient(endpoint=endpoint, credential=credential) as project_client,
):
    job_request = AgentOptimizationJob(
        target_configuration=AgentOptimizationFoundryAgentTargetConfiguration(
            name=agent_name
        ),
        optimization_model_configuration=AgentOptimizationModelConfiguration(
            model=optimization_model
        ),
        optimization_configuration=AgentOptimizationConfiguration(
            evaluation_configuration=AgentOptimizationEvaluationConfiguration(
                training_set=AgentOptimizationTargetCompletionEvaluationSet(
                    source=AgentOptimizationTargetCompletionDatasetReferenceDataSource(
                        name=dataset_name,
                        version=dataset_version,
                    )
                ),
                evaluators=[AgentOptimizationEvaluator(name=evaluator_name)],
                evaluation_model=EvaluationModelConfiguration(model=eval_model),
            ),
            candidate_search_configuration=AgentOptimizationCandidateSearchConfiguration(
                max_candidates=2
            ),
            baseline_agent_configuration=baseline_configuration,
            agent_optimization_space=AgentOptimizationSpace(
                target_attributes=target_attributes
            ),
        ),
    )

    if existing_job_id:
        job_id = existing_job_id
    else:
        poller = project_client.agents.begin_create_optimization_job(
            job=job_request,
            polling=False,
        )
        job_id = poller.details.get("job_id")
        if not isinstance(job_id, str):
            raise RuntimeError(
                "The create operation did not return an optimization job ID."
            )

    print(f"Optimization job ID: {job_id}")

    terminal_statuses = {
        JobStatus.SUCCEEDED,
        JobStatus.FAILED,
        JobStatus.CANCELLED,
    }
    job = project_client.agents.get_optimization_job(job_id=job_id)
    print("Optimization job started, waiting for completion...")
    while job.status not in terminal_statuses:
        print(f"\tstatus=`{job.status}`")
        time.sleep(poll_interval_seconds)
        job = project_client.agents.get_optimization_job(job_id=job_id)
    print(f"\tstatus=`{job.status}`")

    for warning in job.warnings or []:
        print(f"[WARNING] {warning}")

    if job.status == JobStatus.FAILED:
        message = job.error.message if job.error else "<no error message>"
        raise RuntimeError(f"Optimization job `{job.id}` failed: {message}")
    if job.status == JobStatus.CANCELLED:
        raise RuntimeError(f"Optimization job `{job.id}` was cancelled.")
    if job.result is None:
        raise RuntimeError(f"Optimization job `{job.id}` completed without a result.")

    result = job.result
    summary = result.candidate_summary
    if summary:
        print(f"Baseline candidate: {summary.baseline_id}")
        print(f"Best candidate: {summary.best_id}")
        print(f"Completed optimized candidates: {summary.completed_candidate_count}")
        if summary.baseline_score is not None:
            print(f"Baseline score: {summary.baseline_score:.4f}")
        if summary.best_score is not None:
            print(f"Best score: {summary.best_score:.4f}")
    if result.termination_reason:
        print(f"Termination reason: {result.termination_reason}")

    for candidate in project_client.agents.list_optimization_candidates(
        job_id=job_id,
        expand=[AgentOptimizationCandidateExpand.MUTATIONS],
    ):
        details = [
            f"{candidate.name}: candidate_id={candidate.candidate_id}",
            f"status={candidate.status}",
        ]
        if candidate.evaluation:
            if candidate.evaluation.score is not None:
                details.append(f"score={candidate.evaluation.score:.4f}")
            if candidate.evaluation.avg_tokens is not None:
                details.append(f"avg_tokens={candidate.evaluation.avg_tokens:.0f}")
        print(", ".join(details))

Run the script:

python optimize_hosted_agent.py

The optimization job ID prints immediately after submission; use it to monitor progress in the Foundry portal.

When the job succeeds, the script prints the winning candidate and its candidate_id.

Unlike azd ai agent optimize, the Python SDK flow doesn't create a local .agent_configs/baseline/metadata.yaml file. The optimization job metadata stays in the returned job object and in the Foundry service response, including the baseline candidate, best candidate, and scored candidate list.

3. Apply the winning candidate

If you're also working from the local azd project used in the CLI flow above, apply the winning candidate by using the candidate_id returned by the Python script:

azd ai agent optimize apply --candidate <candidate-id>
azd deploy

If you only need to inspect the result, use the candidate scores and evaluation identifiers printed by the script to review the winning configuration in Foundry before promoting it.

C# SDK path

Use the following steps if you want to run the optimizer from .NET instead of using the Azure Developer CLI workflow described earlier.

This path assumes you already have the following resources in an existing Foundry project:

• A hosted agent to optimize.
• A registered training dataset.
• A registered evaluator, such as the built-in builtin.task_adherence evaluator.

Unlike the Azure Developer CLI flow described earlier, the .NET SDK path doesn't scaffold a project or generate a dataset or evaluators for you. If you want to create those assets by using the Azure Developer CLI, install the azure.ai.evaluations extension with azd ext install azure.ai.evaluations (it requires azd 1.33.0 or later), and then run azd ai eval generate.

1. Set your environment variables

Create a console app, and then set these values in your shell before you run it:

FOUNDRY_PROJECT_ENDPOINT=<your-project-endpoint>
FOUNDRY_AGENT_NAME=<your-hosted-agent-name>
DATASET_NAME=<your-registered-dataset-name>
DATASET_VERSION=1
EVALUATOR_NAME=builtin.task_adherence
EVAL_MODEL=<your-eval-model-deployment-name>
OPTIMIZATION_MODEL=<your-optimization-model-deployment-name>

Use the exact project endpoint from your Foundry project's Overview page.

Set EVAL_MODEL and OPTIMIZATION_MODEL to deployment names that already exist in your Foundry project, not just model family names. The optimization model must be a reasoning model that the optimizer supports. If you pick an unsupported deployment, the service returns an error that lists the allowed models.

2. Run the optimization job

Replace Program.cs with the following code.

Agent optimization requires the AgentsOptimization=V2Preview feature header on every request. The FoundryFeaturesPolicy class in this example adds that header to every request the client sends:

using System.ClientModel.Primitives;
using Azure.AI.Projects;
using Azure.AI.Projects.Agents;
using Azure.Identity;

#pragma warning disable AAIP001

// Adds the feature header that agent optimization requires.
public sealed class FoundryFeaturesPolicy(string features) : PipelinePolicy
{
    public override void Process(PipelineMessage message, IReadOnlyList<PipelinePolicy> pipeline, int index)
    {
        message.Request.Headers.Set("Foundry-Features", features);
        ProcessNext(message, pipeline, index);
    }

    public override ValueTask ProcessAsync(PipelineMessage message, IReadOnlyList<PipelinePolicy> pipeline, int index)
    {
        message.Request.Headers.Set("Foundry-Features", features);
        return ProcessNextAsync(message, pipeline, index);
    }
}

public static class Program
{
    public static void Main()
    {
        var endpoint = Environment.GetEnvironmentVariable("FOUNDRY_PROJECT_ENDPOINT")!;
        var agentName = Environment.GetEnvironmentVariable("FOUNDRY_AGENT_NAME")!;
        var datasetName = Environment.GetEnvironmentVariable("DATASET_NAME")!;
        var datasetVersion = Environment.GetEnvironmentVariable("DATASET_VERSION") ?? "1";
        var evaluatorName = Environment.GetEnvironmentVariable("EVALUATOR_NAME") ?? "builtin.task_adherence";
        var evalModel = Environment.GetEnvironmentVariable("EVAL_MODEL")!;
        var optimizationModel = Environment.GetEnvironmentVariable("OPTIMIZATION_MODEL")!;

        var options = new AIProjectClientOptions();
        options.AddPolicy(new FoundryFeaturesPolicy("AgentsOptimization=V2Preview"), PipelinePosition.PerCall);

        AIProjectClient projectClient = new(new Uri(endpoint), new DefaultAzureCredential(), options);
        AgentOptimizationJobs optimizationJobs = projectClient.AgentAdministrationClient.GetAgentOptimizationJobs();

        OptimizationJob job = new()
        {
            Inputs = new OptimizationJobInputs(
                new OptimizationAgentIdentifier(agentName),
                new OptimizationReferenceDatasetInput(datasetName) { Version = datasetVersion },
                new[] { new OptimizationEvaluatorRef(evaluatorName) })
            {
                Options = new OptimizationOptions
                {
                    MaxCandidates = 2,
                    EvalModel = evalModel,
                    OptimizationModel = optimizationModel,
                    // The optimizer needs at least one optimizable target, such as the
                    // baseline system prompt, the tool definitions, or skills.
                    OptimizationConfig =
                    {
                        ["system_prompt"] = BinaryData.FromObjectAsJson(
                            "You are a helpful assistant that answers user requests accurately and concisely."),
                    },
                },
            },
        };

        OptimizationJob created = optimizationJobs.Create(job);
        Console.WriteLine($"Optimization job started: {created.Id}");

        OptimizationJob current = created;
        while (current.Status != AgentsJobStatus.Succeeded
            && current.Status != AgentsJobStatus.Failed
            && current.Status != AgentsJobStatus.Cancelled)
        {
            Thread.Sleep(TimeSpan.FromSeconds(20));
            current = optimizationJobs.Get(created.Id);
            Console.WriteLine($"\tstatus=`{current.Status}`");
        }

        Console.WriteLine($"Final status: {current.Status}");

        if (current.Result is not null)
        {
            Console.WriteLine($"Baseline candidate: {current.Result.Baseline}");
            Console.WriteLine($"Best candidate: {current.Result.Best}");

            foreach (OptimizationCandidate candidate in current.Result.Candidates)
            {
                Console.WriteLine(
                    $"{candidate.Name}: candidate_id={candidate.CandidateId}, " +
                    $"avg_score={candidate.AvgScore:F4}, " +
                    $"avg_tokens={candidate.AvgTokens:F0}");
            }
        }
    }
}

Run the app:

dotnet run

When the job succeeds, the app prints the winning candidate and its candidate_id:

Optimization job started: opt_<job-id>
        status=`in_progress`
        status=`succeeded`
Final status: succeeded
Baseline candidate: cand_opt_<job-id>_0000
Best candidate: cand_opt_<job-id>_0000
baseline: candidate_id=cand_opt_<job-id>_0000, avg_score=1.0000, avg_tokens=0

Unlike azd ai agent optimize, the .NET SDK flow doesn't create a local .agent_configs/baseline/metadata.yaml file. The optimization job metadata stays in the returned job object and in the Foundry service response, including the baseline candidate, best candidate, and scored candidate list.

3. Apply the winning candidate

If you're also working from the local azd project used in the CLI flow, apply the winning candidate by using the candidate_id returned by the app:

azd ai agent optimize apply --candidate <candidate-id>
azd deploy

If you only need to inspect the result, use the candidate scores and evaluation identifiers printed by the app to review the winning configuration in Foundry before promoting it.

Run the optimization in VS Code

Foundry Toolkit includes a native Agent Optimization experience for deployed hosted agents. From the agent playground, you can start an optimization run, compare candidates with the baseline, inspect configuration changes, and deploy the best candidate.

Step 1: Select a deployed hosted agent

1. Select Foundry Toolkit in the Activity Bar.
2. Under My Resources, select Agents.
3. If you have a deployed hosted agent, select it to open the hosted agent playground.
4. If you don't have a deployed hosted agent, complete the VS Code path in Quickstart: Deploy your first hosted agent. After deployment finishes, return to Agents and select the new hosted agent.

Step 2: Start an optimization run

1. Select the Optimize tab.

1. Select New Optimization.

2. In Select Workspace, choose the workspace that contains the selected hosted agent's code:

   • Select Current workspace if the current workspace contains the agent code and its azure.yaml file.
   • Select Browse... to open the workspace that contains the agent code.

   Foundry Toolkit uses the workspace files to prepare the optimization and to apply a candidate to the matching azure.ai.agent service.

1. Foundry Toolkit opens GitHub Copilot Chat and sends an Agent Optimizer request populated with the selected agent's kind, name, and Foundry project endpoint.

2. Answer the four optimization questions in Copilot Chat:

   Input	What to provide
   Evaluation metrics	Enter the metrics or evaluators to use. If you don't have them, choose whether to generate them with azd ai eval generate (use --from prompt for the dataset).
   Dataset	Select the optimization dataset. If you don't have one, choose whether to generate one with azd ai eval generate (use --from prompt).
   Maximum candidates	Enter the maximum number of candidates to generate, such as 2.
   Optimization model	Select an existing deployment from the supported optimization models.

GitHub Copilot waits for these inputs before it starts optimization. The generated request directs Copilot to use the Microsoft Foundry Skill's Agent Optimizer workflow and Azure Developer CLI commands exclusively. It doesn't use Foundry MCP tools. Copilot:

• Inspects the agent code in the selected workspace.
• Initializes an AZD environment from existing azure.yaml and .env values if the project doesn't already have one.
• Wires the agent for optimization and deploys the updated hosted agent.
• Starts the optimization after you review and approve the proposed file changes and commands.

After Copilot submits the job, return to the Optimize tab. The run appears under Optimization runs. The table shows its run ID, status, candidate count, baseline score, best score, and creation time.

Step 3: Compare and deploy the best candidate

1. When the run succeeds, select it under Optimization runs.
2. Compare the Baseline and Best scores. Review Score details for each candidate, and select View changes to inspect its configuration changes.
3. If the best candidate improves on the baseline, select Deploy best candidate to update the current agent. To deploy it as a new agent or change deployment settings, select Custom deploy instead.

Run the optimization with the Microsoft Foundry Skill

Use this path in any coding agent host that supports the Microsoft Foundry Skill, such as GitHub Copilot in Visual Studio Code, Copilot CLI, or Claude Code. The skill resolves the agent context from azure.yaml, loads its Agent Optimizer workflow, and keeps candidate application and deployment behind review gates.

Step 1: Open the agent workspace

Open an empty folder in your coding agent host. Confirm that the microsoft-foundry skill is available. If the skill isn't available, follow Use the Microsoft Foundry Skill in coding agents.

Step 2: Ask the skill to run Agent Optimizer

Submit this prompt to your coding agent:

Use the Microsoft Foundry Skill to run the Agent Optimizer workflow for a
Python hosted agent. If this workspace doesn't contain an agent, initialize the
customer support optimization sample from this template:
https://github.com/microsoft-foundry/foundry-samples/blob/main/samples/python/hosted-agents/bring-your-own/responses/optimization-customer-support/azure.yaml
Resolve the AZD environment and hosted-agent service, verify that the agent is
optimizer-ready, and deploy and invoke the baseline. If the azd ai eval
commands aren't available, run azd ext install azure.ai.evaluations first (it
requires azd 1.33.0 or later). Generate the evaluation
dataset and evaluators by using azd ai eval generate, with --from prompt for the
dataset, and show me them. Set up the evaluation by using azd ai eval init and
azd ai eval create, and run a baseline evaluation by using azd ai eval run
start. Verify that the project has a supported optimization model deployment,
then run Agent Optimizer with two candidates by reusing that evaluation with
azd ai agent optimize --eval. Stop after reporting the operation ID,
portal URL, candidate IDs, and scores. Don't apply or deploy a candidate yet.

The coding agent might ask you to select a subscription, region, Foundry project, agent service, evaluation model, or optimization model when it can't resolve those values from the workspace. Review generated files and cost-bearing resources before you approve any changes or commands.

Step 3: Apply and deploy an approved candidate

After you review the optimization results, submit this follow-up prompt:

Recommend the best optimization candidate and explain the score improvement.
Summarize the candidate changes before applying anything. After I approve the
candidate, apply it locally, show the source diff, and stop again before
deployment. After I approve deployment, run azd deploy, invoke the agent with
"What is your return policy?", and rerun the same azd ai eval evaluation with a
new run name. Compare it with the baseline run to confirm the improvement.

The skill uses azd ai agent optimize apply --candidate <candidate-id> so you can review the optimized configuration locally. It deploys only after your approval, then invokes the updated hosted agent and reruns the evaluation.