你当前正在访问 Microsoft Azure Global Edition 技术文档网站。 如果需要访问由世纪互联运营的 Microsoft Azure 中国技术文档网站,请访问 https://docs.azure.cn。

Azure Agents client library for Java - version 2.10.0

Develop Agents using the Azure AI Foundry platform, leveraging an extensive ecosystem of models, tools, and capabilities from OpenAI, Microsoft, and other LLM providers.

The client library uses a single service version v1 of the AI Foundry data plane REST APIs.

Important

Preview and beta features

  • Build Beta*Client and Beta*AsyncClient instances through AgentsClientBuilder.beta(). These clients automatically opt in to their preview service area; you do not need allowPreview(true) for them.
  • Use AgentsClientBuilder.allowPreview(true) when calling preview APIs on non-Beta clients, such as preview agent definitions, draft agent versions, hosted-agent sessions, session files, and code package operations on AgentsClient / AgentsAsyncClient.
  • Classes and methods annotated with @Beta are preview API surface and may change in future releases. See Preview operation groups and beta clients for details.

Documentation

Various documentation is available to help you get started

Getting started

Prerequisites

Adding the package to your product

<dependency>
    <groupId>com.azure</groupId>
    <artifactId>azure-ai-agents</artifactId>
    <version>2.10.0</version>
</dependency>

Authentication

Azure Identity package provides the default implementation for authenticating the client.

Key concepts

Create an AgentsClient

To interact with the Azure Agents service, you'll need to create an instance of the AgentsClient class.

AgentsClient agentsClient = new AgentsClientBuilder()
                .credential(new DefaultAzureCredentialBuilder().build())
                .endpoint(endpoint)
                .buildAgentsClient();

Alternatively, you can create an asynchronous client using the AgentsAsyncClient class.

AgentsAsyncClient agentsAsyncClient = new AgentsClientBuilder()
                .credential(new DefaultAzureCredentialBuilder().build())
                .endpoint(endpoint)
                .buildAgentsAsyncClient();

The Agents client library has the following sub-clients which group the different operations that can be performed:

  • AgentsClient / AgentsAsyncClient: Perform operations related to agents, including agent generation and optimization. When allowPreview(true) is configured, these clients can also use preview definitions, hosted-agent sessions, session files, and code package operations.
  • ResponsesClient / ResponsesAsyncClient: Create responses that require Azure-specific request fields, such as an explicit AgentReference or structured inputs. For standard OpenAI Responses API calls through an agent endpoint, use an agent-scoped OpenAI client. See the OpenAI Responses API documentation for more information.
  • BetaMemoryStoresClient / BetaMemoryStoresAsyncClient (preview): Manage memory stores and individual memory items for agents.
  • ToolboxesClient / ToolboxesAsyncClient: Manage toolboxes and toolbox versions.
  • VoiceAgentsWebSocketClient / VoiceAgentsWebSocketAsyncClient: Open typed realtime WebSocket sessions with voice agents.
  • BetaVoiceAgentsLiveWebSocketClient / BetaVoiceAgentsLiveWebSocketAsyncClient (preview): Open native GPT Live WebSocket sessions with persisted Live voice agents.
  • VoiceAgentsTelephonyClient / VoiceAgentsTelephonyAsyncClient: Manage telephony bindings, calls, transfer targets, and outbound call jobs.
  • BetaVoiceAgentsTelephonyClient / BetaVoiceAgentsTelephonyAsyncClient (preview): Manage outbound telephony campaigns.
  • VoiceAgentsConversationsClient / VoiceAgentsConversationsAsyncClient: Read and delete persisted voice-agent conversations and retrieve their responses, items, and audio.

OpenAI conversation operations are accessed through the OpenAI Official Java SDK's ConversationService. Persisted voice-agent conversation records use the voice conversation clients listed above. See the OpenAI Conversation API documentation for more information.

To access each sub-client you need to use your AgentsClientBuilder(). The Agents client library takes the Official OpenAI SDK as a dependency, which is used for all operations, except the ones corresponding to direct Agent management.

AgentsClientBuilder builder = new AgentsClientBuilder()
                .credential(new DefaultAzureCredentialBuilder().build())
                .endpoint(endpoint)
                .allowPreview(true); // Only needed for preview APIs on non-Beta clients that support them.

// Agents sub-clients
AgentsClient agentsClient = builder.buildAgentsClient();
AgentsAsyncClient agentsAsyncClient = builder.buildAgentsAsyncClient();
// Responses sub-clients.
ResponsesClient responsesClient = builder.buildResponsesClient();
ResponsesAsyncClient responsesAsyncClient = builder.buildResponsesAsyncClient();
// Memory Stores sub-clients (preview).
BetaMemoryStoresClient memoryStoresClient = builder.beta().buildBetaMemoryStoresClient();
BetaMemoryStoresAsyncClient memoryStoresAsyncClient = builder.beta().buildBetaMemoryStoresAsyncClient();
// Toolboxes sub-clients.
ToolboxesClient toolboxesClient = builder.buildToolboxesClient();
ToolboxesAsyncClient toolboxesAsyncClient = builder.buildToolboxesAsyncClient();

The OpenAI Official Java SDK is imported transitively and can be built directly from the AgentsClientBuilder. Use it to access conversation operations and other OpenAI services:

OpenAIClient openAIClient = builder.buildOpenAIClient();
OpenAIClientAsync openAIAsyncClient = builder.buildOpenAIAsyncClient();

// Agent-scoped OpenAI clients for invoking an agent endpoint.
OpenAIClient agentScopedOpenAIClient = builder.buildAgentScopedOpenAIClient(agentName);
OpenAIClientAsync agentScopedOpenAIAsyncClient = builder.buildAgentScopedOpenAIAsyncClient(agentName);

// ResponsesClient wraps the OpenAI SDK's ResponseService with Azure-specific options.
ResponsesClient responsesClient = builder.buildResponsesClient();
ResponseService responseService = responsesClient.getResponseService();

// OpenAI SDK ConversationService accessed from OpenAIClient
ConversationService conversationService = agentScopedOpenAIClient.conversations();

Agent version drafts

Draft agent versions are preview candidates that are not promoted to the agent's latest released version. Create one with CreateAgentVersionInput.setDraft(true), and pass true as the includeDrafts argument to listAgentVersions when you need to list draft versions. Build the non-Beta client with allowPreview(true) to opt in to the DraftAgents=V1Preview service feature.

See the full samples in AgentDraftSample.java and AgentDraftAsyncSample.java.

Agent tools

The SDK supports a variety of tools that can be attached to agent definitions. Some tools are generally available, while others are in preview and may change in future releases.

Generally available tools:

Tool class Description
A2ATool Agent-to-agent (A2A) protocol
AzureAISearchTool Azure AI Search
AzureFunctionTool Azure Functions
BingGroundingTool Bing grounding
BrowserAutomationTool Browser automation
CaptureStructuredOutputsTool Structured output capture
CodeInterpreterTool Code interpreter
FileSearchTool File search
FunctionTool Custom function calling
ImageGenTool Image generation
McpTool Model Context Protocol (MCP)
NamespaceTool Namespaces for grouping function and custom tools
OpenApiTool OpenAPI spec-based tools
ToolSearchTool Deferred tool search
WebSearchTool Web search

Preview tools:

Tool class Description
BingCustomSearchPreviewTool Bing custom search
BrowserAutomationPreviewTool Deprecated; use BrowserAutomationTool
ComputerUsePreviewTool Computer use
FabricIqPreviewTool Fabric IQ
MemorySearchPreviewTool Memory search
MicrosoftCopilotToolsetPreview Microsoft Copilot built-in tools
MicrosoftFabricPreviewTool Microsoft Fabric
ReminderPreviewTool Reminder scheduling
SharepointPreviewTool SharePoint grounding
WebIqPreviewTool WebIQ MCP servers
WebSearchPreviewTool Web search
WorkIqPreviewTool Work IQ

Supported tool classes may also expose optional name, description, and toolConfigs properties for user-defined labels and per-tool configuration.

Preview operation groups and beta clients

Several operation groups in the Agents client library expose preview service features. These features require the Foundry-Features HTTP header. The SDK populates that header for you; you do not need to set the header value manually.

APIs annotated with @Beta are part of the SDK's preview surface, even when they appear on a non-Beta client. These APIs are subject to breaking changes in future releases and should be used with the same compatibility expectations as other preview features.

Use AgentsClientBuilder.allowPreview(true) when building non-Beta clients that support preview service behavior. For example, AgentsClient and AgentsAsyncClient use this builder setting to allow the service to return preview response types:

AgentsClientBuilder builder = new AgentsClientBuilder()
    .credential(new DefaultAzureCredentialBuilder().build())
    .endpoint(endpoint)
    .allowPreview(true);

AgentsClient agentsClient = builder.buildAgentsClient();

Build clients whose names start with Beta from AgentsClientBuilder.beta(). These clients always opt in to their corresponding preview service area. Requests sent by these clients automatically include the appropriate Foundry-Features header, and their APIs can send or return preview/beta request and response types. You do not need to call allowPreview(true) to use a Beta*Client.

Beta sub-client Automatically populated Foundry-Features value
BetaMemoryStoresClient MemoryStores=V1Preview
BetaVoiceAgentsLiveWebSocketClient / BetaVoiceAgentsLiveWebSocketAsyncClient VoiceAgents=V1Preview
BetaVoiceAgentsTelephonyClient VoiceAgents=V1Preview
BetaVoiceAgentsLiveConversationsClient VoiceAgents=V1Preview

The async Beta*AsyncClient counterparts follow the same behavior.

Realtime voice-agent sessions

Create and manage preview voice agents with VoiceAgentDefinition and an AgentsClient or AgentsAsyncClient built using allowPreview(true). Use VoiceAgentsWebSocketClient or VoiceAgentsWebSocketAsyncClient to open a typed, bidirectional session with an existing voice agent. The client acquires a token for https://ai.azure.com/.default, negotiates the realtime WebSocket subprotocol, and sends the Foundry-Features: VoiceAgents=V1Preview header only when built with allowPreview(true). GPT Live WebSocket clients always send that preview header, independently of allowPreview.

The session API supports text and audio in the format configured on the voice agent, typed streaming server events, response cancellation, client-executed function tools, and optional persisted conversations. See Realtime voice-agent WebSocket examples for complete samples.

Use VoiceAgentWebSocketConnectionOptions with openWebSocketSession to configure session IDs, agent version selection, transport, structured inputs, persistence, buffering, and timeouts. Options are copied when the session is opened, so later changes do not affect the active session.

VoiceAgentWebSocketConnectionOptions options = new VoiceAgentWebSocketConnectionOptions()
    .setAgentSessionId("session-id")
    .setAgentVersionOverride("2")
    .setStructuredInputs("{\"language\":\"en\"}")
    .setStoreEnabled(true);

The SDK owns the WebSocket route, API version, and authentication scope. Endpoint, credential, service version, configuration-based proxy settings, and ClientOptions are reused from AgentsClientBuilder. Custom HTTP clients, pipelines, policies, and retry settings are rejected when building a WebSocket client because the native WebSocket transports cannot apply them.

Agent optimization

AgentsClient and AgentsAsyncClient create and monitor agent optimization jobs. Construct AgentOptimizationJob with AgentOptimizationModelConfiguration and AgentOptimizationConfiguration (or PromptOptimizationConfiguration for prompt optimization). These jobs return scored candidates for instructions, skills, tools, or model improvements. Use estimateOptimizationJob, listOptimizationCandidates, getOptimizationCandidate, and promoteOptimizationCandidate to estimate costs and manage candidates. See Agent optimizer in Foundry Agent Service for the service workflow and the complete examples in AgentOptimizationSample.java and AgentOptimizationAsyncSample.java.

Memory item management

BetaMemoryStoresClient and BetaMemoryStoresAsyncClient manage memory stores and individual memory items. In addition to store-level operations, use createMemory, updateMemory, listMemories, getMemory, and deleteMemory to manage individual memories. ListMemoriesOptions supports filtering by scope and MemoryItemKind, including MemoryItemKind.PROCEDURAL. See MemoryStoreItemsSample and MemoryStoreItemsAsyncSample for complete examples.

For conversational memory workflows, use beginUpdateMemories to extract memories from conversation items, searchMemories to retrieve relevant memories, and deleteScope to remove all memories for a scope. See MemoryStoreAdvancedSample and MemoryStoreAdvancedAsyncSample for complete synchronous and asynchronous examples.

Using OpenAI's official library

If you prefer using the OpenAI official Java client library instead, you can do so by including that dependency in your project instead and following the instructions in the linked repository. Additionally, you will have to set up your OpenAIClient as shown below:

OpenAIClient client = OpenAIOkHttpClient.builder()
    .baseUrl(endpoint.endsWith("/") ? endpoint + "openai/v1" : endpoint + "/openai/v1")
    .credential(BearerTokenCredential.create(AuthenticationUtil.getBearerTokenSupplier(
        new DefaultAzureCredentialBuilder().build(), "https://ai.azure.com/.default")))
    .build();

ResponseCreateParams responseRequest = new ResponseCreateParams.Builder()
    .input("Hello, how can you help me?")
    .model(model)
    .build();

Response result = client.responses().create(responseRequest);

For this direct setup, ensure that the AI Foundry project endpoint path ends with openai/v1, as shown above.

Examples

Prompt Agent

This example creates a prompt agent and uses a conversation to retain context across requests.

Create an Agent

Creating an Agent can be done like in the following code snippet:

PromptAgentDefinition promptAgentDefinition = new PromptAgentDefinition("gpt-4o");
AgentVersionDetails agent = agentsClient.createAgentVersion("my-agent", promptAgentDefinition);

This returns an AgentVersionDetails containing the agent's name and version. By default, the agent endpoint serves the latest version through the Responses protocol using Microsoft Entra authentication.

To pin the endpoint to a specific version, see ConfigureAgentEndpoint.java and ConfigureAgentEndpointAsync.java.

Create conversation

Use the same agent-scoped OpenAI client for conversations and responses:

OpenAIClient agentScopedClient = builder.buildAgentScopedOpenAIClient(agent.getName());
ConversationService conversationsClient = agentScopedClient.conversations();
Conversation conversation = conversationsClient.create();

Use conversation.id() to add messages and create subsequent responses for the same agent:

conversationsClient.items().create(
    ItemCreateParams.builder()
        .conversationId(conversation.id())
        .addItem(EasyInputMessage.builder()
            .role(EasyInputMessage.Role.SYSTEM)
            .content("You are a helpful assistant that speaks like a pirate.")
            .build()
        ).addItem(EasyInputMessage.builder()
            .role(EasyInputMessage.Role.USER)
            .content("Hello, agent!")
            .build()
    ).build()
);

To scope conversation operations to a delegated end user, set FOUNDRY_USER_IDENTITY to an opaque application-generated value and apply it as the x-ms-user-identity header. The caller must have the agents/endpoints/UserIdentityImpersonation/action RBAC permission. See the sync UserIdentityConversation.java and async UserIdentityConversationAsync.java samples.

Text generation with Responses

Invoke the OpenAI Responses API through the same client:

Response response = agentScopedClient.responses().create(ResponseCreateParams.builder()
    .conversation(conversation.id())
    .build());
// To extract Azure-specific response details:
AzureCreateResponseDetails azureResults = ResponsesClient.getAzureFields(response);

For asynchronous calls, use buildAgentScopedOpenAIAsyncClient.

Using Agent tools

Agents can be enhanced with specialized tools for various capabilities. For complete working examples, see the tools/ folder under samples.

In the description below, tools are organized by their Foundry connection requirements: "Built-in Tools" (which do not require a Foundry connection) and "Connection-based Tools" (which require a Foundry connection).

Built-in Tools

These tools work immediately without requiring external connections.


Code Interpreter (documentation)

Write and run Python code in a sandboxed environment, process files and work with diverse data formats.

// Create a CodeInterpreterTool with default auto container configuration
CodeInterpreterTool tool = new CodeInterpreterTool();

See the full sample in CodeInterpreterSync.java.


File Search (documentation)

Search through files in a vector store for knowledge retrieval:

// Create a FileSearchTool with the vector store ID
FileSearchTool tool = new FileSearchTool(Collections.singletonList(vectorStore.id()));

See the full sample in FileSearchSync.java.


Image Generation (documentation)

Generate images from text descriptions:

// Create image generation tool with model, quality, and size
ImageGenTool imageGenTool = new ImageGenTool()
    .setModel(ImageGenToolModel.fromString(imageModel))
    .setQuality(ImageGenToolQuality.LOW)
    .setSize(ImageGenToolSize.fromString("1024x1024"));

See the full sample in ImageGenerationSync.java.


Web Search (Preview) (documentation)

Search the web for current information:

// Create a WebSearchPreviewTool
WebSearchPreviewTool tool = new WebSearchPreviewTool();

See the full sample in WebSearchSync.java.


Computer Use (documentation)

Interact with computer interfaces through simulated actions and screenshots:

ComputerTool tool = new ComputerTool();

See the full sample in ComputerUseSync.java.


Model Context Protocol (MCP) (documentation)

Connect agents to external MCP servers:

// Uses gitmcp.io to expose a GitHub repository as an MCP-compatible server
McpTool tool = new McpTool("api-specs")
    .setServerUrl("https://gitmcp.io/Azure/azure-rest-api-specs")
    .setRequireApproval("always");

See the full sample in McpSync.java.


OpenAPI (documentation)

Call external APIs defined by OpenAPI specifications without additional client-side code:

// Load the OpenAPI spec from a JSON file
Map<String, BinaryData> spec = OpenApiFunctionDefinition.readSpecFromFile(
    SampleUtils.getResourcePath("assets/httpbin_openapi.json"));

OpenApiTool tool = new OpenApiTool(
    new OpenApiFunctionDefinition(
        "httpbin_get",
        spec,
        new OpenApiAnonymousAuthDetails())
        .setDescription("Get request metadata from an OpenAPI endpoint."));

See the full sample in OpenApiSync.java.


Function Tool (documentation)

Define custom functions that allow agents to interact with external APIs, databases, or application logic:

Map<String, Object> locationProp = new LinkedHashMap<String, Object>();
locationProp.put("type", "string");
locationProp.put("description", "The city and state, e.g. Seattle, WA");

Map<String, Object> unitProp = new LinkedHashMap<String, Object>();
unitProp.put("type", "string");
unitProp.put("enum", Arrays.asList("celsius", "fahrenheit"));

Map<String, Object> properties = new LinkedHashMap<String, Object>();
properties.put("location", locationProp);
properties.put("unit", unitProp);

Map<String, BinaryData> parameters = new HashMap<String, BinaryData>();
parameters.put("type", BinaryData.fromObject("object"));
parameters.put("properties", BinaryData.fromObject(properties));
parameters.put("required", BinaryData.fromObject(Arrays.asList("location", "unit")));
parameters.put("additionalProperties", BinaryData.fromObject(false));

FunctionTool tool = new FunctionTool("get_weather", parameters, true)
    .setDescription("Get the current weather in a given location");

See the full sample in FunctionCallSync.java.


Azure Functions

Integrate Azure Functions with agents to extend capabilities via serverless compute. Functions are invoked through Azure Storage Queue triggers, allowing asynchronous execution of custom logic:

// Create Azure Function tool with Storage Queue bindings
AzureFunctionTool azureFunctionTool = new AzureFunctionTool(
    new AzureFunctionDefinition(
        new AzureFunctionDefinitionDetails("queue_trigger", parameters)
            .setDescription("Get weather for a given location"),
        new AzureFunctionBinding(
            new AzureFunctionStorageQueue(queueServiceEndpoint, inputQueueName)),
        new AzureFunctionBinding(
            new AzureFunctionStorageQueue(queueServiceEndpoint, outputQueueName))
    )
);

When the agent handles a response, it enqueues function arguments to the input queue. Your Azure Function processes the request and returns results through the output queue.

See the full sample in AzureFunctionSync.java.


Memory Search (Preview) (documentation)

The Memory Search tool adds memory to an agent, allowing the agent's AI model to search for past information related to the current user prompt:

// Create memory search tool
MemorySearchPreviewTool tool = new MemorySearchPreviewTool(memoryStore.getName(), scope)
    .setUpdateDelaySeconds(1);

See the full sample in MemorySearchSync.java showing how to create an agent with a memory store and use it across multiple conversations.


Connection-Based Tools

These tools require configuring connections in your Microsoft Foundry project and use a projectConnectionId.


Azure AI Search (documentation)

Integrate with Azure AI Search indexes for powerful knowledge retrieval and semantic search capabilities:

// Create Azure AI Search tool with index configuration
AzureAISearchTool aiSearchTool = new AzureAISearchTool(
    new AzureAISearchToolResource(Arrays.asList(
        new AISearchIndexResource()
            .setProjectConnectionId(connectionId)
            .setIndexName(indexName)
            .setQueryType(AzureAISearchQueryType.SIMPLE)
    ))
);

See the full sample in AzureAISearchSync.java.


Bing Grounding (documentation)

Ground agent responses with real-time web search results from Bing to provide up-to-date information:

// Create Bing grounding tool with connection configuration
BingGroundingTool bingTool = new BingGroundingTool(
    new BingGroundingSearchToolParameters(Arrays.asList(
        new BingGroundingSearchConfiguration(bingConnectionId)
    ))
);

See the full sample in BingGroundingSync.java.


Bing Custom Search (Preview) (documentation)

Warning: Grounding with Bing Custom Search uses Grounding with Bing, which has additional costs and terms: terms of use and privacy statement. Customer data will flow outside the Azure compliance boundary.

Use custom-configured Bing search instances for domain-specific or filtered web search results:

// Create Bing Custom Search tool with connection and instance configuration
BingCustomSearchPreviewTool bingCustomSearchTool = new BingCustomSearchPreviewTool(
    new BingCustomSearchToolParameters(Arrays.asList(
        new BingCustomSearchConfiguration(connectionId, instanceName)
    ))
);

See the full sample in BingCustomSearchSync.java.


Microsoft Fabric (Preview) (documentation)

Query data from Microsoft Fabric data sources:

// Create Microsoft Fabric tool with connection configuration
MicrosoftFabricPreviewTool fabricTool = new MicrosoftFabricPreviewTool(
    new FabricDataAgentToolParameters()
        .setProjectConnections(Arrays.asList(
            new ToolProjectConnection(fabricConnectionId)
        ))
);

See the full sample in FabricSync.java.


Fabric IQ (Preview) (documentation)

Connect agents to Fabric IQ project connections for enterprise data grounding:


FabricIqPreviewTool fabricIqTool = new FabricIqPreviewTool(fabricIqConnectionId)
    .setServerLabel("fabric-iq-tool")
    .setRequireApproval("never");

The samples use FOUNDRY_PROJECT_ENDPOINT, FOUNDRY_MODEL_NAME, and the fully qualified FABRIC_IQ_PROJECT_CONNECTION_ID. FOUNDRY_AGENT_NAME and FABRIC_IQ_USER_INPUT are optional. The response text and any returned annotations are printed before the temporary agent version is deleted.

See the full samples in FabricIQSync.java and FabricIQAsync.java.


Work IQ (Preview) (documentation)

Ground agent responses in the signed-in user's Microsoft 365 work context through a Work IQ project connection:

// Create a Work IQ tool with a fully qualified project connection resource ID
WorkIqPreviewTool workIqTool = new WorkIqPreviewTool(workIqConnectionId);

Set FOUNDRY_PROJECT_ENDPOINT, FOUNDRY_MODEL_NAME, and WORK_IQ_PROJECT_CONNECTION_ID before running the sample. FOUNDRY_AGENT_NAME and WORK_IQ_USER_INPUT are optional. Work IQ uses delegated authentication and honors the signed-in user's Microsoft 365 permissions.

See the full samples in WorkIQSync.java and WorkIQAsync.java.


Microsoft SharePoint (Preview) (documentation)

Search through SharePoint documents for grounding:

// Create SharePoint grounding tool with connection configuration
SharepointPreviewTool sharepointTool = new SharepointPreviewTool(
    new SharepointGroundingToolParameters()
        .setProjectConnections(Arrays.asList(
            new ToolProjectConnection(sharepointConnectionId)
        ))
);

See the full sample in SharePointGroundingSync.java.


Browser Automation (documentation)

Interact with web pages using BrowserAutomationTool. For toolbox configuration, use BrowserAutomationToolboxTool.

The legacy BrowserAutomationPreviewTool and BrowserAutomationPreviewToolboxTool are deprecated and no longer available. Use BrowserAutomationTool and BrowserAutomationToolboxTool instead. See Browser Automation documentation.

// Create browser automation tool with connection configuration
BrowserAutomationTool browserTool = new BrowserAutomationTool(
    new BrowserAutomationToolParameters(
        new BrowserAutomationToolConnectionParameters(connectionId)
    )
);

See the full sample in BrowserAutomationSync.java.


Agent-to-Agent (A2A) (documentation)

Enable agent-to-agent communication with remote A2A endpoints:

// Create agent-to-agent tool with A2A protocol version and connection ID
A2ATool a2aTool = new A2ATool(A2AProtocolVersion.V1_0)
    .setProjectConnectionId(a2aConnectionId);

See the full sample in AgentToAgentSync.java.


MCP with Project Connection (documentation)

MCP integration using project-specific connections for accessing connected MCP servers:

// Create MCP tool with project connection authentication
McpTool mcpTool = new McpTool("api-specs")
    .setServerUrl("https://api.githubcopilot.com/mcp")
    .setProjectConnectionId(mcpConnectionId)
    .setRequireApproval("always");

See the full sample in McpWithConnectionSync.java.


OpenAPI with Project Connection (documentation)

This example calls TripAdvisor using tripadvisor_openapi.json. Set OPENAPI_PROJECT_CONNECTION_ID to a Custom Keys connection containing your TripAdvisor API key under key.

// Create OpenAPI tool with project connection authentication
OpenApiTool openApiTool = new OpenApiTool(
    new OpenApiFunctionDefinition(
        "tripadvisor",
        spec,
        new OpenApiProjectConnectionAuthDetails(
            new OpenApiProjectConnectionSecurityScheme(connectionId)))
        .setDescription("TripAdvisor API to get travel information."));

See the full sample in OpenApiWithConnectionSync.java.


Toolbox Tools

Toolbox tools are defined in toolbox versions and managed through ToolboxesClient / ToolboxesAsyncClient. Toolbox versions use ToolboxTool subclasses rather than agent Tool subclasses. Use invokeLatestToolboxMcp to invoke the latest toolbox version through its MCP endpoint.

Toolbox Search lets an agent search the available toolbox tools at runtime. The GA implementation is ToolSearchToolboxTool (toolbox_search), and the preview implementation ToolboxSearchPreviewToolboxTool (toolbox_search_preview) is maintained alongside it for backward compatibility.


ToolSearchToolboxTool toolboxSearchTool = new ToolSearchToolboxTool()
    .setName("search_tools")
    .setDescription("Search over available toolbox tools at runtime.");

ToolboxVersionDetails version = toolboxesClient.createToolboxVersion(
    toolboxName,
    Collections.singletonList(toolboxSearchTool),
    "Toolbox version with a Toolbox Search tool.",
    null,
    null,
    null);

System.out.printf("Created toolbox: %s%n", version.getName());
System.out.printf("Toolbox version: %s%n", version.getVersion());
for (ToolboxTool tool : version.getTools()) {
    System.out.printf("Tool type: %s%n", tool.getType());
}

See the full sample in ToolboxSearchToolboxSample.java.

Reminder (preview)

The Reminder tool lets a hosted agent schedule itself to run again at a future time. It is connectionless and is available only to hosted agents, not prompt agents.


ReminderPreviewToolboxTool reminderTool = new ReminderPreviewToolboxTool()
    .setName("schedule_reminder")
    .setDescription("Schedule a reminder that re-invokes this agent at a future time.");

ToolboxVersionDetails version = toolboxesClient.createToolboxVersion(
    toolboxName,
    Collections.<ToolboxTool>singletonList(reminderTool),
    "Built-in reminder tool for a self-scheduling agent.",
    null,
    null,
    null);

System.out.printf("Created toolbox: %s%n", version.getName());
System.out.printf("Toolbox version: %s%n", version.getVersion());
System.out.printf("Tool type: %s%n", version.getTools().get(0).getType());

See the full samples in ReminderPreviewToolboxSample.java and ReminderPreviewToolboxAsyncSample.java.

Shell

The Shell toolbox tool runs commands in an isolated container. This example uses an automatically provisioned container, which has outbound network access disabled by default. A prompt agent consumes the toolbox through its versioned MCP endpoint.


ShellToolboxTool shellTool = new ShellToolboxTool(new ToolboxShellContainerAutoEnvironment())
    .setDescription("Runs shell commands in a sandboxed container.");

ToolboxVersionDetails toolboxVersion = toolboxesClient.createToolboxVersion(
    toolboxName,
    Collections.<ToolboxTool>singletonList(shellTool),
    "Toolbox with a shell tool running in an auto-provisioned container.",
    null,
    null,
    null);

See the full end-to-end sample in ShellToolboxSample.java.


Streaming responses

An agent-scoped OpenAI client can stream response events as they arrive instead of waiting for the complete response. This is useful for displaying text in real time and observing tool execution progress.

Synchronous streaming

The OpenAI SDK's synchronous createStreaming method returns a StreamResponse<ResponseStreamEvent>. Close it with try-with-resources, and use ResponseAccumulator to collect the events into a final Response:

// Use ResponseAccumulator to collect streamed events into a final Response
ResponseAccumulator responseAccumulator = ResponseAccumulator.create();

// Stream response - text is printed as it arrives
try (StreamResponse<ResponseStreamEvent> events = openAIClient.responses().createStreaming(
        ResponseCreateParams.builder()
            .input("Tell me a short story about a brave explorer.")
            .build())) {

    events.stream().forEach(event -> {
        responseAccumulator.accumulate(event);
        event.outputTextDelta()
            .ifPresent(textEvent -> System.out.print(textEvent.delta()));
    });
}
System.out.println(); // newline after streamed text

// Access the complete accumulated response
Response response = responseAccumulator.response();
System.out.println("\nResponse ID: " + response.id());

See the full samples in SimpleStreamingSync.java, FunctionCallStreamingSync.java, and CodeInterpreterStreamingSync.java.

Asynchronous streaming

The OpenAI SDK's asynchronous createStreaming method returns an AsyncStreamResponse<ResponseStreamEvent>. Use StreamingResponseUtils.toFlux to adapt it to a Reactor Flux and manage the underlying stream lifecycle:

// Adapt OpenAI streaming events to a Reactor Flux.
Mono<Void> streamingCompletion = Mono.defer(() -> {
    ResponseAccumulator responseAccumulator = ResponseAccumulator.create();
    AsyncStreamResponse<ResponseStreamEvent> stream = openAIAsyncClient.responses().createStreaming(
        ResponseCreateParams.builder()
            .input("Tell me a short story about a brave explorer.")
            .build());

    return StreamingResponseUtils.toFlux(stream)
        .doOnNext(event -> responseAccumulator.accumulate(event)
            .outputTextDelta()
            .ifPresent(textEvent -> System.out.print(textEvent.delta())))
        .then()
        .doOnSuccess(unused -> {
            System.out.println(); // newline after streamed text

            // Access the complete accumulated response
            Response response = responseAccumulator.response();
            System.out.println("\nResponse ID: " + response.id());
        });
});

See the full samples in SimpleStreamingAsync.java, FunctionCallStreamingAsync.java, and CodeInterpreterStreamingAsync.java.

Stream hosted agent session logs

Hosted agent session logs can be streamed as SessionLogEvent values after a session has been created. The data property contains the log payload as an opaque string.

Session log streams are long-lived and may remain open until the client cancels or the session ends. Bound the stream with take, timeout, by disposing the subscription, or by breaking iteration when appropriate.

Synchronous session log streaming

The synchronous session log method returns IterableStream<SessionLogEvent>, which can be consumed with a standard for-each loop:

IterableStream<SessionLogEvent> sessionLogs =
    agentsClient.getSessionLogStream(agentName, agentVersion, sessionId);

int logsRead = 0;
for (SessionLogEvent event : sessionLogs) {
    System.out.printf("[%s] %s%n", event.getEvent(), event.getData());

    // Session log streams are long-lived; connection is closed on client disconnection
    if (++logsRead == 100) {
        break;
    }
}

Asynchronous session log streaming

The asynchronous session log method returns Flux<SessionLogEvent>, integrating naturally with Reactor pipelines:

agentsAsyncClient.getSessionLogStream(agentName, agentVersion, sessionId)
    .take(100)
    .doOnNext(event -> System.out.printf("[%s] %s%n", event.getEvent(), event.getData()))
    .blockLast();

Structured inputs

Structured inputs allow you to define named parameters on an agent that get substituted into its prompt template at runtime. This is useful when you want the same agent definition to handle different users or contexts by simply changing the input values.

Define structured inputs on an agent

When creating the agent, declare each structured input with a description and whether it is required. Use {{inputName}} placeholders in the instructions to reference them:

// Create an agent with structured input definitions
Map<String, StructuredInputDefinition> structuredInputDefinitions = new LinkedHashMap<>();
structuredInputDefinitions.put("userName",
    new StructuredInputDefinition().setDescription("User's name").setRequired(true));
structuredInputDefinitions.put("userRole",
    new StructuredInputDefinition().setDescription("User's role").setRequired(true));

AgentVersionDetails agent = agentsClient.createAgentVersion("structured-input-agent",
    new PromptAgentDefinition(model)
        .setInstructions("You are a helpful assistant. "
            + "The user's name is {{userName}} and their role is {{userRole}}. "
            + "Greet them and confirm their details.")
        .setStructuredInputs(structuredInputDefinitions));

Create a response with structured input values

When creating a response, pass a Map<String, BinaryData> whose keys match the structured input names declared on the agent. The values are substituted into the prompt template before the model processes the request:

// Build the structured input values that match the agent's definitions
Map<String, BinaryData> structuredInputValues = new LinkedHashMap<>();
structuredInputValues.put("userName", BinaryData.fromObject("Alice Smith"));
structuredInputValues.put("userRole", BinaryData.fromObject("Senior Developer"));

// Create a response using AzureCreateResponse, which flattens agent_reference
// and structured_inputs as top-level properties in the request body
Response response = responsesClient.createAzureResponse(
    new AzureCreateResponseOptions()
        .setAgentReference(new AgentReference(agent.getName()).setVersion(agent.getVersion()))
        .setStructuredInputs(structuredInputValues),
    ResponseCreateParams.builder().input("Hello! Can you confirm my details?")
);

Streaming is also supported via createStreamingAzureResponse, which returns an IterableStream<ResponseStreamEvent> (sync) or Flux<ResponseStreamEvent> (async).

See the full sample in CreateResponseWithStructuredInput.java.


Voice agent samples

The following voice-agent samples cover agent management and persisted conversations.

Scenario Samples
Lifecycle VoiceAgentBasicSample.java and VoiceAgentBasicAsyncSample.java create, retrieve, update, list, enable, disable, and delete voice agents.
Versions and drafts VoiceAgentVersionsSample.java creates and lists released and draft versions.
Guided generation VoiceAgentGenerateSample.java generates and creates a voice agent from high-level input.
Audio and tools VoiceAgentWithToolsSample.java configures PCM audio, transcription, voice activity detection, function tools, and system tools.
Persisted conversations VoiceAgentReadConversationSample.java reads responses and transcripts, while VoiceAgentReadConversationAudioSample.java downloads call and item audio.

Authenticate with DefaultAzureCredential. Every voice sample requires FOUNDRY_PROJECT_ENDPOINT. Samples that create explicit definitions optionally use FOUNDRY_VOICE_MODEL, FOUNDRY_VOICE_MODEL_TYPE, and FOUNDRY_VOICE_AGENT_NAME. The persisted-conversation samples require FOUNDRY_VOICE_AGENT_NAME and FOUNDRY_VOICE_CONVERSATION_ID.

Realtime voice-agent WebSocket examples

Realtime WebSocket sessions provide bidirectional text and audio communication with a voice agent. Create the voice agent before opening a session; the lifecycle samples above demonstrate how to create one.

Create a realtime WebSocket client

Build a synchronous or asynchronous realtime client from AgentsClientBuilder. Realtime handshakes omit the preview feature header by default; set allowPreview(true) before building the client to opt into voice-agent preview features.

AgentsClientBuilder builder = new AgentsClientBuilder()
    .credential(new DefaultAzureCredentialBuilder().build())
    .endpoint(endpoint);

VoiceAgentsWebSocketClient realtimeClient
    = builder.buildVoiceAgentsWebSocketClient();
VoiceAgentsWebSocketAsyncClient realtimeAsyncClient
    = builder.buildVoiceAgentsWebSocketAsyncClient();

Send a synchronous text turn

Connections require an https:// or wss:// project endpoint. Insecure endpoints are rejected before acquiring a token. This also applies to localhost; use certificate-verified TLS for local servers.

Unknown server event types are returned as RawRealtimeServerEvent; getRawEvent() preserves the complete JSON object. Use sendEvent(BinaryData) to send raw JSON objects, including event types or fields not modeled by this SDK. Sessions receive UTF-8 JSON in text or binary WebSocket messages.

VoiceAgentWebSocketConnectionOptions is copied when a connection is opened. Configure it before connecting; later changes do not affect the active session:

  • setReceiveBufferCapacity sets a bounded event queue (default 256, range 1-65536).
  • setOverflowStrategy defaults to ERROR, which closes an overflowing connection. DROP_OLDEST and DROP_LATEST explicitly permit data loss and should only be used when the application can tolerate missing events.
  • setMaxMessageSize limits accepted message bytes (default 32 MiB). Oversized messages terminate the connection. The sync transport checks size after receiving a complete message; this does not bound the transport's allocation.
  • Malformed JSON or invalid UTF-8 terminates reception by default. Set setMalformedEventHandler to report and skip malformed events while continuing reception. This callback must not block; throwing from it terminates the session.
VoiceAgentWebSocketConnectionOptions options
    = new VoiceAgentWebSocketConnectionOptions()
        .setReceiveBufferCapacity(512)
        .setMaxMessageSize(8 * 1024 * 1024)
        .setOverflowStrategy(VoiceAgentWebSocketOverflowStrategy.ERROR);
try (VoiceAgentsWebSocketSessionClient session = realtimeClient.openWebSocketSession(agentName, options)) {
    session.sendEvent(BinaryData.fromString(
        "{\"type\":\"response.create\",\"event_id\":\"response-1\"}"));
    for (RealtimeServerEvent event : session.receiveEvents()) {
        if (event instanceof RawRealtimeServerEvent) {
            BinaryData payload
                = ((RawRealtimeServerEvent) event).getRawEvent();
            System.out.println("Received an unrecognized event with " + payload.getLength() + " bytes.");
        }
    }
}

Connect to the voice agent, add the user's text to the conversation, and request a response. Consume the typed server events until the response finishes. A session supports only one consumer of receiveEvents().

For bounded synchronous waits, use receiveEvents(Duration) with a positive per-event timeout. A timeout raises IllegalStateException with a TimeoutException cause, leaves the session open, and allows the same iterator to retry. Use close(code, reason) or asynchronous closeAsync(code, reason) to send a custom close frame. Close reasons must fit in 123 UTF-8 bytes and close codes must be valid WebSocket codes. The first asynchronous close request wins.

try (VoiceAgentsWebSocketSessionClient session = realtimeClient.openWebSocketSession(agentName)) {
    session.sendText("Hello! Tell me about the services you provide.");
    session.createResponse();

    for (RealtimeServerEvent event : session.receiveEvents()) {
        if (event instanceof RealtimeResponseTextDeltaEvent) {
            System.out.print(((RealtimeResponseTextDeltaEvent) event).getDelta());
        } else if (event instanceof RealtimeErrorEvent) {
            RealtimeErrorEvent error = (RealtimeErrorEvent) event;
            System.out.println("Session error: " + error.getError().message());
        } else if (event instanceof RealtimeResponseDoneEvent) {
            break;
        }
    }
}

Use sendText and createResponse again for subsequent turns while the session remains open. Call cancelResponse to interrupt an active response.

Send an asynchronous text turn

The asynchronous client returns a Mono when connecting and a Flux<RealtimeServerEvent> when receiving events. Mono.usingWhen closes the session on completion, error, or cancellation.

Mono.usingWhen(
    realtimeAsyncClient.openWebSocketSession(agentName),
    session -> session.sendText("Hello! Tell me about the services you provide.")
        .then(session.createResponse())
        .thenMany(session.receiveEvents())
        .doOnNext(event -> {
            if (event instanceof RealtimeResponseTextDeltaEvent) {
                System.out.print(((RealtimeResponseTextDeltaEvent) event).getDelta());
            }
        })
        .takeUntil(event -> event instanceof RealtimeResponseDoneEvent)
        .then(),
    VoiceAgentsWebSocketSessionAsyncClient::closeAsync,
    (session, error) -> session.closeAsync(),
    VoiceAgentsWebSocketSessionAsyncClient::closeAsync)
    .block();

Stream audio and handle function tools

Use appendInputAudio to send bytes in the input format configured on the voice agent (the included live audio sample uses PCM16). Use commitInputAudio to commit buffered audio when server-side voice activity detection is not configured, and clearInputAudio to discard pending input. Audio output arrives through RealtimeResponseAudioDeltaEvent events. When a RealtimeResponseFunctionCallArgumentsDoneEvent requests a client-side tool, execute the function and call sendFunctionCallOutput with its call ID and serialized result; the helper also requests the next response.

Scenario Complete sample
Synchronous live text VoiceAgentLiveTextConversationSample.java
Asynchronous live text VoiceAgentLiveTextConversationAsyncSample.java
Asynchronous live audio VoiceAgentLiveAudioConversationAsyncSample.java
Live function tool VoiceAgentLiveFunctionToolSample.java

All realtime examples require FOUNDRY_PROJECT_ENDPOINT and optionally use FOUNDRY_VOICE_AGENT_NAME. The function-tool example also optionally uses FOUNDRY_VOICE_MODEL and FOUNDRY_VOICE_MODEL_TYPE. The asynchronous text and audio examples delete their generated agents by default; set FOUNDRY_KEEP_VOICE_AGENT=true to retain them.

The synchronous live text sample deletes and replaces the selected agent if creation returns a conflict, then deletes the replacement on exit, including on failure. Use a dedicated sample-agent name: this also applies when FOUNDRY_VOICE_AGENT_NAME is set. This synchronous sample does not support retaining its agent.

The live audio example requires a Java Sound-compatible microphone and speaker. It streams signed, little-endian, mono PCM16 audio at 24 kHz. These examples use WebSocket transport. Although the generated protocol models include WebRTC signaling events, the Java client does not provide a WebRTC peer connection or media implementation.

GPT Live WebSocket examples (preview)

GPT Live is a different wire protocol from Realtime. Create a persisted agent with VoiceAgentLiveDefinition, then use BetaVoiceAgentsLiveWebSocketClient or BetaVoiceAgentsLiveWebSocketAsyncClient to connect. These clients use wss://{project-endpoint}/agents/{agent_name}/endpoint/protocols/voice/live/sessions?api-version=v1, send the Foundry-Features: VoiceAgents=V1Preview header, and do not negotiate the realtime WebSocket subprotocol. allowPreview(true) is not required.

Foundry sends the initial native session.start. Wait for session.started before sending application events; do not send session.start yourself. Use sendEvent(ClientEvent) and receiveEvents() with the com.openai.models.live event models from the SDK's existing openai-java dependency. These are distinct from Realtime event models. The synchronous client also offers receiveEvents(Duration) for bounded waits. The sendEvent(BinaryData) and receiveRawEvents() APIs expose complete JSON objects when the original wire representation is needed, including unknown properties, nested response.event backend events, and the Foundry conversation_id on session.started. OpenAI typed models retain extra properties and unknown event variants, but typed serialization need not preserve the original JSON formatting. The existing samples named VoiceAgentLive*ConversationSample above demonstrate live connections using the Realtime protocol; use the separate GPT Live clients for VoiceAgentLiveDefinition agents.

For example, configure and open a synchronous Live session for an agent using Responses delegation:

BetaVoiceAgentsLiveWebSocketClient liveClient = builder.beta().buildBetaVoiceAgentsLiveWebSocketClient();
VoiceAgentLiveWebSocketConnectionOptions options = new VoiceAgentLiveWebSocketConnectionOptions()
    .setReceiveBufferCapacity(512)
    .setMaxMessageSize(8 * 1024 * 1024)
    .setOverflowStrategy(VoiceAgentWebSocketOverflowStrategy.ERROR)
    .setHandshakeTimeout(Duration.ofSeconds(30))
    .setCloseTimeout(Duration.ofSeconds(10))
    .setStoreEnabled(false);

try (BetaVoiceAgentsLiveWebSocketSessionClient session = liveClient.openWebSocketSession(agentName, options)) {
    for (ServerEvent event : session.receiveEvents()) {
        if (event.isSessionStarted()) {
            session.sendEvent(ClientEvent.ofResponseCreate(ResponseCreateEvent.builder().build()));
        } else if (event.isSessionOutputTranscriptDelta()) {
            System.out.print(event.asSessionOutputTranscriptDelta().delta());
        } else if (event.isSessionClosed() || event.isErrorEvent()) {
            break;
        }
    }
}

Send only native Live commands valid for the configured delegation mode, whether typed or raw. In particular, response.item.create and response.create are Responses-delegation-only commands. Service error events arrive as events for the application to inspect. Typed receivers decode JSON and fail on malformed events unless a malformed-event callback was explicitly configured. Raw receivers do not parse JSON or construct typed models; transport size limits and UTF-8 validation still apply.

Configure timeouts, bounded receive capacity, overflow handling, message size, persistence, agent version, and structured inputs with VoiceAgentLiveWebSocketConnectionOptions. Structured inputs are passed as unencoded JSON to setStructuredInputs and URL-encoded by the SDK into the structured_inputs query parameter. Do not log endpoint URLs containing sensitive template inputs. Realtime-only transport and agent-session-ID options are not exposed. Options are copied when opening a session.

Both transports retain the realtime clients' credential, proxy, custom-header and builder validation behavior. Custom HTTP clients, pipelines, policies, and retry settings are unsupported and fail at build time rather than being ignored. Each session permits one receiver, shared by typed and raw receive APIs. The default receive buffer holds 256 events; overflow terminates the session unless a drop strategy explicitly permits event loss.

Runnable GPT Live scenarios

These examples create a uniquely named VoiceAgentLiveDefinition agent and connect using native GPT Live WebSockets. Most use Responses delegation and delete their generated agent on exit, including on failure. The connection lifecycle sample uses client delegation and deletes only the agent version it created. None modifies or deletes an existing agent. Set these environment variables before running a sample's main method:

Variable Value
FOUNDRY_PROJECT_ENDPOINT Your Foundry project endpoint. Authentication uses DefaultAzureCredential.
FOUNDRY_GPT_LIVE_MODEL Your GPT Live deployment name, not a model catalog ID. Required by the connection lifecycle sample; optional for other samples, which default to gpt-live-1.
FOUNDRY_GPT_LIVE_BACKEND_MODEL Optional. Your Responses backend deployment name in the Live resource. Defaults to gpt-5.6-terra when unset.
FOUNDRY_GPT_LIVE_STORE Optional, text and microphone/speaker examples. Set to true to persist conversation content and audio. Defaults to false.
FOUNDRY_KEEP_GPT_LIVE_AGENT Optional. Set to true to retain the sample-created agent; defaults to false.
FOUNDRY_VOICE_AGENT_NAME Optional prefix used by the connection lifecycle sample. Defaults to sample-live-connection.

Each sample reads, defaults, and validates these settings and configures its client builder directly in its own main method. Shared helpers receive explicit values rather than reading environment variables. Override the model variables if your deployment names differ from these defaults. Explicitly blank values are invalid. Generated agent names use gpt-live-<scenario>-<UUID> and fit the service's 63-character limit.

Scenario Runnable sample
Synchronous multi-turn text GptLiveTextConversationSample.java
Asynchronous multi-turn text GptLiveTextConversationAsyncSample.java
Asynchronous connection lifecycle (no audio devices) GptLiveConnectionAsyncSample.java
Asynchronous microphone and speaker GptLiveAudioConversationAsyncSample.java
Responses backend function tool GptLiveFunctionToolSample.java
Advanced synchronous console input and speaker GptLiveInteractiveTextConversationSample.java
Advanced asynchronous console input and speaker GptLiveInteractiveTextConversationAsyncSample.java

Read persisted GPT Live conversations and audio

These REST examples read existing persisted data; they do not start a WebSocket session, create an agent, or delete resources.

Scenario Runnable sample
Synchronous history and transcripts GptLiveReadConversationSample.java
Asynchronous history and transcripts GptLiveReadConversationAsyncSample.java
Synchronous stored audio download GptLiveReadConversationAudioSample.java
Asynchronous stored audio download GptLiveReadConversationAudioAsyncSample.java

Use an agent/session configured with store=true and retain the agent. With either text WebSocket example or the microphone/speaker example, set both FOUNDRY_GPT_LIVE_STORE=true and FOUNDRY_KEEP_GPT_LIVE_AGENT=true. The first sets the session persistence override; the second prevents sample cleanup from deleting the agent. Keeping the agent alone does not enable persistence. The text, advanced interactive, and microphone examples print the Foundry conversation_id from session.started when provided. If absent, startup output explicitly reports that no persisted conversation ID was provided; it never substitutes the native session ID. Use session.started.conversation_id, not the native session ID. Set:

Variable Value
FOUNDRY_PROJECT_ENDPOINT Required. Your Foundry project endpoint; authentication uses DefaultAzureCredential. No default.
FOUNDRY_GPT_LIVE_AGENT_NAME Required. The existing Live agent's name. No default.
FOUNDRY_GPT_LIVE_CONVERSATION_ID Required. The persisted Foundry conversation ID. No default.
FOUNDRY_GPT_LIVE_ITEM_ID Optional, audio examples only: a speech item's ID, not a delegation item. Defaults to unset (merged audio only); explicitly blank values are invalid.

Model deployment settings such as FOUNDRY_GPT_LIVE_MODEL and FOUNDRY_GPT_LIVE_BACKEND_MODEL are not needed: these samples read existing conversations rather than creating agents. Run a history sample's main method with no arguments. For audio, pass one existing local output directory; there is no default output directory. Audio examples create uniquely named WAV files without overwriting existing files. History examples explicitly advance the after cursor until an empty page because the generated list clients do not expose automatic continuation; missing or repeated cursors fail explicitly. They show conversation status and usage seconds, input/output transcripts, and delegation targets/response IDs. Stored speech segments are not native turns or proof that audio was played. Treat the printed transcripts and downloaded audio as sensitive data.

Wait for session completion and recording finalization before downloading. HTTP errors propagate: 404 can mean missing audio; 409 can mean recording_not_ready or recording_unavailable. Inspect Retry-After before retrying. BYOS metadata is reported without calling Foundry content endpoints; use the metadata's blob URI and your own Azure Storage credentials to download BYOS audio. The current generated Live REST builder methods require explicit preview opt-in, so these samples add Foundry-Features: VoiceAgents=V1Preview using an HTTP pipeline header policy.

Using the text WebSocket samples

The primary synchronous and asynchronous text examples show agent and Responses-delegation configuration, connection, typed sends, event handling, and graceful closure directly. Each command-line argument is a prompt; with no arguments, they send What can you help me with? and exit after the reply. They wait for session.started, send response.item.create followed by response.create, and wait for the nested response.completed before advancing to the next turn. They print backend text without console input, speaker playback, microphone capture, or silent audio uploads.

Using the advanced interactive WebSocket samples

The separate synchronous and asynchronous interactive examples read terminal input. After each reply, they prompt for another message; submit a blank line or enter exit or quit to close the session. Each example owns its conversation flow; shared helpers provide audio-device plumbing and transcript rendering. Both request a spoken opening greeting with session.commentary.append, play the voice frontend's 24-kHz PCM16 audio, and print its spoken transcript rather than backend working notes. The synchronous example uses synchronous session send/receive calls, with workers for receiving events and reading console input. Before displaying the first You: prompt, each waits for both greeting audio and transcript, one second without speech activity, and completion of queued speaker playback. Continuous silent audio does not restart the timer. For this heuristic, PCM16 frames with peak sample magnitude at most 256 (about -42 dBFS) are treated as silence. The quiet period is a heuristic because native GPT Live has no speech-completed event: a long pause within a greeting can be mistaken for its end. Missing greeting output fails with a timeout instead of silently starting input before the introduction. It sends 20 milliseconds of silent PCM (960 bytes) every 20 milliseconds after startup, providing continuous audio input without opening a microphone. This consumes voice-session usage while you type or wait. The silent stream is stopped and pending audio sends complete before the sample requests session.close. If sending falls behind, silent-frame production waits for demand instead of accumulating timer ticks or failing with an interval overflow; it resumes without a burst of stale frames. Speaker playback runs off the WebSocket receive thread. If a speaker is unavailable, the sample reports that and still prints transcripts, like the Realtime text example. Agent creation and deletion occur on the main thread.

Console input is line-oriented: press Enter to submit a message. The sample flushes the You: prompt immediately; character echo while typing is controlled by Windows CMD/PowerShell and its terminal host, not by the service. Interactive input uses a normal blocking line read on a worker rather than polling Reader.ready(), matching the Realtime sample's console interaction. The WebSocket receiver continues independently. When a native Java console is attached, input comes from System.console().reader() rather than System.in. Without a console, the sample reads System.in, supporting redirected input. End-of-input (EOF), a blank line, or an explicit exit command closes the session. The terminal echoes characters on the You: line as you type. Assistant transcript fragments are assembled into utterances independently of typed turns and backend completion. A one-second speech-inactivity period marks an utterance boundary using the same heuristic as the greeting; a long pause within speech can still divide the text. Completed utterances are printed with one Assistant: label each. While input is active, completed text waits until Enter so it cannot overwrite typed characters; incomplete text stays buffered across input submissions. Audio playback and input remain available. This dependency-free approach can delay captions until the next Enter. Session closure flushes any remaining transcript. Completed and incomplete text share a 65,536-character buffer limit; exceeding it fails the sample explicitly instead of dropping transcript text or growing memory indefinitely. The native console's echoed Enter already starts a new line; the sample avoids adding another blank line. After a remote disconnect, a console read that the operating system cannot interrupt may require Enter to release the worker. The sample does not close shared standard input to force a pending read to return. Backend response.completed permits the next typed prompt but does not mean that the frontend finished speaking: native GPT Live has no output-audio-done event. The sample keeps receiving audio and transcripts while you type. See GPT-Live audio and transcript events. Server remote_hangup closes the conversation normally. Other unexpected closures remain failures and include the complete session.closed event, including its reason, rather than hiding the service diagnostic.

Using the microphone and tool samples

The audio example requires a Java Sound-compatible microphone and speaker supporting raw, mono PCM16LE at 24 kHz. After session.started, it sends a greeting request followed by response.create so the agent can open the conversation. It labels spoken transcript fragments as You: or Assistant: and omits backend text deltas, which are not necessarily the words spoken by the voice frontend. Press Enter to end the call; asking the model to end it does not itself close the session. Service moderation (content_filter) ends the call with an explicit error and the original service event; it is not treated as a successful completion. Audio devices and the WebSocket are released on failure. The standalone audio and text samples' main methods also shut down shared Reactor schedulers on exit to avoid lingering timer threads under exec:java. Do not invoke those main methods inside an application that shares Reactor schedulers.

The audio sample sends session.input_audio.append, plays base64-decoded session.output_audio.delta chunks, and prints native input/output transcript deltas. It does not send Realtime audio-buffer commit or response-cancel commands. Capture and playback use dedicated threads; the playback backlog is bounded to 60 seconds. Device errors and overflow end the call explicitly. Press Enter to stop capture and finalize the session. This example must be the only reader of standard input.

The function-tool example configures get_weather on the Responses backend, consumes completed function_call items from nested response.output_item.done events, then sends function_call_output items with the original call_id before requesting continuation. Its weather results are simulated demonstration data, not live weather. It does not assume lifecycle snapshots contain output items, and does not add Foundry-managed-agent tool-result routing.

All WebSocket scenarios request session.close and wait for session.closed before normal transport cleanup. The asynchronous text examples apply a 60-second timeout to startup, each response, and finalization; the advanced interactive example waits for terminal input without a timeout. The synchronous text and tool examples use a 60-second event-wait timeout; the microphone example uses a 60-second startup/finalization timeout. An unexpected disconnect or native service error fails the sample rather than being reported as success.

Service API versions

The client library targets the latest service API version by default. The service client builder accepts an optional service API version parameter to specify which API version to communicate.

Select a service API version

You have the flexibility to explicitly select a supported service API version when initializing a service client via the service client builder. This ensures that the client can communicate with services using the specified API version.

When selecting an API version, it is important to verify that there are no breaking changes compared to the latest API version. If there are significant differences, API calls may fail due to incompatibility.

Always ensure that the chosen API version is fully supported and operational for your specific use case and that it aligns with the service's versioning policy.

Tracing

azure-ai-agents emits experimental OpenTelemetry GenAI spans and metrics for agent creation and response (chat / invoke_agent) operations. To use this experimental feature, set AZURE_EXPERIMENTAL_ENABLE_GENAI_TRACING=true and configure an OpenTelemetry implementation for the client through ClientOptions or globally. Customer-controlled prompts, responses, messages, instructions, function arguments, tool results, and equivalent agent content are omitted unless you also set OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=true. Review your privacy requirements before enabling content recording. The former AZURE_TRACING_GEN_AI_CONTENT_RECORDING_ENABLED setting remains available as a compatibility fallback when the OpenTelemetry setting is absent.

Typed and raw-response methods are covered, including synchronous and asynchronous streaming. Streaming spans remain open until the stream completes, fails, is cancelled, or is explicitly closed.

// Set AZURE_EXPERIMENTAL_ENABLE_GENAI_TRACING=true before running this sample.
// Configure OpenTelemetry exporters and sampling, then pass the SDK through ClientOptions.
OpenTelemetrySdk openTelemetry = OpenTelemetrySdk.builder().build();
AgentsClient agentsClient = new AgentsClientBuilder().endpoint(endpoint)
    .credential(new DefaultAzureCredentialBuilder().build())
    .clientOptions(new ClientOptions()
        .setTracingOptions(new OpenTelemetryTracingOptions().setOpenTelemetry(openTelemetry)))
    .buildAgentsClient();

AgentDefinition definition
    = new PromptAgentDefinition("gpt-4o").setInstructions("You are a helpful assistant.");
AgentVersionDetails agent = agentsClient.createAgentVersion("my-agent", definition);
System.out.printf("Created agent %s (traced).%n", agent.getName());

See TracingSample.java for a runnable example. To inspect exported traces in the Foundry tracing UI, connect the Foundry project to an Application Insights resource and configure an Azure Monitor OpenTelemetry exporter. See TracingAzureMonitorSample.java for the exporter setup.

Troubleshooting

Enable client logging

You can set the AZURE_LOG_LEVEL environment variable to view logging statements made in the client library. For example, setting AZURE_LOG_LEVEL=2 would show all informational, warning, and error log messages. The log levels can be found here: log levels.

To log full HTTP request and response bodies (including headers), set:

export AZURE_LOG_LEVEL=verbose
export AZURE_HTTP_LOG_DETAIL_LEVEL=body_and_headers

Default HTTP Client

All client libraries by default use the Netty HTTP client. Configuring or changing the HTTP client is detailed in the HTTP clients wiki.

Default SSL library

All client libraries, by default, use the Tomcat-native Boring SSL library to enable native-level performance for SSL operations. The Boring SSL library is an uber jar containing native libraries for Linux / macOS / Windows, and provides better performance compared to the default SSL implementation within the JDK. For more information, including how to reduce the dependency size, refer to the performance tuning section of the wiki.

Next steps

Contributing

For details on contributing to this repository, see the contributing guide.

  1. Fork it
  2. Create your feature branch (git checkout -b my-new-feature)
  3. Commit your changes (git commit -am 'Add some feature')
  4. Push to the branch (git push origin my-new-feature)
  5. Create new Pull Request