VoiceAgentDefinition interface
The voice agent definition. Its configuration (model, instructions, audio, tools, and optional avatar) drives a
managed speech-to-speech experience. Establish realtime voice sessions through
GET /agents/{agent_name}/endpoint/protocols/voice. Every create or update produces a new immutable version.
- Extends
Properties
| audio | The audio configuration, including input and output formats, voice, turn detection, noise reduction, and transcription. These values are session defaults; a client may override supported fields when connecting. |
| avatar | Optional avatar configuration. These values are session defaults and may be overridden when connecting. |
| conversation_engine | The engine that owns conversation handling for this voice agent. Exactly one of this property and the model-backed configuration ( |
| greeting | Optional session-start greeting. Template mode speaks exact rendered text; LLM-generated mode asks the session model to author the opening response and may use configured tools. |
| include | Additional fields to include in service outputs. |
| instructions | A system (or developer) message inserted into the model's context. Supports template substitution via |
| interim_response | Interim-response settings for latency and tool execution. |
| kind | The kind discriminator for a voice agent definition. Always |
| max_output_tokens | The maximum output-token count for one response. |
| model | The model to use for this agent. Required with |
| model_type | How the model backing this voice agent is served. Required with |
| output_modalities | The output modalities the agent produces. Defaults to |
| parallel_tool_calls | Whether the model may call multiple tools in parallel. |
| soundscape | Optional background ambience and event cues, stored with the agent version and separate from the audio configuration. Currently rendered only for telephony calls, not browser WebRTC or direct voice WebSocket sessions. Omit this property to disable all auxiliary audio. Persisted conversation recordings do not include the soundscape mix. |
| store | Whether conversations with this agent are persisted. A single, all-or-nothing persistence switch that defaults to
|
| structured_inputs | Set of structured inputs that participate in prompt template substitution, rendered per session before the live session starts. |
| subagent_config | Optional configuration for sibling Foundry text agents that this voice agent may consult as background specialists. |
| tools | The tools the voice agent may use. Supported tool kinds are |
| tool_choice | How the model chooses tools for generated responses. |
Inherited Properties
| rai_config | Configuration for Responsible AI (RAI) content filtering and safety features. |
Property Details
audio
The audio configuration, including input and output formats, voice, turn detection, noise reduction, and transcription. These values are session defaults; a client may override supported fields when connecting.
audio?: VoiceAgentAudioConfig
Property Value
avatar
Optional avatar configuration. These values are session defaults and may be overridden when connecting.
avatar?: VoiceAgentAvatarConfig
Property Value
conversation_engine
The engine that owns conversation handling for this voice agent. Exactly one of this property and the model-backed configuration (model_type with model) must be provided. When this property is provided, model_type, model, instructions, tools, and tool_choice must be omitted, and greeting.tool_choice cannot be required, because the engine owns the conversation logic. The initial implementation supports a hosted-agent engine.
conversation_engine?: VoiceConversationEngineUnion
Property Value
greeting
Optional session-start greeting. Template mode speaks exact rendered text; LLM-generated mode asks the session model to author the opening response and may use configured tools.
greeting?: VoiceAgentGreetingConfigUnion
Property Value
include
Additional fields to include in service outputs.
include?: VoiceAgentSessionIncludeOption[]
Property Value
instructions
A system (or developer) message inserted into the model's context. Supports template substitution via structured_inputs, rendered per session before the live session starts.
instructions?: string
Property Value
string
interim_response
Interim-response settings for latency and tool execution.
interim_response?: VoiceAgentInterimResponseConfigUnion
Property Value
kind
The kind discriminator for a voice agent definition. Always voice.
kind: "voice"
Property Value
"voice"
max_output_tokens
The maximum output-token count for one response.
max_output_tokens?: VoiceAgentMaxOutputTokens
Property Value
model
The model to use for this agent. Required with model_type for a model-backed voice agent and omitted when conversation_engine is provided. The model must support realtime or cascaded voice.
model?: string
Property Value
string
model_type
How the model backing this voice agent is served. Required with model for a model-backed voice agent and omitted when conversation_engine is provided. This is independent of the architecture (realtime or cascaded), which the service derives from the selected model.
model_type?: VoiceModelType
Property Value
output_modalities
The output modalities the agent produces. Defaults to ["audio"]. animation and avatar are available
when an avatar is configured.
output_modalities?: VoiceOutputModality[]
Property Value
parallel_tool_calls
Whether the model may call multiple tools in parallel.
parallel_tool_calls?: boolean
Property Value
boolean
soundscape
Optional background ambience and event cues, stored with the agent version and separate from the audio configuration. Currently rendered only for telephony calls, not browser WebRTC or direct voice WebSocket sessions. Omit this property to disable all auxiliary audio. Persisted conversation recordings do not include the soundscape mix.
soundscape?: VoiceAgentSoundscapeConfig
Property Value
store
Whether conversations with this agent are persisted. A single, all-or-nothing persistence switch that defaults to
false (privacy-safe: off by default). When true, Foundry persists the full conversation — the transcript/event
timeline and raw audio. When false, nothing is persisted and no conversation is surfaced. There is no separate
audio-logging control; audio is persisted only as part of this switch. Latency/performance telemetry (e.g.
time-to-first-audio, inter-token latency, interruption) is observability-only (customer trace / App Insights) and
is not part of the persisted conversation content.
store?: boolean
Property Value
boolean
structured_inputs
Set of structured inputs that participate in prompt template substitution, rendered per session before the live session starts.
structured_inputs?: Record<string, StructuredInputDefinition>
Property Value
Record<string, StructuredInputDefinition>
subagent_config
Optional configuration for sibling Foundry text agents that this voice agent may consult as background specialists.
subagent_config?: VoiceAgentSubagentConfig
Property Value
tools
The tools the voice agent may use. Supported tool kinds are function (executed by the client), mcp,
system (service-managed session controls), and toolbox. Server-side tools such as web_search,
azure_ai_search, and openapi are provided through a toolbox rather than declared directly.
tools?: VoiceAgentToolUnion[]
Property Value
tool_choice
How the model chooses tools for generated responses. none prevents tool calls, auto lets the model decide,
required requires at least one tool call, and a specific function or MCP tool can be selected with an object.
Defaults to auto.
tool_choice?: VoiceAgentToolChoice
Property Value
Inherited Property Details
rai_config
Configuration for Responsible AI (RAI) content filtering and safety features.
rai_config?: RaiConfig
Property Value
Inherited From AgentDefinition.rai_config