Azure Ai Voicelive Dotnet

作者 microsoft354361d83247MIT收录于 2026年10月8日更新于 2026年10月8日

Azure AI Voice Live SDK for .NET. Build real-time voice AI applications with bidirectional WebSocket communication. Use for voice assistants, conversational AI, real-time speech-to-speech, and voice-enabled chatbots. Triggers: "voice live", "real-time voice", "VoiceLiveClient", "VoiceLiveSession", "voice assistant .NET", "bidirectional audio", "speech-to-speech".

精选仅含说明Software Development
AI 生成的概览

指导 .NET 开发者使用 Azure AI Voice Live SDK 构建实时双向语音 AI 应用。

功能
该技能说明如何使用 Azure.AI.VoiceLive .NET SDK 通过 WebSocket 构建实时语音助手。内容涵盖安装、身份验证、会话配置、事件处理、函数调用、语音选项、支持的模型和错误处理,并附有 C# 代码示例。它产出的是指导说明和示例代码,而非可运行脚本。
适用场景
适用于在 Azure AI Voice Live 上构建 .NET 语音助手、对话式 AI 或语音到语音应用的场景。适合涉及 VoiceLiveClient、VoiceLiveSession、双向音频或实时语音聊天机器人的任务。
运行要求
需要 .NET SDK 以及 Azure.AI.VoiceLive、Azure.Identity 和 NAudio 这三个 NuGet 包。需要 Azure AI Voice Live 终结点、模型部署名称,以及 API 密钥或具有 Cognitive Services User 角色的 Microsoft Entra 凭据。需要访问 Azure 的网络连接。不包含脚本。

Azure.AI.VoiceLive (.NET)

Real-time voice AI SDK for building bidirectional voice assistants with Azure AI.

Installation

bash
dotnet add package Azure.AI.VoiceLivedotnet add package Azure.Identitydotnet add package NAudio                    # For audio capture/playback

Current Versions: Stable v1.0.0, Preview v1.1.0-beta.1

Environment Variables

bash
AZURE_VOICELIVE_ENDPOINT=https://<resource>.services.ai.azure.com/  # Required: Voice Live endpointAZURE_VOICELIVE_MODEL=gpt-4o-realtime-preview  # Required: model deployment nameAZURE_VOICELIVE_VOICE=en-US-AvaNeural  # Optional: Voice Live voice nameAZURE_VOICELIVE_API_KEY=<your-api-key>  # Only required for AzureKeyCredential authAZURE_TOKEN_CREDENTIALS=prod  # Required only if DefaultAzureCredential is used in production

Authentication

Microsoft Entra Token Credential

csharp
using Azure.Identity;using Azure.AI.VoiceLive;
Uri endpoint = new Uri("https://your-resource.cognitiveservices.azure.com");// Local dev: DefaultAzureCredential. Production: set AZURE_TOKEN_CREDENTIALS=prod or AZURE_TOKEN_CREDENTIALS=<specific_credential>var credential = new DefaultAzureCredential(    DefaultAzureCredential.DefaultEnvironmentVariableName);// Or use a specific credential directly in production:// See https://learn.microsoft.com/dotnet/api/overview/azure/identity-readme?view=azure-dotnet#credential-classes// var credential = new ManagedIdentityCredential();VoiceLiveClient client = new VoiceLiveClient(endpoint, credential);

Required Role: Cognitive Services User (assign in Azure Portal → Access control)

API Key

csharp
Uri endpoint = new Uri("https://your-resource.cognitiveservices.azure.com");AzureKeyCredential credential = new AzureKeyCredential("your-api-key");VoiceLiveClient client = new VoiceLiveClient(endpoint, credential);

Client Hierarchy

VoiceLiveClient└── VoiceLiveSession (WebSocket connection)    ├── ConfigureSessionAsync()    ├── GetUpdatesAsync() → SessionUpdate events    ├── AddItemAsync() → UserMessageItem, FunctionCallOutputItem    ├── SendAudioAsync()    └── StartResponseAsync()

Core Workflow

1. Start Session and Configure

csharp
using Azure.Identity;using Azure.AI.VoiceLive;
var endpoint = new Uri(Environment.GetEnvironmentVariable("AZURE_VOICELIVE_ENDPOINT"));var client = new VoiceLiveClient(endpoint, new DefaultAzureCredential());
var model = "gpt-4o-mini-realtime-preview";
// Start sessionusing VoiceLiveSession session = await client.StartSessionAsync(model);
// Configure sessionVoiceLiveSessionOptions sessionOptions = new(){    Model = model,    Instructions = "You are a helpful AI assistant. Respond naturally.",    Voice = new AzureStandardVoice("en-US-AvaNeural"),    TurnDetection = new AzureSemanticVadTurnDetection()    {        Threshold = 0.5f,        PrefixPadding = TimeSpan.FromMilliseconds(300),        SilenceDuration = TimeSpan.FromMilliseconds(500)    },    InputAudioFormat = InputAudioFormat.Pcm16,    OutputAudioFormat = OutputAudioFormat.Pcm16};
// Set modalities (both text and audio for voice assistants)sessionOptions.Modalities.Clear();sessionOptions.Modalities.Add(InteractionModality.Text);sessionOptions.Modalities.Add(InteractionModality.Audio);
await session.ConfigureSessionAsync(sessionOptions);

2. Process Events

csharp
await foreach (SessionUpdate serverEvent in session.GetUpdatesAsync()){    switch (serverEvent)    {        case SessionUpdateResponseAudioDelta audioDelta:            byte[] audioData = audioDelta.Delta.ToArray();            // Play audio via NAudio or other audio library            break;                    case SessionUpdateResponseTextDelta textDelta:            Console.Write(textDelta.Delta);            break;                    case SessionUpdateResponseFunctionCallArgumentsDone functionCall:            // Handle function call (see Function Calling section)            break;                    case SessionUpdateError error:            Console.WriteLine($"Error: {error.Error.Message}");            break;                    case SessionUpdateResponseDone:            Console.WriteLine("\n--- Response complete ---");            break;    }}

3. Send User Message

csharp
await session.AddItemAsync(new UserMessageItem("Hello, can you help me?"));await session.StartResponseAsync();

4. Function Calling

csharp
// Define functionvar weatherFunction = new VoiceLiveFunctionDefinition("get_current_weather"){    Description = "Get the current weather for a given location",    Parameters = BinaryData.FromString("""        {            "type": "object",            "properties": {                "location": {                    "type": "string",                    "description": "The city and state or country"                }            },            "required": ["location"]        }        """)};
// Add to session optionssessionOptions.Tools.Add(weatherFunction);
// Handle function call in event loopif (serverEvent is SessionUpdateResponseFunctionCallArgumentsDone functionCall){    if (functionCall.Name == "get_current_weather")    {        var parameters = JsonSerializer.Deserialize<Dictionary<string, string>>(functionCall.Arguments);        string location = parameters?["location"] ?? "";                // Call external service        string weatherInfo = $"The weather in {location} is sunny, 75°F.";                // Send response        await session.AddItemAsync(new FunctionCallOutputItem(functionCall.CallId, weatherInfo));        await session.StartResponseAsync();    }}

Voice Options

Voice TypeClassExample
Azure StandardAzureStandardVoice"en-US-AvaNeural"
Azure HDAzureStandardVoice"en-US-Ava:DragonHDLatestNeural"
Azure CustomAzureCustomVoiceCustom voice with endpoint ID

Supported Models

ModelDescription
gpt-4o-realtime-previewGPT-4o with real-time audio
gpt-4o-mini-realtime-previewLightweight, fast interactions
phi4-mm-realtimeCost-effective multimodal

Key Types Reference

TypePurpose
VoiceLiveClientMain client for creating sessions
VoiceLiveSessionActive WebSocket session
VoiceLiveSessionOptionsSession configuration
AzureStandardVoiceStandard Azure voice provider
AzureSemanticVadTurnDetectionVoice activity detection
VoiceLiveFunctionDefinitionFunction tool definition
UserMessageItemUser text message
FunctionCallOutputItemFunction call response
SessionUpdateResponseAudioDeltaAudio chunk event
SessionUpdateResponseTextDeltaText chunk event

Best Practices

  1. Always set both modalities — Include Text and Audio for voice assistants
  2. Use AzureSemanticVadTurnDetection — Provides natural conversation flow
  3. Configure appropriate silence duration — 500ms typical to avoid premature cutoffs
  4. Use using statement — Ensures proper session disposal
  5. Handle all event types — Check for errors, audio, text, and function calls
  6. Use DefaultAzureCredential — Never hardcode API keys

Error Handling

csharp
if (serverEvent is SessionUpdateError error){    if (error.Error.Message.Contains("Cancellation failed: no active response"))    {        // Benign error, can ignore    }    else    {        Console.WriteLine($"Error: {error.Error.Message}");    }}

Audio Configuration

  • Input Format: InputAudioFormat.Pcm16 (16-bit PCM)
  • Output Format: OutputAudioFormat.Pcm16
  • Sample Rate: 24kHz recommended
  • Channels: Mono

Related SDKs

SDKPurposeInstall
Azure.AI.VoiceLiveReal-time voice (this SDK)dotnet add package Azure.AI.VoiceLive
Microsoft.CognitiveServices.SpeechSpeech-to-text, text-to-speechdotnet add package Microsoft.CognitiveServices.Speech
NAudioAudio capture/playbackdotnet add package NAudio

Reference Links

来源与署名

来源:microsoft/skills位于.github/plugins/azure-sdk-dotnet/skills/azure-ai-voicelive-dotnet提交354361d

许可证: MIT

内容归原作者所有。SourceWeft 从公开仓库中收录这些内容。

举报或申请下架