Speech Recognition

Speech Recognition

Transcribe live and pre-recorded audio to text using Apple's Speech framework. Covers SpeechAnalyzer / SpeechTranscriber (iOS 26+) and SFSpeechRecognizer (iOS 10+) fallback guidance.

Scope boundary: Use this skill for speech-to-text recognition, speech authorization, microphone capture plumbing, and result handling. Hand off text analysis, language identification after transcription, sentiment, embeddings, and translation to natural-language; hand off audio playback UI to avkit; hand off summarization or generation over transcripts to apple-on-device-ai.

Contents

• SpeechAnalyzer Strategy (iOS 26+)
• SFSpeechRecognizer Setup
• Authorization
• Live Microphone Transcription
• Pre-Recorded Audio File Recognition
• On-Device vs Server Recognition
• Handling Results
• Common Mistakes
• Review Checklist
• References

SpeechAnalyzer Strategy (iOS 26+)

Use SpeechAnalyzer for modern iOS 26+ speech analysis, especially long-form recordings, live transcription, time-indexed transcripts, and fully on-device flows. Keep SFSpeechRecognizer for iOS 10+ deployment targets, server-backed locale coverage, or existing callback/delegate implementations.

Read SpeechAnalyzer patterns [blocked] when implementing an iOS 26+ transcription pipeline, model asset handling, volatile results, or file/buffer examples.

SpeechAnalyzer setup checklist

1. Choose the module:
   • SpeechTranscriber for the newer general-purpose on-device model.
   • DictationTranscriber when SpeechTranscriber is unavailable for the current device or locale and dictation-compatible support is acceptable.
   • SpeechDetector only in conjunction with a transcriber when voice activity detection is worth the accuracy/power tradeoff.
2. Check support before creating the session:
   • SpeechTranscriber.isAvailable
   • SpeechTranscriber.supportedLocale(equivalentTo:)
   • SpeechTranscriber.installedLocales / supportedLocales when showing language choices.
3. Pick a documented preset:
   • .transcription for basic accurate transcription.
   • .progressiveTranscription for live UI updates.
   • .timeIndexedProgressiveTranscription when playback highlighting needs audioTimeRange.
4. Install required assets with AssetInventory.assetInstallationRequest.
5. Convert live audio buffers to SpeechAnalyzer.bestAvailableAudioFormat(compatibleWith:) before yielding AnalyzerInput.
6. Consume module results from their AsyncSequence in a separate task.
7. Finish explicitly with finalizeAndFinish(through:), finalizeAndFinishThroughEndOfInput(), or cancelAndFinishNow().

Do not use an offlineTranscription preset; Apple does not document one. Finishing an AsyncStream input sequence does not finish the analyzer session.

SFSpeechRecognizer Setup

Creating a recognizer with locale

import Speech
// Default locale (user's current language)let recognizer = SFSpeechRecognizer()
// Specific localelet recognizer = SFSpeechRecognizer(locale: Locale(identifier: "en-US"))
// Check if recognition is available for this localeguard let recognizer, recognizer.isAvailable else {    print("Speech recognition not available")    return}

Monitoring availability changes

final class SpeechManager: NSObject, SFSpeechRecognizerDelegate {    private let recognizer = SFSpeechRecognizer()!
    override init() {        super.init()        recognizer.delegate = self    }
    func speechRecognizer(        _ speechRecognizer: SFSpeechRecognizer,        availabilityDidChange available: Bool    ) {        // Update UI — disable record button when unavailable    }}

Authorization

Request both speech recognition and microphone permissions before starting live transcription. Add these keys to Info.plist:

• NSSpeechRecognitionUsageDescription
• NSMicrophoneUsageDescription

import Speechimport AVFoundation
func requestPermissions() async -> Bool {    let speechStatus = await withCheckedContinuation { continuation in        SFSpeechRecognizer.requestAuthorization { status in            continuation.resume(returning: status)        }    }    guard speechStatus == .authorized else { return false }
    let micStatus: Bool    if #available(iOS 17, *) {        micStatus = await AVAudioApplication.requestRecordPermission()    } else {        micStatus = await withCheckedContinuation { continuation in            AVAudioSession.sharedInstance().requestRecordPermission { granted in                continuation.resume(returning: granted)            }        }    }    return micStatus}

Live Microphone Transcription

The standard pattern: AVAudioEngine captures microphone audio → buffers are appended to SFSpeechAudioBufferRecognitionRequest → results stream in.

import Speechimport AVFoundation
final class LiveTranscriber {    private let recognizer = SFSpeechRecognizer(locale: Locale(identifier: "en-US"))!    private let audioEngine = AVAudioEngine()    private var recognitionRequest: SFSpeechAudioBufferRecognitionRequest?    private var recognitionTask: SFSpeechRecognitionTask?
    func startTranscribing() throws {        // Cancel any in-progress task        recognitionTask?.cancel()        recognitionTask = nil
        // Configure audio session        let audioSession = AVAudioSession.sharedInstance()        try audioSession.setCategory(.record, mode: .measurement, options: .duckOthers)        try audioSession.setActive(true, options: .notifyOthersOnDeactivation)
        // Create request        let request = SFSpeechAudioBufferRecognitionRequest()        request.shouldReportPartialResults = true        self.recognitionRequest = request
        // Start recognition task        recognitionTask = recognizer.recognitionTask(with: request) { result, error in            if let result {                let text = result.bestTranscription.formattedString                print("Transcription: \(text)")
                if result.isFinal {                    self.stopTranscribing()                }            }            if let error {                print("Recognition error: \(error)")                self.stopTranscribing()            }        }
        // Install audio tap        let inputNode = audioEngine.inputNode        let recordingFormat = inputNode.outputFormat(forBus: 0)        inputNode.installTap(onBus: 0, bufferSize: 1024, format: recordingFormat) {            buffer, _ in            request.append(buffer)        }
        audioEngine.prepare()        try audioEngine.start()    }
    func stopTranscribing() {        audioEngine.stop()        audioEngine.inputNode.removeTap(onBus: 0)        recognitionRequest?.endAudio()        recognitionRequest = nil        recognitionTask?.cancel()        recognitionTask = nil    }}

Pre-Recorded Audio File Recognition

Use SFSpeechURLRecognitionRequest for audio files on disk:

func transcribeFile(at url: URL) async throws -> String {    guard let recognizer = SFSpeechRecognizer(), recognizer.isAvailable else {        throw SpeechError.unavailable    }    let request = SFSpeechURLRecognitionRequest(url: url)    request.shouldReportPartialResults = false
    return try await withCheckedThrowingContinuation { continuation in        var didResume = false        recognizer.recognitionTask(with: request) { result, error in            guard !didResume else { return }            if let error {                didResume = true                continuation.resume(throwing: error)            } else if let result, result.isFinal {                didResume = true                continuation.resume(                    returning: result.bestTranscription.formattedString                )            }        }    }}

On-Device vs Server Recognition

SFSpeechRecognizer can use on-device recognition for supported locales on iOS 13+. If supportsOnDeviceRecognition is false, the recognizer requires a network connection. requiresOnDeviceRecognition only has effect when the recognizer supports it.

let recognizer = SFSpeechRecognizer(locale: Locale(identifier: "en-US"))!
// Check if on-device is supported for this localeif recognizer.supportsOnDeviceRecognition {    let request = SFSpeechAudioBufferRecognitionRequest()    request.requiresOnDeviceRecognition = true  // Force on-device}

SFSpeechRecognizer requests may still be a poor fit for long-form capture. Apple documents a roughly one-minute task limit for speech recognition and other service limits. For long recordings on iOS 26+, prefer SpeechAnalyzer; otherwise chunk or restart recognition before the limit and preserve transcript state across tasks.

Handling Results

Partial vs final results

With shouldReportPartialResults, replace the displayed partial transcript until SFSpeechRecognitionResult.isFinal commits it. This is separate from SpeechTranscriber.Result.isFinal, whose volatile attributed range must be replaced by the final result for that range. The live example and references/speechanalyzer-patterns.md [blocked] contain the canonical loops.

Accessing alternative transcriptions and confidence

recognizer.recognitionTask(with: request) { result, error in    guard let result else { return }
    // Best transcription    let best = result.bestTranscription
    // All alternatives (sorted by confidence, descending)    for transcription in result.transcriptions {        for segment in transcription.segments {            print("\(segment.substring): \(segment.confidence)")        }    }}

Adding punctuation (iOS 16+)

let request = SFSpeechAudioBufferRecognitionRequest()request.addsPunctuation = true

Contextual strings

Improve recognition of domain-specific terms:

let request = SFSpeechAudioBufferRecognitionRequest()request.contextualStrings = ["SwiftUI", "Xcode", "CloudKit"]

Common Mistakes

Load references/speechanalyzer-patterns.md [blocked] for complete analyzer finalization and volatile-result code.

Review Checklist

• NSSpeechRecognitionUsageDescription is in Info.plist
• NSMicrophoneUsageDescription is in Info.plist (if using live audio)
• Authorization is requested before starting recognition
• SFSpeechRecognizerDelegate is set to handle availabilityDidChange
• Audio engine is stopped and tap removed when recognition ends
• recognitionRequest.endAudio() is called when done recording
• Previous recognitionTask is canceled before starting a new one
• supportsOnDeviceRecognition is checked before requiring on-device mode
• Partial results are handled separately from final (isFinal) results
• SFSpeechRecognizer one-minute/service limits are accounted for
• For iOS 26+: AssetInventory assets are installed before using SpeechAnalyzer
• For iOS 26+: SpeechTranscriber.isAvailable and locale support are checked
• For iOS 26+: live buffers are converted to the analyzer-compatible format
• For iOS 26+: analyzer sessions are explicitly finalized or canceled
• For iOS 26+: volatile results are replaced by finalized results, not duplicated

References

• Speech framework
• SpeechAnalyzer
• SpeechTranscriber
• SpeechTranscriber.Preset
• DictationTranscriber
• SpeechDetector
• SFSpeechRecognizer
• SFSpeechAudioBufferRecognitionRequest
• SFSpeechURLRecognitionRequest
• SFSpeechRecognitionResult
• SFSpeechRecognitionRequest
• AssetInventory
• Asking Permission to Use Speech Recognition
• Recognizing Speech in Live Audio
• Bring advanced speech-to-text to your app with SpeechAnalyzer