Whisper Speech-to-Text Softphone in C#

How to register to a PBX and transcribe incoming SIP audio with Whisper

Download:

Whisper_Speech_to_Text.zip

This example shows how to build a console-based Ozeki softphone that registers to a PBX using a SIP account, automatically answers incoming calls, detects speech activity on the remote audio stream, records each utterance into a WAV file, and sends it to a Whisper transcription endpoint for speech-to-text processing.

The layout and section flow below follow the tutorial style of the referenced Ozeki registration example page, which introduces the purpose, required objects, registration steps, and source-code analysis in structured blocks.

Why do we need Whisper Speech-to-Text Softphone?

The Whisper Speech-to-Text Softphone is needed to automatically convert spoken conversations in VoIP calls into searchable, readable text, without requiring manual note-taking or transcription. It helps call centers, support teams, and business users capture the exact wording of customer interactions, making it easier to review calls, comply with regulations, and integrate transcripts into other systems. By embedding Whisper directly into the softphone, you gain an end-to-end solution where calls are handled, recorded, and transcribed in a single workflow, reducing integration complexity and improving productivity.

How does it work

After startup, the application creates a softphone with a defined RTP port range, builds a SIP account object from the configured credentials, and registers the phone line to the PBX. Once registration succeeds, the program waits for incoming calls.

When a call arrives, it answers automatically, starts the microphone, attaches call media sender and receiver objects, and listens to the remote audio stream through a VAD filter. Voice activity starts a new WAV recording, and silence ends the current utterance and triggers transcription.

How to build your solution

Create a new Console Application in Visual Studio, add references to the Ozeki VoIP SDK assemblies, then place the example logic into your Program.cs file. The tutorial page this is based on also starts with creating a console application and adding the SDK reference before explaining the source code.

How to test the solution

To test the solution, first start the softphone application, verify that it registers successfully to the PBX, and confirm there are no errors in the console output. Then place a test call to the SIP account, speak a few sentences, and check that the application detects your voice, records each utterance, and sends the audio to the Whisper endpoint. Finally, verify that the transcribed text appears in the console and that temporary WAV files are cleaned up, ensuring the full speech-to-text pipeline works end to end.

Key source code section

This source code uploads a local WAV audio file to a Whisper speech-to-text HTTP endpoint and retrieves the transcribed text response. It handles basic error conditions such as missing files, non-success HTTP responses, and unexpected exceptions, logging problems to the console instead of throwing them further. After processing, it performs cleanup by deleting the temporary audio file, ensuring no leftover artifacts remain on disk.

// Sends the recorded WAV file to the Whisper model and prints the transcribed text
static async Task SendToWhisperAsync(string filename)
{
    try
    {
        if (!File.Exists(filename))
        {
            Console.WriteLine($"Audio file not found: {filename}");
            return;
        }

        using var form = new MultipartFormDataContent();
        using var fileStream = File.OpenRead(filename);
        var fileContent = new StreamContent(fileStream);
        fileContent.Headers.ContentType = new MediaTypeHeaderValue("audio/wav");
        form.Add(fileContent, "file", Path.GetFileName(filename));
        form.Add(new StringContent(WhisperModel), "model");

        using var response = await _httpClient.PostAsync(WhisperUrl, form);
        response.EnsureSuccessStatusCode();

        var json = await response.Content.ReadAsStringAsync();
        using var doc = JsonDocument.Parse(json);
        var text = doc.RootElement.GetProperty("text").GetString();

        Console.WriteLine($"Whisper: {text}");
    }
    catch (Exception ex)
    {
        Console.WriteLine("Error sending audio to Whisper: " + ex.Message);
    }
    finally
    {
        // Delete the temporary audio file
        try { if (File.Exists(filename)) File.Delete(filename); } catch { }
    }
}

Whisper configuration

Source code section

This source code defines the Whisper speech-to-text endpoint URL, the model name to use, and a reusable HTTP client instance for sending transcription requests. By centralizing these values into static readonly fields, the application ensures that all Whisper-related calls share the same configuration and avoid repeated object creation overhead. This setup makes it easy to change the endpoint or model in one place and provides a single long-lived HttpClient for efficient network communication.

// Whisper speech-to-text configuration
static readonly string WhisperUrl = "http://192.168.0.100:8123/v1/audio/transcriptions";
static readonly string WhisperModel = "whisper-large-v3";
static readonly HttpClient _httpClient = new HttpClient();

Whisper HTTP communication in Wireshark

What is a Whisper HTTP Request PDU

A Whisper HTTP Request PDU is the HTTP message that the softphone sends to the Whisper server, carrying the recorded WAV file and model name as a multipart/form-data POST request. It contains the request line, headers such as content type and content length, and the binary audio body that Whisper will process for transcription. In Wireshark, this PDU appears as an outgoing HTTP POST packet toward the Whisper endpoint, representing the start of a speech-to-text operation.

What is a Whisper HTTP Response PDU

A Whisper HTTP Response PDU is the HTTP message that the Whisper server sends back to the softphone after processing an uploaded audio file. It contains the status line, response headers, and a JSON body that includes the recognized transcription text and possibly additional metadata. In Wireshark, this PDU appears as an incoming HTTP response packet from the Whisper endpoint, confirming that the speech-to-text operation completed and returning the result to the application.

Main objects used in the example

Why do we need SIP Registration in a VoIP Softphone?

A PBX needs to know exactly where to send a call before it can deliver one. Phones don't have a fixed, permanent location on the network: they can be assigned a different IP address each time they connect, move between networks, or simply be switched off. SIP registration solves this by having each phone periodically tell the PBX "this is who I am, and this is where you can currently reach me." Without this step, the PBX would have no reliable way of finding a phone, and incoming calls to that extension would have nowhere to go.

PBX registration details

Like the reference tutorial, this example depends on a SIP account object that contains the basic PBX registration parameters such as registration requirement, display name, user name, authentication ID, password, domain host, and domain port.

Source code section

// SIP account registration data, (supplied by your VoIP service provider)
var registrationRequired = true;
var userName = "1001";
var displayName = "1001";
var authenticationId = "1001";
var registerPassword = "1001";
var domainHost = "192.168.0.173";
var domainPort = 5060;

var account = new SIPAccount(registrationRequired, displayName, userName, authenticationId, registerPassword, domainHost, domainPort);

// Send SIP registration request
RegisterAccount(account);   

Application flow

1. Create the softphone object with an RTP port range.
2. Create the SIP account object with PBX registration data.
3. Register the phone line.
4. Wait for incoming calls.
5. Answer the call automatically.
6. Attach media sender and receiver objects.
7. Detect remote speech with the VAD filter.
8. Record each detected utterance to a WAV file.
9. Stop recording after a silence timeout.
10. Send the WAV file to Whisper for transcription.
11. Print the returned text to the console.
12. Clean up media devices, timers, and temporary files.

Source code analysis

The original tutorial structure explains the example by separating setup, registration, event handling, and object usage into dedicated sections, and this skeleton follows that same documentation pattern.

The Main() method initializes the softphone, prepares the SIP registration settings, starts registration, initializes media components, and keeps the application alive with Console.ReadLine().

This method creates the phone line from the SIP account, subscribes to registration state change events, subscribes to incoming call events, and sends the SIP REGISTER request through RegisterPhoneLine(). The reference page also centers registration around creating a phone line and then registering it on the softphone.

This event handler monitors whether registration succeeds or fails. If the state is RegistrationSucceeded, the softphone is online; if the state is NotRegistered or Error, registration failed.

When an incoming call arrives, the program stores the call reference, reads the caller ID, subscribes to call state changes, and answers the call automatically.

This method monitors the call lifecycle. When the call is answered it sets up the media path, and when the call ends it shuts down and disposes the related objects.

This method starts the microphone, connects it to the audio sender, creates and configures the VAD filter, connects the remote audio stream to that filter, creates the silence timer, and attaches sender and receiver handlers to the active call.

The VAD event signals that incoming speech has been detected. If there is no active recording yet, a new utterance recording starts, and the silence timer is reset every time new voice is detected.

A new temporary WAV filename is generated for each utterance, a WaveStreamRecorder is created, the remote audio receiver is connected to the recorder, and recording begins.

If no new voice is detected within the configured silence period, the current WAV recording is finalized. After the file is closed safely, it is queued for upload to the Whisper service.

This method sends the completed WAV file to the configured Whisper HTTP endpoint as multipart form data. After a successful response, it reads the returned JSON, extracts the text property, writes the transcription to the console, and deletes the temporary audio file.

This method stops the timer, finalizes any still-active recording, detaches media components from the call, disconnects and disposes the VAD filter, releases the microphone, disposes the media connector, and clears the timer reference.

Notes

• Use valid SIP credentials for your PBX.
• Adjust the RTP port range if your firewall policy requires it.
• Set the VAD activation level based on your audio environment.
• Change the silence timeout to control utterance segmentation.
• Make sure the Whisper endpoint is reachable from the machine running the app.

Next steps

After the registration and media pipeline work correctly, this example can be extended with call logging, WAV archiving, speaker playback, outbound dialing, real-time transcript display, or database storage for recognized text.