How to setup Whisper on Windows

This guide demonstrates how to set up a local Whisper speech recognition server on Windows. You will learn how to install the required dependencies, start the Whisper server using agent-cli to use it for speech-to-text transcription.

What is Whisper?

Whisper is an open-source speech recognition model developed by OpenAI. In this setup, it is run locally on your Windows machine using the faster-whisper backend via agent-cli, which exposes an OpenAI-compatible /v1/audio/transcriptions endpoint. This allows you to send audio to the local server and receive transcribed text in response.

Steps to follow

Before proceeding, make sure Anaconda is installed on your system. FFmpeg can be installed via winget, which is built into Windows 10 and 11. The agent-cli package will be installed via pip during the setup process.

1. Open a terminal window
2. Install FFmpeg
3. Set up the Anaconda environment
4. Install agent-cli with faster-whisper
5. Start the Whisper server
6. Send audio to Whisper from Postman

Quick reference commands

# Install FFmpeg audio processing library
winget install ffmpeg

# Create a Python 3.11 Conda environment
conda create -n whisper-ai python=3.11 -y

# Activate the environment
conda activate whisper-ai

# Install agent-cli with the faster-whisper backend
pip install "agent-cli[faster-whisper]"

# Start the Whisper server using the small model
agent-cli server whisper --model small

How to set up and run Whisper on Windows video

The following video shows how to set up and run the Whisper speech recognition server on Windows step-by-step. The video covers installing FFmpeg, creating the Conda environment, installing agent-cli, and starting the server.

Step 1 - Open a terminal window

Open a terminal window on your system. All setup commands in this guide are run from the terminal. You can use Windows PowerShell, or the standard Command Prompt (Figure 1).

Step 2 - Install FFmpeg

Install FFmpeg using the winget package manager. FFmpeg is required by the Whisper backend to handle audio file processing before transcription (Figure 2).

winget install ffmpeg

Step 3 - Set up the Anaconda environment

Create a dedicated Conda environment with Python 3.11 for the Whisper server. Using a separate environment keeps its dependencies isolated from other Python projects on your system (Figure 3).

conda create -n whisper-ai python=3.11 -y

Activate the newly created environment. Your terminal prompt will update to show the active environment name (Figure 4).

conda activate whisper-ai

Step 4 - Install agent-cli with faster-whisper

With the environment active, install agent-cli together with the faster-whisper backend using pip. The faster-whisper extra installs all dependencies needed to run the Whisper speech recognition model locally (Figure 5).

pip install "agent-cli[faster-whisper]"

Step 5 - Start the Whisper server

Start the Whisper server using agent-cli with the small model. You can choose between the different model sizes depending on how powerful your computer is (Figure 6).

agent-cli server whisper --model small

On the first run, agent-cli will automatically download and install the Wyoming protocol dependency. Wait for this process to complete before proceeding (Figure 7).

Once the dependency installation finishes, run the server start command again. This is only required on the first run (Figure 8).

agent-cli server whisper --model small

The Whisper server is now running and listening for transcription requests. Keep this terminal open for the duration of your session (Figure 9).

How to send audio to Whisper from Postman

Postman is an API platform and desktop application used by developers to build, test, debug, and document HTTP-based APIs efficiently. It provides an easy-to-use interface for sending requests, inspecting responses, organizing collections, automating tests, and collaborating on API workflows across teams. Postman can be downloaded for Windows, macOS, and Linux from the official download page at https://www.postman.com/downloads/ using the appropriate installer for your operating system.

In Postman, create a new POST request and set the request URL to http://IP_ADDRESS:PORT/v1/audio/transcriptions, replacing IP_ADDRESS and PORT with the Ubuntu Whisper server’s actual network address. Under the Body tab choose form-data and add a file field of type File, then select the audio file you want to transcribe from your local machine. Add another form-data field named model and set its value to the served Whisper model name running on the vLLM server (for example openai/whisper-small), then save the request so it can be reused for future tests.

Once the request is configured, click the Send button in Postman to submit the selected audio file and model name to the Whisper endpoint on the Ubuntu machine. Postman will upload the multipart form-data payload, where the binary audio content is sent in the file field and the target Whisper model is specified in the model field. The vLLM-based Whisper server receives this request at /v1/audio/transcriptions, processes the audio, and generates a transcription that will be returned in the HTTP response.

After the Whisper server finishes processing, Postman displays the text response in the Response pane, typically as a JSON object containing the recognized transcription. This transcription corresponds to the spoken content of the audio file you sent and can be copied directly from Postman for use in your application or documentation. By reviewing the response format for different test audio files, you can verify that the Ubuntu Whisper server and network configuration are working correctly from the Windows client machine.

To sum it up

You now have a dedicated Conda-based Whisper environment on Windows, with FFmpeg, agent-cli and faster-whisper installed, and the server running locally to process speech recognition requests over /v1/audio/transcriptions. By sending audio from tools like Postman to this endpoint, you can reliably obtain fast, private, fully local speech-to-text results that integrate cleanly into your own applications and workflows without using any external cloud service. This setup gives you full control over the transcription pipeline, from model selection and resource usage to how the returned text is stored or forwarded inside your Ozeki-powered systems.