# Convert Audio & Video to Text FREE using Whisper Locally This guide shows how to transcribe audio and video files locally with Whisper WebGPU. Once the model is downloaded, transcription runs on your computer. Author: Vladislav Guzey Published: 2024-10-13 Canonical: https://proflead.dev/posts/how-to-convert-audio-and-video-to-text-free/
This guide shows how to transcribe audio and video files locally with Whisper WebGPU. Once the model is downloaded, transcription runs on your computer. Once you have a transcript, you can use the transcript-based steps in my [video summarization tutorial](/posts/how-to-summarize-a-youtube-video-with-ai-in-2024/) to pull out the main points. ## Initial Requirements to Run Whisper Model Locally We are going to build a web application, so you would need: - Terminal - Git - Node.js & NPM - Browser: Chrome or Firefox - Windows 10+, Linux, or Mac (note: Safari on MacOS not fully support WebGPU) The basic hardware requirements: - Multi-core processor i5 or higher. - 16 GB of RAM is preferable for large audio or video files. ## What is Whisper from OpenAI Whisper is an advanced speech recognition system developed by OpenAI. It’s designed to transcribe spoken language into written text and can also translate different languages. Whisper is known for its accuracy and ability to understand a variety of accents, languages, and even background noise, making it one of the most reliable tools for converting audio to text. ![What is Whisper from OpenAI](/assets/blog/how-to-convert-audio-and-video-to-text-free/how-to-convert-audio-and-video-to-text-free-2.webp) One of the best things about Whisper is that it’s open-source, meaning anyone can access it and use it for free. It can be run on cloud servers or even on your local computer, depending on your needs. Official website: [https://openai.com/index/whisper/](https://openai.com/index/whisper/) ## Hugging Face’s Transformers.js and ONNX Runtime Web We are going to use Whisper WebGPU [https://github.com/xenova/whisper-web/tree/experimental-webgpu](https://github.com/xenova/whisper-web/tree/experimental-webgpu) project. ![Hugging Face’s Transformers.js and ONNX Runtime Web](/assets/blog/how-to-convert-audio-and-video-to-text-free/how-to-convert-audio-and-video-to-text-free-3.webp) This project runs OpenAI’s Whisper model on your device using WebGPU, Hugging Face’s Transformers.js, and ONNX Runtime Web. After downloading the model, transcription can run without an internet connection. ### Key Features of Whisper WebGPU: - **Real-Time In-Browser Processing**: The technology allows for real-time speech recognition within the browser, enhancing user privacy by eliminating the need to send data to external servers. - **Multilingual Support**: It supports transcription and translation in 100 languages, making it a versatile tool for global applications. - **Local Computation**: WebGPU runs the model on your device. Once downloaded, it can work offline. - **Model Characteristics**: The core model used is Whisper-base, which is optimized for web inference with a size of approximately 200 MB. This makes it lightweight yet powerful enough for real-time applications. ## How to Run Whisper Model Locally (Ubuntu, Linux) I will show you how to run it on Ubuntu (Linux). However, if you use Windows or Mac, you can follow the same steps inside, but you have to use the terminal. ### Step 1. Istall GIT, Node.JS, and NPM If you are using Ubuntu, Git should be already there. However, if it’s not, use this command: ``` sudo apt update sudo apt install git ``` Install Node.js: ``` sudo apt install nodejs ``` Install NPM: ``` sudo apt install npm ``` ### Step 2. Turn on WebGPU in the Browser Ensure your browser is configured to support WebGPU. Inside address bar in Crome Browser write **chrome://flags**, then find “**Unsafe WebGPU Support**” enable it, and relaunch the browser. ![Turn on WebGPU](/assets/blog/how-to-convert-audio-and-video-to-text-free/how-to-convert-audio-and-video-to-text-free-4.webp) This is still an experimental feature in some browsers, so you may need to enable it in browser settings. You can check the WebGPU status by opening chrome://gpu/ in your browser. ![WebGPU Status](/assets/blog/how-to-convert-audio-and-video-to-text-free/how-to-convert-audio-and-video-to-text-free-5.webp) In some cases on Ubuntu, even after relaunching, WebGPU could be disabled. In this case, try to open the browser with the following command: ``` /opt/google/chrome/chrome --enable-unsafe-webgpu ``` ### Step 3. Clone Repository and Install Dependencies Clone the Whisper WebGPU project by following the command: ``` git clone https://github.com/xenova/whisper-web.git ``` Once the cloning process is finished, go inside the folder whisper-web: ``` cd whisper-web ``` Then run the following command: ``` npm install ``` After that run: ``` npm run dev ``` To start a web server. The URL of your web application will be available in the terminal window. E.g. **http://localhost:5174/** ![Vite Server](/assets/blog/how-to-convert-audio-and-video-to-text-free/how-to-convert-audio-and-video-to-text-free-6.webp) ### Step 4. Run the Application Go to your browser and open the URL from the terminal to see your application. ![Whisper Web App in the Browser](/assets/blog/how-to-convert-audio-and-video-to-text-free/how-to-convert-audio-and-video-to-text-free-7.webp) This web application supports various audio and video formats and even recording from your microphone. To start the transcription process, simply provide the URL to the audio or upload the video file from your local computer. If you prefer to transcribe an audio file, you can first use a [video-to-MP3 converter](https://videotomp3.online/) to extract the audio from your recording. This step is optional, since Whisper also supports video files directly. ![Transcribing](/assets/blog/how-to-convert-audio-and-video-to-text-free/how-to-convert-audio-and-video-to-text-free-8.webp) ## Video Tutorial Watch on YouTube: [Audio and Video to text converter](https://www.youtube.com/watch?v=YQWNuRTCcUk). ## Conclusion Whisper WebGPU runs transcription and translation in the browser using Whisper, WebGPU, Transformers.js, and ONNX Runtime Web. After downloading the model, you can use it offline. If you like this tutorial, please follow me on [YouTube](https://www.youtube.com/@proflead/videos?sub_confirmation=1), or join my [Telegram](https://t.me/profleaddev). Thanks! :)