Every community workshop, technical presentation, panel discussion, and open lecture produces valuable spoken knowledge. Yet raw audio recordings remain closed, opaque data silos: they cannot be indexed by search engines, skimmed quickly, cross-referenced, or easily cited. This project delivers a high-speed, local-first speech-to-text pipeline for the Macedonian language, utilizing OpenAI's Whisper models accelerated by CTranslate2 to execute smoothly on accessible consumer hardware without sending audio to corporate cloud surveillance pipelines.

Transformer Acceleration via CTranslate2 & faster-whisper

Standard PyTorch implementations of Whisper suffer from heavy memory allocation overhead and slow execution loops. We achieve production-grade performance by executing models through the CTranslate2 engine and faster-whisper: - Dedicated Tensor Computation Kernels: CTranslate2 replaces Python-level interpreter overhead with custom C++ tensor execution kernels optimized for NVIDIA CUDA/cuDNN on GPUs and SIMD vector extensions (AVX-512, AVX2, ARM NEON) on modern CPUs. - 4x Throughput Multiplier: Transcribes Macedonian audio streams up to four times faster than vanilla PyTorch implementations while preserving identical word error rates and acoustic precision.

Local GPU & CPU Quantization (INT8 / FP16)

The breakthrough enabling local community deployment without multi-thousand-euro GPU clusters is neural weight quantization: - 8-Bit Integer Quantization (INT8): By mapping 32-bit floating-point weights to 8-bit integers with dynamic per-tensor scaling, the memory footprint of the flagship Whisper large-v3 model drops by over 65%—from 6.2 GB down to just ~2.1 GB. - Consumer GPU Execution: The full large-v3 model runs effortlessly on entry-level consumer GPUs equipped with 6GB or 8GB VRAM (such as NVIDIA GTX 1660 Ti, RTX 3060, or refurbished workstations). - Efficient Multi-Core CPU Inference: On consumer laptops powered by modern multi-core processors (AMD Ryzen, Intel Core i5/i7, or Apple Silicon ARM), the engine utilizes 4 to 8 parallel compute threads with int8_float16 hybrid kernels, executing faster than real-time speech (RTF < 0.6).

Open Ingestion Pipeline: Silero VAD & Word Timestamps

To suppress hallucination artifacts and produce synchronized transcripts, audio passes through a modular automated pipeline: 1. Voice Activity Detection (Silero VAD): Audio is segmented using a 30ms sliding analysis window. Background silence, HVAC fan hum, room reverberation, and applause are pruned before reaching the transformer encoder, preventing degenerative token looping. 2. Batched Beam Search Decoding: Utilizes a 5-candidate beam search with an adaptive temperature fallback schedule (0.0, 0.2, 0.4, 0.6, 0.8) triggered whenever repetition thresholds are detected. 3. Word-Level Timestamp Alignment: Inspecting cross-attention weight matrices within the decoder maps precise start and end millisecond timestamps to every recognized word. 4. Automated Multi-Format Export: Directly outputs time-aligned SRT and WebVTT subtitle files for video platforms, alongside structured Markdown transcripts tagged by speaker.

Preserving Public Knowledge & Open Archives

Open transcription transforms fleeting spoken words into an enduring digital commons: - Searchable Audio Archives: Public lectures on DIY solar installations, community microgrids, and open hardware become instantly searchable by local search engines and web crawlers. - Offline Kiwix Integration: Transcripts are bundled directly into local knowledge mirrors served by our off-grid solar edge stations, making critical documentation available even in remote rural mountain hubs. - Regional Linguistic Inclusion: Lowering barriers for cross-Balkan collaboration by providing accurate source transcripts for downstream open translation models. All pipeline scripts, CLI utilities, and Docker containers are open source and freely available in our git repository.