WhisperLiveKit/whisperlivekit/simul_whisper
google-labs-jules[bot] 70e854b346 feat: Allow loading fine-tuned models in simulstreaming
This change modifies the `simulstreaming` backend to support loading fine-tuned Whisper models via the `--model_dir` argument.

The `SimulStreamingASR` class has been updated to:
- Use the `model_dir` path directly to load the model, which is the correct procedure for fine-tuned `.pt` files.
- Automatically disable the `faster-whisper` and `mlx-whisper` fast encoders when `model_dir` is used, as they are not compatible with standard fine-tuned models.

The call site in `core.py` already passed the `model_dir` argument, so no changes were needed there. This change makes the `simulstreaming` backend more flexible and allows users to leverage their own custom models.
2025-09-27 07:29:30 +00:00
..
whisper 0.2.8 : only the decoder of whisper is loaded in memory when a different encoder is used 2025-09-02 21:12:25 +02:00
__init__.py refactor(simulstreaming): extract backend + online module into separate files from whisper streaming 2025-08-08 18:07:51 +02:00
backend.py feat: Allow loading fine-tuned models in simulstreaming 2025-09-27 07:29:30 +00:00
beam.py modify import paths in simul whisper backend so that it works in lib mode 2025-07-01 20:34:47 +02:00
config.py audio_max_len to 30 -> 20, ffmpeg timeout 5 -> 20 2025-08-17 09:32:08 +02:00
eow_detection.py modify import paths in simul whisper backend so that it works in lib mode 2025-07-01 20:34:47 +02:00
generation_progress.py latest version of simulstreaming 2025-07-31 16:44:23 +02:00
license_simulstreaming.py each SimulStreamingOnlineProcessor now contains PaddedAlignAttWhisper instance. SimulStreamingASR only contains loaded whisper model 2025-08-11 08:24:14 +02:00
mlx_encoder.py mlx/fasterWhisper encoders are loaded once and shared in simulstreaming 2025-09-01 12:33:19 +02:00
simul_whisper.py translation compatible with auto and detected language 2025-09-22 11:20:00 +02:00
token_buffer.py modify import paths in simul whisper backend so that it works in lib mode 2025-07-01 20:34:47 +02:00