admin-valentin/WhisperLiveKit
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Merge pull request #113 from needabetterusername/implement-107

Browse source 
Allow CTranslate2 backend to choose device and compute types.
This commit is contained in:
Quentin Fuxa 2025-04-07 14:42:57 +02:00 committed by GitHub
commit 40cca184c1
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 3 additions and 2 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

5
whisperlivekit/whisper_streaming_custom/backends.py
View file

@ -105,8 +105,9 @@ class FasterWhisperASR(ASRBase):
model_size_or_path = modelsize
else:
raise ValueError("Either modelsize or model_dir must be set")
device = "cuda" if torch and torch.cuda.is_available() else "cpu"
compute_type = "float16" if device == "cuda" else "float32"
device = "auto" # Allow CTranslate2 to decide available device
compute_type = "auto" # Allow CTranslate2 to decide faster compute type
model = WhisperModel(
model_size_or_path,