❓ONNX model not support batching inputs ?

[yanyusong]

❓ Questions and Help

Hi.
Thank you for the provided repo and models.

ONNX model not support batching inputs ?

I seen the onnx model only receive 1-D dim data, how can I input the data of the shape like [ batch_size, 1024] .

[adamnsandle]

Hi!
Currently onnx model does not support batching

[adamnsandle]

Latest ONNX version (v4) supports both 8k and 16k sampling rates as well as batching.
https://github.com/snakers4/silero-vad/wiki/Version-history-and-Available-Models

[mhThomsen]

@Simon-chai As far as I recall, if you just input tensors shaped [b,a] where b is batch dim and a is audio dim, then you should get a [b,1] shaped output

[Simon-chai]

import torch,soundfile as sf,numpy as np

torch.set_num_threads(1)
cur_model, utils = torch.hub.load(repo_or_dir='path_to_models/snakers4_silero-vad_master',
                                        model='silero_vad',source='local',trust_repo=True,onnx=True)
(get_speech_timestamps,save_audio,read_audio,VADIterator,collect_chunks) = utils
with torch.no_grad():
    audio_data,sr = sf.read('shuang.mp3') #audio file with 2 channels
    print(audio_data.shape)
    input = np.array([audio_data[:, 0],audio_data[:, 1]])
    print(input.shape)
    some = get_speech_timestamps(input,cur_model,sampling_rate=sr)
    print(some)

The above code show output like :

(162400, 2)
(2, 162400)
Traceback (most recent call last):
File "path_to_project/test_silero.py", line 14, in
some = get_speech_timestamps(input,cur_model,sampling_rate=sr)
File "path_to_models/snakers4_silero-vad_master\utils_vad.py", line 240, in get_speech_timestamps
raise ValueError("More than one dimension in audio. Are you trying to process audio with 2 channels?")
ValueError: More than one dimension in audio. Are you trying to process audio with 2 channels?

Should I use another function？

[mhThomsen]

I needed to get closer to the model predictions so I never tried it with the get_speech_timestamps but i got it to work with just the model itself:

import torch

model, _ = torch.hub.load(
            repo_or_dir="snakers4/silero-vad", model="silero_vad"
        )

#Create sampled data, resembling audio in 16kHz.

#Shape is 16 (batch size) x 512. iirc, 512 is the chunk size used in the model.
audio_batch = torch.normal(0,0.025,size=(16,512))

#assuming samplerate is 16000
pred_probs = model(audio_batch, 16000)

print(pred_probs.shape)

outputs torch.Size([16, 1])

[mhThomsen]

Note the the VAD model in the get_speech_timestamps uses some some algorithm to give you the speech timestamps. According to this the model was trained on these 512 sized chunks which is 31.25 ms in a with sample rate 16000.

To get the speech timestamps for full audio you need to iterate your audio data as a rolling window in chunks of size 512. Take a look on the algorithm behind the get_speech_timestamps to know individual model predictions can be made into speech predictions

[Simon-chai]

@mhThomsen Thanks a lot. I have already successfully batching process by model. It do help in my offline vad scenario. With little change the get_speech_timestamps can process the batching input as well. And the algorithm behind the get_speech_timestamps can help filtering some 'noise' that model consider as speech,it's good enough for me.