how to achieve acceleration for the S2S Speech-to-Speech mode？

[eyree]

Thank you very much for your work. Is it possible to achieve acceleration for the S2S mode, and if so, when is it expected to be supported?
I am currently attempting to develop an S2S version based on the already implemented S2T vllm acceleration, and I have encountered a major issue: In FunAudioChatDecoder, the crq_generate_forward (SRH head) needs to sample the next token based on speech_id, and this next token then becomes the input for SRH. However, vLLM handles the updating and merging of speech_id at the engine level. I would also appreciate guidance from experts on this problem.

[nemoramo]

Thank you very much for your work. Is it possible to achieve acceleration for the S2S mode, and if so, when is it expected to be supported? I am currently attempting to develop an S2S version based on the already implemented S2T vllm acceleration, and I have encountered a major issue: In FunAudioChatDecoder, the crq_generate_forward (SRH head) needs to sample the next token based on speech_id, and this next token then becomes the input for SRH. However, vLLM handles the updating and merging of speech_id at the engine level. I would also appreciate guidance from experts on this problem.

I think a better method is to adpot vllm-omni repo to support S2S structure. I can help to release a version of VLLM-omni, but it may take a while. Stay tuned.

[nemoramo]

Btw, I am not a TTS guy. So this implementation may take a while. Need to understand the "2S" part of this model

[eyree]

Thank you very much for your work. Is it possible to achieve acceleration for the S2S mode, and if so, when is it expected to be supported? I am currently attempting to develop an S2S version based on the already implemented S2T vllm acceleration, and I have encountered a major issue: In FunAudioChatDecoder, the crq_generate_forward (SRH head) needs to sample the next token based on speech_id, and this next token then becomes the input for SRH. However, vLLM handles the updating and merging of speech_id at the engine level. I would also appreciate guidance from experts on this problem.

I think a better method is to adpot vllm-omni repo to support S2S structure. I can help to release a version of VLLM-omni, but it may take a while. Stay tuned.

That's great! Thank you so much!

[eyree]

Btw, I am not a TTS guy. So this implementation may take a while. Need to understand the "2S" part of this model

Thanks, I extend sincere gratitude to your invaluable contribution: support funaudiochat ST2T mode inference-only in vLLM Thanks very much!!

[nemoramo]

Btw, I am not a TTS guy. So this implementation may take a while. Need to understand the "2S" part of this model

Thanks, I extend sincere gratitude to your invaluable contribution: support funaudiochat ST2T mode inference-only in vLLM Thanks very much!!

will be updated these days, stay tuned

[nemoramo]

vllm-project/vllm-omni#1748 @eyree if you are still interesting in vllm based funaudiochat s2s inference

[eyree]

vllm-project/vllm-omni#1748 @eyree if you are still interesting in vllm based funaudiochat s2s inference

it's very impressive! I am very grateful for your work.

[eyree]

vllm-project/vllm-omni#1748 @eyree if you are still interesting in vllm based funaudiochat s2s inference

@nemoramo I noticed that the PR hasn’t been merged into vllm-omni yet. I was wondering if you might have time to revisit it when convenient. I would greatly appreciate it!