STREAM: [NFR] Can it support mixed configuration of multiple streams and ignore when the additional stream has no data?

[dreamsxin]

Generally used for audio stream synthesis or special effects.

TRANS_BY_GPT3

[winlinvip]

Mixing is quite complicated, we'll see if future versions can support it.

TRANS_BY_GPT3

[dreamsxin]

Or provide the cutting stream function, which can facilitate third-party tools for mixing and pushing. For example:
streamOut serves as the external output stream, by default pulling the stream from streamIn.
streamIn is the client input stream.

If a third party wants to mix StreamIn, they can first pull the stream from streamIn, then synthesize a new stream streamMux, and push it to streamOut. If this switching can be seamless, it would also be a solution.

TRANS_BY_GPT3

[w5750584]

@dreamsxin
May I ask, what third-party tool are you using for mixing?
I tested using ffmpeg for mixing, but if one of the streams is interrupted, there will be issues with streamOut.
./ffmpeg -i rtmp://srs.xxx.com:1935/live/1 -i rtmp://srs.xxx.com:1935/live/2 -filter_complex "[0:a][1:a]amerge=inputs=2[aout]" -map "[aout]" -f flv rtmp://srs.xxx.com:1935/live/test
When stream 1 or 2 is disconnected, the test stream will encounter an error.

TRANS_BY_GPT3

[dreamsxin]

@w5750584 Mixing, mixing in an audio file, specifying time and length, there will be no interruption problem, using a stream will definitely cause errors, unless you write it yourself

TRANS_BY_GPT3

[winlinvip]

I plan to mix the audio at the PCM level myself. SRS sends AAC or Speex or Opus frames to an external program, which decodes them into PCM, mixes them, encodes them back into AAC, and then sends them back to SRS.

TRANS_BY_GPT3

[winlinvip]

This program can be written in GOLANG, which makes it convenient to communicate with SRS using HTTP. Some work has already been done, including using GOLANG to call the libraries for SPEEX and AAC, as well as resampling.

1. https://github.com/winlinvip/go-speex
2. https://github.com/winlinvip/go-fdkaac
3. https://github.com/winlinvip/go-aresample

TRANS_BY_GPT3

[winlinvip]

Currently, it seems that mixing may not be necessary, and possible directions could be:

1. RTC communication direction, such as WebRTC, where the server typically does audio ranking instead of mixing. WebRTC: Support WebRTC, for video meeting, SFU server, video converence. #307
2. RTSP monitoring direction, such as GB18181, Do you support integration with GB28181? Monitoring, intelligent cameras, national standard streaming. #1500, where there may be audio transcoding to AAC.
3. Low-latency live streaming direction, such as WebRTC streaming, where Opus to AAC conversion may exist. WebRTC: Support WebRTC, for video meeting, SFU server, video converence. #307

TRANS_BY_GPT3