Hoping to further understanding of readrate and forced read rate (-re) in 4.1.1 update.

[StevenAEdwards]

Hi! I use this awesome package for my own discord streambot that streams IPTV, jellyfin, and twitch links to discord. I was updating packages and wanted to use 4.1.1 to get rid of liabilities. I swapped over and noticed that my streams were choppy. I've always had nativeReadFps off since most of the time it caused my streams to have stuttering/audio desync specifically when using the jellyfin file links. I noticed that it was discussed in the patch notes and that it should always be defaulted to on so I was hoping to understand better as to why? Is this a problem on my end that I can fix to make my stream quality more consistent? I don't think this deserves an issue since currently I'm using 4.0.0 and streams work fine with -re off. I read the ffmpeg docs and it seems like my issues should be reversed but they are not. I was just hoping the community could help me better understand how it works / what I'm doing wrong. Thanks!

from https://ffmpeg.org/ffmpeg.html

-readrate speed (input) Limit input read speed. Its value is a floating-point positive number which represents the maximum duration of media, in seconds, that should be ingested in one second of wallclock time. Default value is zero and represents no imposed limitation on speed of ingestion. Value 1 represents real-time speed and is equivalent to -re. Mainly used to simulate a capture device or live input stream (e.g. when reading from a file). Should not be used with a low value when input is an actual capture device or live stream as it may cause packet loss. It is useful for when flow speed of output packets is important, such as live streaming.

-re (input)
Read input at native frame rate. This is equivalent to setting -readrate 1.

[dank074]

What the -re flag does is that the Ffmpeg process will read the media at its native frame rate instead of reading and transcoding the whole media file as fast as possible.

Previously when readAtNativeFps was disabled we used setTimeout inside of a node Stream to artificially limit Ffmpeg to read at the set fps. However this was no longer possible after the recent refactoring for fixing some A/V desync issues: the ffmpeg output was sent to a demux function instead of being piped directly into VideoStream so the setTimeout was not holding back ffmpeg from reading the entire file at once, which could potentially fill up your RAM if you have a large video file.

The recent changes should've not caused more desync issues, they should've solved them. What kind of files are causing issues for you? You said IPTV and twitch, so is it livestreams that are having these issues? We did test HLS livestreams and they weren't having issues on our end. Let us know more details so we can investigate. Can you also try v4.1.0 and let us know if that one works for you

[StevenAEdwards]

If this should be in an issue let me know I can create one and add as much context as possible!

To answer your questions,
My first use case is twitch/IPTV links, these are livestreams. I have found that livestreams with -re tend to constantly stutter / skip with the newest update. It's severe and is significantly better when reverting my version and even when it's working the audio will end up desyncing after a few stutters. I have been able to stream the source stream in VLC locally without any stutters or desync so it's not the source. I have also been able to stream that VLC through my screen share without any issues.

Second use case is locally hosted jellyfin download links. These are straight from a file. Which if it was downloading the whole file in memory and streaming it, that would make sense why it was so buttery smooth. When I use -re with the jellyfin links the audio is desynced by like 500ms. This happens on either version and is probably either a config issue on my end or something unrelated to the update. But its something that is completely solved without -re on the old version. (but i understand why that wasn't ideal). I can watch the source file without any desync just fine through jellyfin.

If there is any other data I can provide let me know!

Desynced Jellyfin File Media Info

Video
Title: 1080p AV1 SDR
Codec: AV1
Profile: Main
Level: 8
Resolution: 1920x816
Aspect ratio: 2.35:1
Interlaced: No
Framerate: 23.976025
Bitrate: 4346 kbps
Bit depth: 10 bit
Video range: SDR
Video range type: SDR
Color transfer: bt709
Color primaries: bt709
Pixel format: yuv420p10le
Ref frames: 1

Audio
Title: English - OPUS - 5.1 - Default
Language: eng
Codec: OPUS
Layout: 5.1
Channels: 6 ch
Bitrate: 291 kbps
Sample rate: 48000 Hz
Default: Yes
Forced: No
External: No

stuttery IPTV stream ffmpeg -i

Input #0, hls, from 'stream_url':
Duration: N/A, start: 2400.398033, bitrate: N/A
Program 0
Metadata:
variant_bitrate : 0
Stream #0:0: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p(tv, bt709), 1280x720 [SAR 1:1 DAR 16:9], Closed Captions, 29.97 fps, 29.97 tbr, 90k tbn
Metadata:
variant_bitrate : 0
Stream #0:1: Audio: aac (LC) ([15][0][0][0] / 0x000F), 48000 Hz, stereo, fltp
Metadata:
variant_bitrate : 0