Flickering with CPU use

[davepl]

I'm getting a lot of changes in overall brightness of the matrix second to second when the CPU is being used on the Pi.

I am receiving video frames over wifi/lan and decompressing them on the fly. No core is fully saturated, sitting at about 75% of a single core.

I'm running as root, have set the isocpus=3 in boot. Is there anything else I can try to remediate this?

[hzeller]

Unfortunately, the Pi shares internal data busses of what is needed to do the regular GPIO output and whatever it needs to for e.g. uncompressing, so even if the cores are not saturated some IO over the network or USB can already influence the flickering. Besides the usual recommendations w.r.t flickering found in the README, reducing as much as possible memory churn is the best you can do; the usual suspects: not using Python (it has a huge garbage collection impact), avoid scaling of images on the Pi, send them pre-scaled, possibly avoid uncompression, but just sent images as-is ...

For remote network sending I typically use the flaschen-taschen protocol for which there is a server implementation for the rpi-rgb-led-matrix ).

[davepl]

I’m not a Linux head, but I’m creating my worker thread with std::thread. I wonder if setting the priority on the worker thread to be lower than the main thread, which manages the matrix, would help? - Dave

…

On Aug 18, 2024, at 10:41 AM, Henner Zeller ***@***.***> wrote: Unfortunately, the Pi shares internal data busses of what is needed to do the regular GPIO output and whatever it needs to for e.g. uncompressing, so even if the cores are not saturated some IO over the network or USB can already influence the flickering. Besides the usual recommendations w.r.t flickering found in the README, reducing as much as possible memory churn is the best you can do; the usual suspects: not using Python (it has a huge garbage collection impact), avoid scaling of images on the Pi, send them pre-scaled, possibly avoid uncompression, but just sent images as-is ... For remote network sending I typically use the flaschen-taschen <https://github.com/hzeller/flaschen-taschen> protocol for which there is a server implementation for the rpi-rgb-led-matrix <https://github.com/hzeller/flaschen-taschen/tree/master/server#rgb-matrix-panel-display> ). — Reply to this email directly, view it on GitHub <#1698 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA4HCF6IIAYJDEXNM3SJTALZSDMF7AVCNFSM6AAAAABMWRCRH2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEOJVGMZTOMJRGQ>. You are receiving this because you authored the thread.

[hzeller]

std::thread sounds good and should work fine. The relevant GPIO-pushing thread is already high priority so any 'normal' thread should be lower prio and not interfere from the CPU level.

But I suspect it is not actually the CPU-core utilization that is limiting, but internal memory busses inside the Pi (in particular with older Pis before Pi4).
The included video-viewer also does scaling on-the-fly, and I have seen some flickering in particular if it has do downscale from a large video input.

If you have a stream of images that you send, I recommend double-buffering using the SwapOnVSync() feature if you're not doing that already, otherwise there might be tearing which sometimes can look like flicker (though I still suspect the memory churn the decompression creates is responsible for the observed flicker).

Another thing: are you using the GUI on the Raspberry Pi or using it headless, just via ssh ? Anything running besides your main LED application can create flicker. So running headless with a minimum installation is good. Even running top in the background results in periodic flicker whenever it updates its output.
Pushing several Megahertz of Pixels out in a thread that somewhat requires real-time-ness and unobstructed memory-bus is somewhat problematic on the Pi with the limited resources...

Maybe you can play with --led-limit-refresh to create a time buffer to eliminate visual flicker (the README describes the process).

I’m not a Linux head,

No worries, I on the other hand have never used Windows -- it is always a bit hard to get used to different systems.
I like your YouTube channel!

[davepl]

Thanks! If you’re curious, you check out the code at https://github.com/davepl/NightDriver-Pi.git Basically, it opens a socket that receives the color data for each frame. Those frames are generated about 3 seconds in the future on the PC and sent to the Pi and placed in a circular buffer until their timestamp comes due, then they are displayed and I call SwapOnVSync. It works great up to about 100fps. Effects are all generated on the PC side, and can even use the GPU when needed. It also supports lz compression, which is pretty effective. But it does increase CPU load, which exacerbates the flicker. I don’t think I have anything else running on the Pi, though I just used a default Raspberry Pi image - so while I’ve never connected a monitor to it, it *could* be a full install, not 100% sure. For sure no one is logged in that way :-) - Dave 

…

On Aug 18, 2024, at 1:35 PM, Henner Zeller ***@***.***> wrote: std::thread sounds good and should work fine. The relevant GPIO-pushing thread is already high priority so any 'normal' thread should be lower prio and not interfere from the CPU level. But I suspect it is not actually the CPU-core utilization that is limiting, but internal memory busses inside the Pi (in particular with older Pis before Pi4). The included video-viewer <https://github.com/hzeller/rpi-rgb-led-matrix/tree/master/utils#video-viewer> also does scaling on-the-fly, and I have seen some flickering in particular if it has do downscale from a large video input. If you have a stream of images that you send, I recommend double-buffering using the SwapOnVSync() feature if you're not doing that already, otherwise there might be tearing which sometimes can look like flicker (though I still suspect the memory churn the decompression creates is responsible for the observed flicker). Another thing: are you using the GUI on the Raspberry Pi or using it headless, just via ssh ? Anything running besides your main LED application can create flicker. So running headless with a minimum installation is good. Even running top in the background results in periodic flicker whenever it updates its output. Pushing several Megahertz of Pixels out in a thread that somewhat requires real-time-ness and unobstructed memory-bus is somewhat problematic on the Pi with the limited resources... Maybe you can play with --led-limit-refresh to create a time buffer to eliminate visual flicker (the README describes the process). I’m not a Linux head, No worries, I on the other hand have never used Windows -- it is always a bit hard to get used to different systems. I like your YouTube channel! — Reply to this email directly, view it on GitHub <#1698 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA4HCF33BO34EZ56SEW6S5DZSEAQNAVCNFSM6AAAAABMWRCRH2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEOJVGM4DENJTGY>. You are receiving this because you authored the thread.

[hzeller]

Had a quick look

• You're using SwapOnVSync() in the MatrixDraw, however not with double-buffering as you still write in the active matrix. For double-buffering, you'd first need to create a single offscreen-buffer (offscreen = matrix->CreateFrameCanvas() somewhere in the beginning), then fill the offscreen buffer (which implements the Canvas interface); then offscreen = matrix->SwapOnVSync(offscreen); swaps the offscreen buffer and returns the other buffer to fill in the next round. A good example might be seen in the clock.cc example code.
  For that you'd need some state (the offscreen buffer), which would mean having to change the MatrixDraw to non-static functions...
• The fairly tight polling is probably something I'd avoid and rather have the BufferManager return the next time (I'd use struct timespec) which you then can wait on with an absolute time clock_nanosleep(CLOCK_REALTIME, TIMER_ABSTIME, &frame_time, NULL);. Depending on if the frames come in out-of-sequence, there might be a different strategy. Anyway, probably not too much of a big problem w.r.t. the flickering, but any unnecessary calculation that can be avoided (like every millisecond figuring out what the age of the buffer is now, including locking and unlocking a mutex and doing a floating point calculation) makes it also easier to soothe programmer OCD :)

[davepl]

Thanks… I have some code that does the latter, but didn't bother to integrate it; I will! I honestly thought I wouldn’t double-buffer until I see visible tearing and have a problem to solve! Is there a chance it’ll help with the variations in brightness? - Dave

…

On Aug 18, 2024, at 5:01 PM, Henner Zeller ***@***.***> wrote: nd rather have the BufferManager return the next time (I'd use struct timespec) which you then can wait on with an absolute time clock_nanosleep(CLOCK_REALTIME, TIMER_ABSTIME, &frame_time, NULL);. Depending on if the frames come in out-of-sequence, there might be a different strategy. Anyway, probably not too much of a big problem w.r.t. the flickering, but any unnecessary calculation that can be avoided (like every millisecond figuring out what the age of the buffer is now, including locking and unlocking a mutex and doing a floating point calculation) makes it also easier to soothe programmer OCD :)

[hzeller]

Is there a chance it’ll help with the variations in brightness?

Probably not if the brightness variations are not actually due to subtle tearing.

I see you're using the adafruit-hat as default in your Library. That will always result in subtle flicker unless you do the necessary hardware mod and then switch to adafruit-hat-pwm.

Without that, the timing of the various bit-planes has to be done in software, otherwise we can use a pin on the Pi that allows a hardware-generated timing. That is the single-most important thing to combat flicker.

Then afterwards, the observing the --led-show-refresh and choosing a value that is slightly below the typical variation with --led-limit-refresh.

[davepl]

I got it all working well enough, here’s a summary of what I changed! I had already done the hardware mod, but was running adafruit-hat instead of adafruit-hat-pwm. Still always worked, and changing didn’t seem to have any impact. I changed my wait function to sleep for the precise amount of time until the next frame is due rather than looping on a 1ms delay. I capped refresh rate at 60Hz and disabled busy waiting. These two were essential in reducing CPU, which was close to 90%, down to about the 35-40% total. And that’s while lz-decompressing 1.2M/sec of video data, so not bad! It’s not perfect, but the amount of flicker that remains is easily tolerable for the content I’m running. Thanks very much for investing the time in making this library! It was impressively easy to integrate with, and the demo apps were instructive. The commands line processing is also cool. Cheers, Dave

…

On Aug 18, 2024, at 6:01 PM, Henner Zeller ***@***.***> wrote: Is there a chance it’ll help with the variations in brightness? Probably not if the brightness variations are not actually due to subtle tearing. I see you're using the adafruit-hat <https://github.com/davepl/NightDriver-Pi/blob/main/globals.h#L12> as default in your Library. That will always result in subtle flicker unless you do the necessary hardware mod <https://github.com/hzeller/rpi-rgb-led-matrix?tab=readme-ov-file#improving-flicker> and then switch to adafruit-hat-pwm. Without that, the timing of the various bit-planes has to be done in software, otherwise we can use a pin on the Pi that allows a hardware-generated timing. That is the single-most important thing to combat flicker. Then afterwards, the observing the --led-show-refresh and choosing a value that is slightly below the typical variation with --led-limit-refresh. — Reply to this email directly, view it on GitHub <#1698 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA4HCF5PMUEWWNS3SDQY7RLZSE7X3AVCNFSM6AAAAABMWRCRH2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEOJVGQ4DINZQGM>. You are receiving this because you authored the thread.

[hzeller]

Nice, glad you got it working. I still would strongly recommend using the adafruit-hat-pwm as it will in paricular help brightness flicker.

60Hz refresh is somewhat slow and would at least for me create some headache watching :) Usually I'd go with whatever the natural --led-show-refresh shows and then round it down a little. Say you see typical 150Hz with glitches down to 137Hz (--led-show-refresh shows the minimum refresh observed), maybe 140Hz is a good choice.

In a multi-core Pi (anything essentially but the very first one and Pi Zero), if you have isolcpu=3 setting, the busy-waiting change should not make a difference (except using more CPU and power on that core) - that core is reserved entirely for the matrix pushing anyway, it using 35% or 100% time won't help other cores. But busy-waiting might generate more fluctuations in timings, so could result in more flicker. Anyway, things to experiment with.

[sityware]

I've experimented a lot with this. I have quite a hard time as I'm using Python and using memory hogging Selenium Chromium browser on a Pi Zero 2W

Yes the PWM mod is a must.

The double buffering so
canvas = rgb_matrix.SwapOnVSync(canvas)
graphics.DrawText(canvas ....

Important to monitor the processor usage and the relationship between.
--led-pwm-lsb-nanoseconds and --led-limit-refresh

I don't know the technicals exactly of why but if you have
--led-pwm-lsb-nanoseconds at 130 and --led-limit-refresh=100 this would use 88% cpu core
--led-pwm-lsb-nanoseconds at 130 and --led-limit-refresh=200 this would use 70% cpu core
Guessing the limit uses more cpu the more it has to stop it from reaching its max fps? So it seems better to have a higher fps as it uses less CPU.

I'm currently using --led-pwm-lsb-nanoseconds=127 and a 190 fps which uses about 74% cpu core. Only have one 64x32 display