Bin Count and Latency Questions

[stellarpower]

Hi,

I have been trying out the SDFT with some of my data, but the results I am seeing appear largely similar with the FFT, and I'm not sure if I have missed something in how I call into the library, or if what I am seeing is inherent, and my understanding of the algorithm is a bit off.

The basic rolling Short-Time FFT isn't working for me, primarily because there's too much of a tradeoff between the bin count and the roll-on/roll-off time. I guess my input signal is highly non-stationary. I know latency can mean a couple of different things - but in my case, at 512Hz input sample rate, the computational cost isn't so much of a problem, I think we can ignore it, but it's the fact that the response time between seeing the feature in the input data and seeing the relevant bandpower increase/decrease is too great. I can get better time resolution but poor frequency resolution with a shorter window, and vice versa:

Here are a few examples just to illustrate what I am seeing; the input signal is in white, and those bursts visible are about 8-12 Hz. The SDFT is in blue, and the purple is the STFT built into the software I am using - this is the band amplitude for 8-12Hz, so in an ideal world, they would line up. These are calculated online in realtime - although I have simulated the data offline and see largely the same results - with a reasonably high bin count, there is about a second of delay between the non-causal reference FFT and the SDFT or the STFT. All use a Hanning window.

A significant problem is as the data is processed online, the calculation is always after the fact - with a symmetric window, by the time the sample I care about has reached the middle of the window, it's too late to do anything useful with the bandpower, as the input signal has changed. Obviously I can't work with data that's not arrived and been sitting around for a little bit of time, it's going to have to be a tradeoff, but if I were to plot the offline windowed FFT as the reference on the same axes, the peaks would obviously be shifted to the left, as this can look into the future, and my comprehension was that the recursive calculations in the sliding DFT should get us closer to the reference FFT, allowing us to respond more rapidly to a change in the input signal than an FFT. Which perhaps is not a correct understanding of the algorithm.

I am also wondering, is it possible the non-linear scale in the QDFT would help here? As the bands I care about make up only a relatively short portion of those in the input series, and it seems spacing between the start of the feature in the time series and the hump of the bandpower seems to be linear with the number of bins. This of course makes intuitive sense, however I was under the impression that greater frequency resolution without hitting the latency disadvantages of a longer window in the STFT was one reason we might use the SDFT. I am looking to be able to resolve low frequencies between 1.0Hz and about 3Hz precisely, but also need to be able to react to changes in bandpower quickly. Given that the "attack" time for a smaller number of bins with the SDFT looks better, is it likely that ignoring the frequencies we don't care about but calculating for a similar number of bins with the QDFT would help here?

Do you know if I'm limited here to an asymmetric window? This was my first thought some time ago, but I don't see much mention online, however, my exact situation is that I need the algorithm to place weight on the sample just received and not worry as much about what happened in the past. So this does make sense as the solution.

Or perhaps am I doing something wrong, and none of this should be seen? Using some of the code from the Analysis python example, an offline imshow plot of the SDFT versus the STFT provided in the code seems to line up better along the time axis - the SDFT takes a bit of time to warm up, and then after 8 seconds or so of data, there's reasonably good lineup of the data. But I haven't yet broken apart the sliding window the implementation provided is using, and maybe it is operating more like an online than an offline FFT.

Any thoughts on what I may be doing wrong?

Thanks a lot!

[jurihock]

First of all, I have absolutely no experience with infrasonics. In my understanding, a frequency detector needs more time to detect a slow wave than the fast one. Even SDFT has a delay line determined by the DFT size. In conjunction with the sample rate, the delay line must be large enough to cover the desired wave period. And since it's a "delay line", you'll get the frequency estimation with a "delay". The DSP is always about "what happened in the past". The slower the wave, the larger is the time delay to detect it. The same thing applies to the QDFT too. The difference between SDFT and QDFT is the frequency scale. The logarithmic QDFT is more applicable for "music" analysis, because that's the way we perceive music.

Please don't get confused by my statement "real-time low latency analysis and synthesis capability". There is an asterisk note "compared to STFT latency". It's about the algorithmic analysis-synthesis latency in real-time applications (e.g. audio effects) and not the physical latency.

So your question seems to me to be somewhat underspecified. What do you plan to do within 1-3 Hz range?

Just detecting the total power?

• Apply a bandpass filter and measure the power.

Just detecting the power of few well-known frequencies?

• Apply a bandpass filter bank and measure the power
• Implement a single-bin SDFT (search e.g. for papers by Rick Lyons; this SDFT implementation is about a "wideband" analysis)

Detecting the power of N unknown frequencies?

• Hm, maybe apply a lowpass, decimate (e.g. virtually reduce the sampling rate), and then try to feed it in a "wideband" analyzer

[stellarpower]

Thanks a lot for the detailed answer!

Even SDFT has a delay line determined by the DFT size.

Okay, so this confirms why it had greater lag with higher bin counts. And then it's obviously not an ideal solution, but at least the QDFT might perform better, because I can resolve the bins I care about with the same sort of accuracy but with a smaller matrix size (albeit on a logarithmic scale - which I'd argue is not unlike music, and would be better for EEG too).

In conjunction with the sample rate, the delay line must be large enough to cover the desired wave period.

Yeah, I agree with this. I kept thinking I was confusing it with the Nyquist limit because I couldn't justify it mathematically, but my intuition says of course to detect slow waves you by definition need a relatively large window. If nothing else you need to be able to have resolved higher frequencies to confirm that the samples are not contributing to those, and those can be detected quickly. A bit like how the DC bias is the average of the whole signal - it's like what is left over that hasn't been claimed by higher frequencies.

It's about the algorithmic analysis-synthesis latency in real-time applications (e.g. audio effects) and not the physical latency.
Thanks, as I often see the two used interchangeably. I've also seen projects described as "real-time" except they are not at all optimised for use on continual data - just that it is assumed in most scenarios on a reasonable computer they will operate fast enough to clear the incoming data.

So it's reducing the computational cost - and proving itself equivalent to the "online" STFT (with some lag) rather than a heuristic method for estimating the "offline" STFT (that can see samples after the one in the centre of the window, and which I can get after the data have been recorded). I figured this might be the case once I was seeing these patterns - but wanted to rule out anything I might have messed up!

I did try the bandpass option prior to this, but this was also introducing a lag for me, that in the end also looked identical to the FFT once I'd rectified and smoothed. I barely know much about filter design - but I asked someone who is knowledgeable who thought I'd more or less see the same in any setup - some amount of reasonably significant phase shift. We were considering if a neural net could learn to look ahead in the signal and predict what would be seen as an alternative, Would be easy to train at least.

I'm trying the SWIFT at the moment to see if the asymmetry helps at all. I think the approach the wavelet transform takes is also recommended, so may revisit it. Currently even getting the lag down by about half would be a good step forwards - so the QDFT with a smaller number of closely-spaced bins could be an iterative improvement.

Thanks again!

[jurihock]

As I understand, the SWIFT is patented...

Theoretically, you could also modify the QDFT to use a custom frequency scale, e.g. linear. The analysis frequencies are determined here. The factors alpha and beta must be considered too. Unfortunately, I don't remember me on details in section 6 spontaneously.

The particular difference between SDFT und QDFT is, the QDFT introduces a frequency bin dependent delay line and the SDFT a common delay line for all frequency bins.

As described in the original paper, there is also an option to virtually shift the analysis time point, which corresponds to the latency argument in QDFT.

[stellarpower]

The particular difference between SDFT und QDFT is, the QDFT introduces a frequency bin dependent delay line and the SDFT a common delay line for all frequency bins.

That is interesting, thank you. I will take some time later to break it apart and understand better how it is working. That might suit well for tailoring the algorithm differently for different bands I will want to filter in the longer term.

As described in the original paper, there is also an option to virtually shift the analysis time point, which corresponds to the latency argument in QDFT.

And this also looks good to investigate. From what I could tell, in the SDFT it seems the latency parameter only affects the inverse transform (synthesis) - but this parameter did seem very relevant. Looking at the code, it seems to be used for the analysis transform in the QDFT, is that right? I think we could definitely take a hit of 33% accuracy loss if that improved the latency.

As of last night, a single-bin QDFT seems to be working quite nicely for 7.5-12.5Hz, as a stop-gap solution to pick out just that one band:

The white is the raw trace; that's the QDFT output in the orange in the middle; and smoothed through a lowpass in yellow at the bottom. Two FFT settings are the top two - so much tighter latency than those! From memory, I think a single bin should operate like the equivalent of one filter(?), so that would suggest the same could be achieved with an appropriately-designed filter, but afraid this is beyond my knowledge! Will probably still try training an LSTM later as I have been highly recommended this.

I had seen that patent also - from memory software patents are not enforced in a lot of countries, was interested to find out. If so, this is only for personal use right now anyway. But good to be aware of.

Thanks!

[jurihock]

Yes, the SDFT "latency" parameter affects the synthesis response only.

According to the time-shifting property of the Fourier transform, the synthesis time point can be (circularly) shifted within the time domain window, and is centered by default. In case of the hann window function, the amplitude of the windowed time domain signal fragment is then suppressed by the falling sidelobes of the window function. Thus, decreasing of the SDFT response latency unavoidable degrades the SNR. This feature is basically relevant for real-time applications, e.g. audio effects, but useless for signal analysis.

The meaning of the QDFT "latency" is different. It's shifting the analysis time point of particular frequency bins. Which can be useful or not, depending on the purpose. E.g. time synchronous harmonic analysis will not profit from that...

[stellarpower]

Apologies for the delay. Thanks for all the detailed explanations! I'll probably come back to understanding this later. I have some other sources of latency in my pipeline I've been looking at, and may also try a Neural net architecture to see if we can predict things any faster. I can keep you posted on that if it's of any interest. Some colleagues think it could be very promising, although if I have performance concerns at 512Hz, for audio I would definitely be worried about the inference speed!