Another USB audio question... tud_audio_n_fb_set feedback endpoint nearly always busy?

[kf6gpe]

Sorry for all of the questions. I hope to give back to the community at some point once I know what I'm doing!!

I've gotten my descriptors all set up (with loads of thanks to @HiFiPhile !) and audio is happily streaming isochronously again to my Windows 11 host. I've implemented a periodic call to tud_audio_n_fb_set -- I'm on an STM32H7, so I can't use SOF, but I'm calling it periodically (once every 48 frames at 48KHz, so that's a 1ms tick), and I'm seeing that after the first time tud_audio_n_fb_set feedback is invoked, the feedback endpoint is busy --- audiod_fb_send isn't being invoked. Once the audio stream from the Windows host stops and starts sending 0's, I don't call tud_audio_n_fb_set. When I start playback of audio again, the same thing happens --- a single call to audiod_fb_send, and then the feedback endpoint is busy until the audio stops.

I thought at first it might be a descriptor problem (me crossing the wires and attempting to write to one of the isochronous descriptors). Here's my descriptor assignments:

// Endpoints
//	The high bit indicates direction; set is incoming.
//  See USB Interface Specification, Section 9.6.6 

// Output endpoint
#define EPNUM_AUDIO_OUT		(0x01)
// Input endpoint
#define EPNUM_AUDIO_IN		(0x02 | 0x80)
// Feedback endpoint -- the output endpoint in the opposite direction
#define EPNUM_FEEDBACK_OUT	(0x03 | 0x80)

uint8_t const desc_configuration[] =
{
    // Interface count, string index, total length, attribute, power in mA
    TUD_CONFIG_DESCRIPTOR(1, ITF_NUM_TOTAL, 0, CONFIG_TOTAL_LEN, 0x00, 100),

	// string index, EP Out & EP In address, Feedback ep
    TUD_AUDIO_HEADSET_STEREO_FEEDBACK_DESCRIPTOR(2, EPNUM_AUDIO_OUT, EPNUM_AUDIO_IN, EPNUM_FEEDBACK_OUT)
};

That seems OK, though.

I can't see why what feedback I write down the endpoint would cause any problems, but FWIW, here's what it looks like.

void TinyUSB::SetSampleFeedback(const SInt32 usbSamplesSinceLast, const UInt32 audioSamplesSinceLast)
{
#if CFG_TUD_AUDIO_ENABLE_FEEDBACK_EP
	float feedbackRatio = (float)usbSamplesSinceLast/(float)audioSamplesSinceLast;
	UInt32 feedback = feedbackRatio * 65536;	// Convert to 16.16 notation

	TU_LOG2("SetSampleFeedback usb %d audio %d, ratio %f",
		usbSamplesSinceLast, audioSamplesSinceLast, feedbackRatio);

	tud_audio_n_fb_set(0, feedback);
#endif
}

So I don't think that my feedback routine should be causing any problems.

FWIW, I'm at tip-of-tree as of this post. Previously I'd been using a version I'd statically copied from June 2023, which shows the same behavior.

Advice? Comments? General laughter (understandable!) at my misunderstanding the API?

[kf6gpe]

Aha! So I was mistaken about always busy.

I hadn't realized that audiod_xfer_c also calls audiod_fb_send, and that the endpint isn't busy at that point.

Should have sleuthed more and posted less.

Still curious to see if anyone has any general advice about Windows not getting -- or at least not acting on --- the feedback information. I'm going to see about trying to get Windows logs now, but it looks tricky, requiring specific versions of Windows (see this blog post. I am hoping to be able to do that, so that I can see what Windows thinks about all of this, rather than just using Wireshark to inspect the packets; I trust TinyUSB enough to know that it's likely sending the right stuff over the wire.

[HiFiPhile]

Windows not getting -- or at least not acting on --- the feedback information

What makes you think Windows doesn't acting on feedback, FIFO still overflow/underflow or something else ?

[kf6gpe]

Thanks, @HiFiPhile! Yeah, still getting FIFO underflows.

I'm in the process of setting up a macOS test bed now to see how it behaves. Fingers crossed that it tells me something useful. I'm also setting up a Windows 10 host per https://matthewvaneerde.wordpress.com/2017/10/23/how-to-gather-and-read-logs-for-microsofts-usb-audio-2-0-class-driver/ to see about doing some raw logging to see what Windows thinks about what I'm sending. I read someplace that Windows is picky about the packet window size it gets from the client, and I imagine that it could be that it's not happy with whatever ratio I'm sending (both the ratio value as well as the format; it looks like it's 16.16 for FS, which I'm running) and something else for HS, which we eventually want to support. I see CFG_TUD_AUDIO_ENABLE_FEEDBACK_FORMAT_CORRECTION for conversion when using tud_audio_feedback_update.

Speaking of tud_audio_feedback_update, am I safe to assume that that's clock cycles it takes, and really only used for interrupt signaling of feedback? I'm using tud_audio_n_fb_set, as I mentioned, as I don't have easy use of SOF on this MCU (although at some point I might want to investigate it, I suppose.)

Thanks again. 

[kf6gpe]

The plot thickens!

It looks like I'm passing bad feedback to the windows driver. I haven't yet gotten windows logging set up on the PC (needed to install an older version of Windows on a test machine) but I'm consistently seeing that my feedback value is outside the range 0x2c.0-0x2d.0 (44.0-45.0)., which I gather taking from #1234 (comment) isn't helping the situation.

[kf6gpe]

So, I definitely misunderstood the units feedback is in. I'm still having problems, but getting closer: I understand now that feedback is samples deviation in an audio frame, pinned to +1/-1 audio frame for Windows. I think that's right, yes?

[HiFiPhile]

it's 16.16 for FS, which I'm running) and something else for HS

For HS it's the same thing, +-1 slot.

        uint32_t const frame_div  = (TUSB_SPEED_FULL == tud_speed_get()) ? 1000 : 8000;
        audio->feedback.min_value = (fb_param.sample_freq/frame_div - 1) << 16;
        audio->feedback.max_value = (fb_param.sample_freq/frame_div + 1) << 16;

as I mentioned, as I don't have easy use of SOF on this MCU (although at some point I might want to investigate it, I suppose.)

Long story short, currently there is feedback calculation included in UAC driver which use the method of MCLK counting defined in USB 2.0 specification 5.12.4.2.

tinyusb/src/class/audio/audio_device.c

Line 2190 in a239080

#if CFG_TUD_AUDIO_ENABLE_EP_OUT && CFG_TUD_AUDIO_ENABLE_FEEDBACK_EP

However in real it will work poorly since MCLK can't be measured error free, and the drift will let FIFO overflow/underflow eventually.
As I know the only reliable method is adjusting feedback value based on FIFO level, eg. send feedback_min when FIFO is full and feedback_max when FIFO is empty.

[kf6gpe]

Excellent, thank you! Yes, after I posted I found the frame-div code.

I think at least part of my problem is that I'm sending values other than feedback_min and feedback_max; I was trying to scale the values.

Back to the code!

[kf6gpe]

Worked out what I needed to with help from you, @HiFiPhile !

For others looking for similar info, here's what worked for me:

void TinyUSB::SetSampleFeedback(const UInt32 sampleRate)
{
#if CFG_TUD_AUDIO_ENABLE_FEEDBACK_EP
	// From https://learn.microsoft.com/en-us/windows-hardware/drivers/audio/usb-2-0-audio-drivers
        // Signaling must be between +/- one audio slot in 16.16 format
       // Taken from audio_device.c
	UInt32 const frameDiv  = (TUSB_SPEED_FULL == tud_speed_get()) ? 1000 : 8000;
	UInt32 minValue = (kOutputSampleRate / frameDiv - 1) << 16;
	UInt32 maxValue = (kOutputSampleRate / frameDiv + 1) << 16;
	UInt32 feedback = (kOutputSampleRate / frameDiv) << 16;
    if (sampleRate < m_sampleRate)
    {
		feedback = minValue;
    }
    else if (m_sampleRate < sampleRate)
    {
		feedback = maxValue;
    }

	tud_audio_n_fb_set(0, feedback);
#endif
}

This is called periodically during our audio loop where we carry samples from USB out to our audio chain. As you say, this just pokes a "speed up / slow down" message to Windows every USB frame.

[kf6gpe]

OK, hopefully the last question here.

On a FS interface, is there a good way to switch between the feedback formats for explicit feedback?

Windows expects 16.16. I get that. It looks very much like macOS and related Apple products adhere closer to the USB spec, and want 14.10 for the clock feedback.

Any thoughts as to how I should handle this for a cross-platform audio device?

Thanks!

[HiFiPhile]

I think it won't be easy, there are maybe some OS detection techniques such as Windows ask for MSOS descriptor. But I'm not sure if it's reliable enough.

[kf6gpe]

Thanks! It does not -- what we did was to use the implicit feedback signal to adjust our sample rate converter by a few hertz in each direction at a time. It seems to be working pretty well.