Implement support for FTDI virtual COM port over WebUSB

[reillyeon]

No description provided.

[ForbiddenEra]

Just noticed this after posting my issue;

https://github.com/Shaped/webusb-ftdi

As I said, not done yet; I will probably finish it soon; Linux kernel driver source is a good start for info as FTDI doesn't seem to want to help the open source community in any way.

I may actually need to finish this ASAP for a project as WebSerial doesn't seem to be available in Chromium on Linux on Arm64 for some reason and I really don't know why.

[reillyeon]

The Web Serial API implementation should work in Chromium on ARM64 Linux. Even though there are no official Chrome builds for ARM64 the implementation is fully supported on ARM64 Chrome OS devices which share the same code.

[ForbiddenEra]

Was just going through some old notifications, I never did notice any responses.

The one problem I with implementing this at all is that, at least on Windows, and for FTDI devices, maybe others, you have to make sure that an existing driver isn't loaded, generally using Zadig or similar to revert to WinUSB or libusb (memory failing me on which specifically) .. I'm not sure that this is an issue on other operating systems; IIRC it wasn't but it could still require whatever FTDI-specific driver to be unloaded at least.

Most average users aren't going to likely understand this or want to do it. It also has the side-effect of preventing them from using the device in other software expecting a serial port until the FTDI driver is reinstated.

Not sure if WebSerial is planned to land in Chromium but if it is, it would completely sidestep that issue as the FTDI driver just provides a serial port.

With that in mind, I'm not sure how relevant it'd be to add FTDI support. If WebSerial isn't going to happen or happen anytime soon then perhaps it still might be worth adding, at least with a documented note on the driver issue for those who might really want to use it.

Open to thoughts on this, if there's still interest I can look at pushing a PR for FTDI support.

[reillyeon]

Was just going through some old notifications, I never did notice any responses.

The one problem I with implementing this at all is that, at least on Windows, and for FTDI devices, maybe others, you have to make sure that an existing driver isn't loaded, generally using Zadig or similar to revert to WinUSB or libusb (memory failing me on which specifically) .. I'm not sure that this is an issue on other operating systems; IIRC it wasn't but it could still require whatever FTDI-specific driver to be unloaded at least.

Most average users aren't going to likely understand this or want to do it. It also has the side-effect of preventing them from using the device in other software expecting a serial port until the FTDI driver is reinstated.

That is correct. The intended use case for this library is platforms which don't provide a driver for a given serial interface chip or where it is worth the effort to replace it with a different implementation.

Not sure if WebSerial is planned to land in Chromium but if it is, it would completely sidestep that issue as the FTDI driver just provides a serial port.

The Web Serial API has indeed been available in Chromium since March 2021 and is the preferred solution. I still have no explanation for why you were having trouble using it on ARM64 Linux.

With that in mind, I'm not sure how relevant it'd be to add FTDI support. If WebSerial isn't going to happen or happen anytime soon then perhaps it still might be worth adding, at least with a documented note on the driver issue for those who might really want to use it.

Open to thoughts on this, if there's still interest I can look at pushing a PR for FTDI support.

Platforms like Android provide neither USB serial drivers or an API for applications to use them. Now that the Web Serial API is available in browsers this library is still useful for supporting such platforms.

[ForbiddenEra]

That is correct. The intended use case for this library is platforms which don't provide a driver for a given serial interface chip or where it is worth the effort to replace it with a different implementation.

AFAIK many platforms do have an FTDI driver available. The 'official' driver is only available on Windows IIRC though.

On Windows, the user would have to have installed the FTDI driver previously for a serial port to be available, the WebUSB driver could be useful in these cases.

On other platforms, from what I can gather (at least Linux-based and OS X) the driver should be built into the kernel in most cases, IIRC.

The Web Serial API has indeed been available in Chromium since March 2021 and is the preferred solution. I still have no explanation for why you were having trouble using it on ARM64 Linux.

Note that my original post was made Feb 2021; it wouldn't have been available juuust yet. :-)

Though, that would've likely not been a full solution for me anyway; most devices I have use an FTDI converter, whether they're breakout boards for the converter, microcontroller dev boards or finished products from various manufacturers. The only devices I can think of OTOH that use USB-CDC actually are microcontrollers that have built-in USB implementations. My specific use-case was likely going to use FTDI, although the hardware wasn't finalized at that point and it's possible I'd have switched to a microcontroller using built in USB instead of adding a separate FTDI chip.

Platforms like Android provide neither USB serial drivers or an API for applications to use them. Now that the Web Serial API is available in browsers this library is still useful for supporting such platforms.

So none of the drivers/modules from the Linux kernel (eg, from https://github.com/torvalds/linux/blob/master/drivers/usb/serial/)are available in Android at all? I had assumed that these drivers were generally built into the kernel directly and not being loaded as a kernel module; of course, one could configure the kernel to not include them and from your statement I can suppose that's the case with Android - sorry, my low-level Android knowledge lacks here.

If that's the case, then indeed it would still be useful, though I'm pretty sure that at least a few apps exist for Android that use FTDI devices; while it's true they could've implemented a driver themselves, it wasn't particularly straight forward for me and asking FTDI for help got me nowhere, even if all the information I needed was already open-sourced in the Linux kernel.

I haven't tested it on Android yet and there was an issue opened on my driver for Android; I'll have to check into that.

If indeed it will still be useful for Android users (and part of my initial reason was to support FTDI devices on Android in a browser for firmware updates, etc) then I'm happy to try and invest some time in the near future to make sure it works with Android, perhaps implement some sort of tests (though for most important tests I'm not entirely sure how well that would work without emulating a device or somehow having one connected to a CI runner, heh) and then look at making a PR to include it within the polyfill here.

Thanks for responding; since from what you say Android shouldn't have that driver and it can still be useful, then I'll look into testing on Android and shoring up the driver to a less alpha-quality state and then see about PRing it into the polyfill.

[reillyeon]

Though, that would've likely not been a full solution for me anyway; most devices I have use an FTDI converter, whether they're breakout boards for the converter, microcontroller dev boards or finished products from various manufacturers. The only devices I can think of OTOH that use USB-CDC actually are microcontrollers that have built-in USB implementations. My specific use-case was likely going to use FTDI, although the hardware wasn't finalized at that point and it's possible I'd have switched to a microcontroller using built in USB instead of adding a separate FTDI chip.

The in-browser implementation of the Web Serial API uses the platform's serial API and so if there is an FTDI driver available then it will work.

Platforms like Android provide neither USB serial drivers or an API for applications to use them. Now that the Web Serial API is available in browsers this library is still useful for supporting such platforms.

So none of the drivers/modules from the Linux kernel (eg, from https://github.com/torvalds/linux/blob/master/drivers/usb/serial/)are available in Android at all? I had assumed that these drivers were generally built into the kernel directly and not being loaded as a kernel module; of course, one could configure the kernel to not include them and from your statement I can suppose that's the case with Android - sorry, my low-level Android knowledge lacks here.

I don't know exactly what the Android kernel configuration is but in addition to the drivers definitely being missing on some devices there's an even worse problem which is that sometimes they aren't missing but because the Android SDK doesn't include a serial API we can't actually use them, even though now they've claimed the USB interface. Chromium issue 1099521 tracks allowing WebUSB on Android to forcibly claim interfaces on such devices to work around the problem.

If that's the case, then indeed it would still be useful, though I'm pretty sure that at least a few apps exist for Android that use FTDI devices; while it's true they could've implemented a driver themselves, it wasn't particularly straight forward for me and asking FTDI for help got me nowhere, even if all the information I needed was already open-sourced in the Linux kernel.

That is exactly how Android apps that use USB serial devices (CDC, FTDI or otherwise) work.

Thanks for responding; since from what you say Android shouldn't have that driver and it can still be useful, then I'll look into testing on Android and shoring up the driver to a less alpha-quality state and then see about PRing it into the polyfill.

Looking forward to the PR. Thanks!

[ForbiddenEra]

Looking forward to the PR. Thanks!

Thanks for your responses and clarification; makes sense that different devices might have different configurations and such.

The issue you mentioned could be a definite problem for some people but I guess this would also apply to apps that use such devices. I haven't yet but I'll have to look at the linked issue and perhaps track it; I'm curious that it's a Chromium issue if it's seemingly an Android problem but again I haven't yet read the issue and am not super familiar with lower level Android, from your statement I'm guessing perhaps it may be possible for an app (perhaps with the right permissions or something) to "forcibly claim .. such devices" as you say.

At least if WebSerial is available now that should be a workable/preferred alternative on Windows at least, though it'll be up to developers to determine what's available and fallback as required although that may have some UI/UX oddities/considerations being that using WebSerial the device would be addressed as a COM port and otherwise by deviceId/usb port id.

I'm a bit surprised to learn that anyone would actually implement their own driver in their app; not particularly because it's difficult but because FTDI is seemingly rather strict about giving out details. When writing this driver, I contacted them and while they seemed possibly initially interested, I wasn't able to get any information from them, whether the information kept under NDA and whether or not the actual implementation was going to be released opensource or not - which was semi-frustrating as it turned out that the main information I needed was already implemented in the Linux kernel whose source seemingly indicates at the time (albeit that was probably 20+ years ago) they weren't against sharing. Although a hardware vendor buying enough lots of chips or perhaps especially one licensing their own ID through FTDI would likely have more pull in this regard compared to someone in the community making opensource mainly targeted at my own low volume hardware and at tinkerers with Arduinos.

Admittedly perhaps the Linux driver may have been more 'worthy' (and I'm guessing they had no interested in making their own at least at the time) but I at least personally thought that with WebUSB being a thing and web apps becoming more and more of a standard way to run apps with more APIs becoming available for traditionally native things like hardware access.

Back to the topic at hand - I'll try and find some time to clean up and finalize the code, I can't make any commitments on any time frame but if it can be/will be useful to more than a few people (which is seemingly the case) then that's definitely some motivation to get 'er done.

Since you mention the primary potential usefulness would be on Android devices, I'll do some testing and try to make it work on some of the Android devices I have access to and perhaps the emulator as well if possible - if that's where it's mostly useful then I don't see much value in making a PR if it doesn't work on Android.

Hopefully I'm able to find out whether any USB drivers are loaded or pinned to a device without root.

Thanks again for your response; I'll come back and update if/when I make some progress on the Android front if perhaps I'm looking for feedback or just as an update, otherwise I'll just make a PR when suitable.