Keep Dart callbacks for later use

[vhdirk]

The library I'm developing implements a (or several) long running task that should use the platform sensors at multiple instances. It is not limited to mobile development. As such, I have implemented this in my rust library as a trait defining several methods. Each platform provides an implementation of the trait, and so should the flutter bridge.

So in a nutshell, this is what I am trying to do (I know async traits aren't stable, but at this point, that does not matter):

pub trait MyInterface {
    async fn test(&self, name: String) -> String;
}

#[frb(opaque)]
pub struct MyImplementation {
    // pub instance: DartOpaque,
    pub cb: Arc<Mutex<Option<Box<dyn Fn(String) -> DartFnFuture<String>>>>>,
}

impl MyImplementation {
    #[frb(sync)]

    pub fn new() -> Self {
        Self {
            cb: Default::default(),
        }
    }

    pub async fn set_callback(&self, dart_callback: impl Fn(String) -> DartFnFuture<String> + 'static)
    {
        let mut cb = self.cb.lock().unwrap();
        cb.replace(Box::new(dart_callback));
    }

    pub async fn do_callback(&self, name: String) -> String {
        let cb = self.cb.lock().unwrap();
        cb.as_ref().unwrap()(name).await
    }
}

impl MyInterface for MyImplementation {
    async fn test(&self, name: String) -> String {
        self.do_callback(name).await
    }
}

I'm not particularly sure this can even work. It yields a long error that I'm not sure how to solve (see below).
Has anyone ever tried something like this before?

error[E0277]: `(dyn Fn(String) -> Pin<Box<(dyn std::future::Future<Output = String> + std::marker::Send + 'static)>> + 'static)` cannot be sent between threads safely
   --> lox-dart/src/frb_generated.rs:519:1
    |
519 | / flutter_rust_bridge::frb_generated_moi_arc_impl_value!(
520 | |     flutter_rust_bridge::for_generated::rust_async::RwLock<MyInterface>
521 | | );
    | |_^ `(dyn Fn(String) -> Pin<Box<(dyn std::future::Future<Output = String> + std::marker::Send + 'static)>> + 'static)` cannot be sent between threads safely
    |
    = help: the trait `std::marker::Send` is not implemented for `(dyn Fn(String) -> Pin<Box<(dyn std::future::Future<Output = String> + std::marker::Send + 'static)>> + 'static)`
    = note: required for `Unique<(dyn Fn(String) -> Pin<Box<(dyn std::future::Future<Output = String> + std::marker::Send + 'static)>> + 'static)>` to implement `std::marker::Send`
note: required because it appears within the type `Box<(dyn Fn(String) -> Pin<Box<(dyn std::future::Future<Output = String> + std::marker::Send + 'static)>> + 'static)>`
   --> /home/me/rust/nightly/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/alloc/src/boxed.rs:195:12
    |
195 | pub struct Box<
    |            ^^^
note: required because it appears within the type `Option<Box<(dyn Fn(String) -> Pin<Box<(dyn std::future::Future<Output = String> + std::marker::Send + 'static)>> + 'static)>>`
   --> /home/me/rust/nightly/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/option.rs:570:10
    |
570 | pub enum Option<T> {
    |          ^^^^^^
    = note: required for `Mutex<Option<Box<dyn Fn(String) -> Pin<Box<dyn Future<Output = String> + Send>>>>>` to implement `std::marker::Sync`
    = note: the full type name has been written to '/home/me/lox/target/debug/deps/lox_dart-0531c1a9e1e24cfd.long-type-11707364190590025608.txt'
    = note: 1 redundant requirement hidden
    = note: required for `Arc<Mutex<Option<Box<dyn Fn(String) -> Pin<Box<dyn Future<Output = String> + Send>>>>>>` to implement `std::marker::Send`
    = note: the full type name has been written to '/home/me/lox/target/debug/deps/lox_dart-0531c1a9e1e24cfd.long-type-6811346147299571271.txt'
note: required because it appears within the type `nfc_interface::MyInterface`
   --> lox-dart/src/api/nfc_interface.rs:15:12
    |
15  | pub struct MyInterface {
    |            ^^^^^^^^^^^
    = note: required for `flutter_rust_bridge::rust_async::RwLock<nfc_interface::MyInterface>` to implement `std::marker::Sync`
    = note: 1 redundant requirement hidden
    = note: required for `Arc<flutter_rust_bridge::rust_async::RwLock<nfc_interface::MyInterface>>` to implement `std::marker::Send`
note: required because it appears within the type `MoiArcPoolValue<flutter_rust_bridge::rust_async::RwLock<nfc_interface::MyInterface>>`
   --> lox-dart/src/frb_generated.rs:30:1
    |
30  | / flutter_rust_bridge::frb_generated_boilerplate!(
31  | |     default_stream_sink_codec = SseCodec,
32  | |     default_rust_opaque = RustOpaqueMoi,
33  | |     default_rust_auto_opaque = RustAutoOpaqueMoi,
34  | | );
    | |_^
    = note: required because it appears within the type `(usize, MoiArcPoolValue<flutter_rust_bridge::rust_async::RwLock<nfc_interface::MyInterface>>)`
    = note: required for `hashbrown::raw::RawTable<(usize, MoiArcPoolValue<flutter_rust_bridge::rust_async::RwLock<nfc_interface::MyInterface>>)>` to implement `std::marker::Send`
note: required because it appears within the type `hashbrown::map::HashMap<usize, MoiArcPoolValue<flutter_rust_bridge::rust_async::RwLock<nfc_interface::MyInterface>>, RandomState>`
   --> /rust/deps/hashbrown-0.14.3/src/map.rs:190:12
note: required because it appears within the type `HashMap<usize, MoiArcPoolValue<flutter_rust_bridge::rust_async::RwLock<nfc_interface::MyInterface>>>`
   --> /home/me/rust/nightly/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/collections/hash/map.rs:213:12
    |
213 | pub struct HashMap<K, V, S = RandomState> {
    |            ^^^^^^^
note: required because it appears within the type `MoiArcPoolInner<flutter_rust_bridge::rust_async::RwLock<nfc_interface::MyInterface>>`
   --> lox-dart/src/frb_generated.rs:30:1
    |
30  | / flutter_rust_bridge::frb_generated_boilerplate!(
31  | |     default_stream_sink_codec = SseCodec,
32  | |     default_rust_opaque = RustOpaqueMoi,
33  | |     default_rust_auto_opaque = RustAutoOpaqueMoi,
34  | | );
    | |_^
    = note: required for `std::sync::RwLock<MoiArcPoolInner<flutter_rust_bridge::rust_async::RwLock<nfc_interface::MyInterface>>>` to implement `std::marker::Sync`
note: required by a bound in `lazy_static::lazy::Lazy`
   --> /home/me/rust/nightly/registry/src/index.crates.io-6f17d22bba15001f/lazy_static-1.4.0/src/inline_lazy.rs:19:20
    |
19  | pub struct Lazy<T: Sync>(Cell<Option<T>>, Once);
    |                    ^^^^ required by this bound in `Lazy`
    = note: this error originates in the macro `__lazy_static_create` which comes from the expansion of the macro `flutter_rust_bridge::frb_generated_boilerplate` (in Nightly builds, run with -Z macro-backtrace for more info)

[fzyzcjy]

cannot be sent between threads safely

Looks like it is not Send. I guess first thing may be try impl Fn(String) -> DartFnFuture<String> + 'static + Send

[fzyzcjy]

I am not sure about thread safety; I think is is not sound

May I know a bit more? I think it may be sound, since there is no unsafe code ;) That's one reason why I love Rust - if the compiler says OK then it is usually really OK

[vhdirk]

I've made a small demo project so I can better show what I'm trying to do here: https://github.com/vhdirk/flutter_rust_transceiver

In a nutshell; I want to let the rust lib control the read/write operations from/to the platform NFC. And since there's plenty of Flutter plugins for NFC, I don't want to implement such a thing in Rust (if at all possible).

So, in that demo project, I define a trait NFCTransceive and an implementation that offloads the required trait methods to the dart callbacks (https://github.com/vhdirk/flutter_rust_transceiver/blob/main/rust/src/api/transceiver.rs).
Then, on the dart side, I just create an abstract class AbstractNFCInterface that will initialize the rust binding with the callbacks. And so for each platform, you just have to subclass AbstractNFCInterface for a platform, and it should just work (https://github.com/vhdirk/flutter_rust_transceiver/blob/main/lib/main.dart)

This seems to work, but as said, I am unsure about the thread safety of this. While only a few unsafe code blocks are defined, Send and Sync are unsafe traits to begin with 🤷 . And I have no idea how many unsafe blocks/impls are used in the flutter_rust_bridge crate.

Also, while preparing this demo project, I discovered some unexpected things:

• an exception thrown on the dart side does not get translated into an Err on the rust side, I think?
• the return type of these methods: Result<Vec<u8>, NFCTranceiveError> is interpreted as an opaque type.

[fzyzcjy]

While only a few unsafe code blocks are defined

Could you please provide a link to unsafe blocks? I quickly glanced at https://github.com/vhdirk/flutter_rust_transceiver/blob/main/rust/src/api/transceiver.rs#L12 and seems it is safe.

Send and Sync are unsafe traits to begin with 🤷 . And I have no idea how many unsafe blocks/impls are used in the flutter_rust_bridge crate.

The impl fn is related to things like https://github.com/vhdirk/flutter_rust_transceiver/blob/e8b9db26abee027d1ee40620eb3000356cff8024/rust/src/frb_generated.rs#L51

and thus things like https://github.com/vhdirk/flutter_rust_transceiver/blob/e8b9db26abee027d1ee40620eb3000356cff8024/rust/src/frb_generated.rs#L67

thus you can see it is just a pretty normal rust function. It is the compiler that thinks adding Send/Sync is OK, and frb does not manually mark that. So I guess at least for send/sync thing it looks reasonable.

There do exist some unsafe code, but they are minimal and necessary, e.g. you cannot call between Rust and external language (Dart) without any unsafe!

an exception thrown on the dart side does not get translated into an Err on the rust side, I think?

If so, just create a separate issue and it is a bug (or a feature that needs to be implemented)

the return type of these methods: Result<Vec, NFCTranceiveError> is interpreted as an opaque type.

Also a bug or not implemented feature, feel free to create issues!
Btw if you mean this https://github.com/vhdirk/flutter_rust_transceiver/blob/e8b9db26abee027d1ee40620eb3000356cff8024/rust/src/api/transceiver.rs#L12, try to directly write down Result instead of using type = ... and see whether it works.

Btw, I am thinking to enhance frb such that it can auto translate a trait on Rust to be an abstract class on Dart. Then there is no need for such manual glue. (But maybe no time for this feature in short time)

[vhdirk]

As for unsafe blocks, there's 2 places:

• https://github.com/vhdirk/flutter_rust_transceiver/blob/942a8a792fc8154e9c656f0570df611e1d89289f/rust/src/frb_generated.rs#L49
• https://github.com/vhdirk/flutter_rust_transceiver/blob/942a8a792fc8154e9c656f0570df611e1d89289f/rust/src/frb_generated.rs#L128

For that Future<Result<...>>, I had it written out in full before, it wasn't any different. I'll create issues for it soon.

Having a trait result into an abstract class would be quite cool!

[fzyzcjy]

It is unsafe because IMHO they have to - roughly speaking, it is unsafe if you want to get values from a raw pointer. As for the Dart Object, IIRC it is doing something similar (cast from raw numbers into raw pointer and read its value)