Async (#10)

* First pass at async file, still need futures * Futures test passed! I don't believe it... need some more complex tests * Update src/templates/UniffiHandleMap.java Co-authored-by: Bob Wall <[email protected]> * Update src/templates/UniffiHandleMap.java Co-authored-by: Bob Wall <[email protected]> * Checkpointing to update another branch * Going to require implementing callbacks anyway * So close to working async_traits Right now `CallbackTemplate.java` isn't being included in the generation, causing all build failures * One compilation error away... from probably the next batch of compilation errors! * Compiles, runtime error and need to fill out tests * Structures don't allow boxed type class fields need to make everything primitives if possible. If not, need to make primitives around structures. * Appropriate documentation is important. Structures require a no-args constructor in almost all cases. The Kotlin one happened to be working because all args were defaultable, which from the Java view works as a no-args constructor. Nothing about this is mentioned in the Structure or JNA documentation. * Sooooooo close. Something going on with cancellation looks like it's the cause of the only remaining failures (commented out). * Only cancellation failing Async callbacks weren't waiting on `get` correctly. Once that was fixed needed to unwrap `ExecutionException`s so they could be passed to Rust callbacks correctly. * Normal future cancellation fixed. * Fixed trait future cancellation too * Clean up some TODOs after PR comment pass * What if CI just needs more time? * Add a little more info to assert fail * Only run the failing test * Touch to rebuild * Add all tests back in * Bytes type and coverall test fixture (#16) * Passing tests, coverall generation working Still need to complete coverall tests * Checkpointing because I don't want to lose it * Finished all coverall tests * rogue println --------- Co-authored-by: Bob Wall <[email protected]> Co-authored-by: Colt Frederickson <[email protected]>
IronCoreLabs · Jul 11, 2024 · 61e2d40 · 61e2d40
1 parent dc8113f
commit 61e2d40
Show file tree

Hide file tree

Showing 31 changed files with 3,381 additions and 271 deletions.
diff --git a/Cargo.lock b/Cargo.lock
diff --git a/Cargo.toml b/Cargo.toml
@@ -21,6 +21,7 @@ clap = { version = "4", default-features = false, features = [
   "cargo",
 ] }
 heck = "0.5"
+once_cell = "1.19.0"
 paste = "1"
 regex = "1.10.4"
 serde = "1"
@@ -35,7 +36,10 @@ glob = "0.3"
 itertools = "0.13.0"
 uniffi-example-arithmetic = { git = "https://github.com/mozilla/uniffi-rs.git", branch = "main" }
 uniffi-example-custom-types = { git = "https://github.com/mozilla/uniffi-rs.git", branch = "main" }
+uniffi-example-futures = { git = "https://github.com/mozilla/uniffi-rs.git", branch = "main" }
 uniffi-example-geometry = { git = "https://github.com/mozilla/uniffi-rs.git", branch = "main" }
 uniffi-example-rondpoint = { git = "https://github.com/mozilla/uniffi-rs.git", branch = "main" }
+uniffi-fixture-coverall = { git = "https://github.com/mozilla/uniffi-rs.git", branch = "main" }
+uniffi-fixture-futures = { git = "https://github.com/mozilla/uniffi-rs.git", branch = "main" }
 uniffi-fixture-time = { git = "https://github.com/mozilla/uniffi-rs.git", branch = "main" }
 uniffi_testing = { git = "https://github.com/mozilla/uniffi-rs.git", branch = "main" }
diff --git a/README.md b/README.md
@@ -5,6 +5,10 @@
 * Java 20+: `javac`, and `jar`
 * The [Java Native Access](https://github.com/java-native-access/jna#download) JAR downloaded and its path added to your `$CLASSPATH` environment variable.
 
+## Notes
+
+- failures in CompletableFutures will cause them to `completeExceptionally`. The error that caused the failure can be checked with `e.getCause()`. When implementing an async Rust trait in Java, you'll need to `completeExceptionally` instead of throwing. See `TestFixtureFutures.java` for an example trait implementation with errors.
+
 ## Unsupported features
 
 * Defaults aren't supported in Java so [uniffi struct, method, and function defaults](https://mozilla.github.io/uniffi-rs/proc_macro/index.html#default-values) don't exist in the Java code. *Note*: a reasonable case could be made for supporting defaults on structs by way of generated builder patterns. PRs welcome.
@@ -14,3 +18,9 @@
 We pull down the pinned examples directly from Uniffi and run Java tests using the generated bindings. Just run `cargo t` to run all of them.
 
 Note that if you need additional toml entries for your test, you can put a `uniffi-extras.toml` as a sibling of the test and it will be read in addition to the base `uniffi.toml` for the example. See [CustomTypes](./tests/scripts/TestCustomTypes/) for an example.
+
+## TODO
+
+- optimize when primitive and boxed types are used. Boxed types are needed when referencing builtins as generics, but we could be using primitives in a lot more function arguments, return types, and value definitions.
+- methods that return `Result<T, SpecificError>` in Rust should probably `T blah() throws SpecificException` in Java. As is, there are a lot of hard to handle `RuntimeException`s and the same thing needs to be done when someone implements a trait in Java (see `TestFixtureFutures.java`).
+- our use case almost certainly requires older Java versions than 20/21. Investigate supporting back to Java 8, which seems to be the common library target.
diff --git a/src/gen_java/callback_interface.rs b/src/gen_java/callback_interface.rs
@@ -0,0 +1,34 @@
+/* This Source Code Form is subject to the terms of the Mozilla Public
+ * License, v. 2.0. If a copy of the MPL was not distributed with this
+ * file, You can obtain one at http://mozilla.org/MPL/2.0/. */
+
+use super::CodeType;
+use crate::ComponentInterface;
+
+#[derive(Debug)]
+pub struct CallbackInterfaceCodeType {
+    id: String,
+}
+
+impl CallbackInterfaceCodeType {
+    pub fn new(id: String) -> Self {
+        Self { id }
+    }
+}
+
+impl CodeType for CallbackInterfaceCodeType {
+    fn type_label(&self, ci: &ComponentInterface) -> String {
+        super::JavaCodeOracle.class_name(ci, &self.id)
+    }
+
+    fn canonical_name(&self) -> String {
+        format!("Type{}", self.id)
+    }
+
+    fn initialization_fn(&self) -> Option<String> {
+        Some(format!(
+            "UniffiCallbackInterface{}.INSTANCE.register",
+            self.id
+        ))
+    }
+}
diff --git a/src/gen_java/mod.rs b/src/gen_java/mod.rs
@@ -2,6 +2,7 @@ use anyhow::{Context, Result};
 use askama::Template;
 use core::fmt::Debug;
 use heck::{ToLowerCamelCase, ToShoutySnakeCase, ToUpperCamelCase};
+use once_cell::sync::Lazy;
 use serde::{Deserialize, Serialize};
 use std::{
     borrow::Borrow,
@@ -13,6 +14,7 @@ use uniffi_bindgen::{
     interface::*,
 };
 
+mod callback_interface;
 mod compounds;
 mod custom;
 mod enum_;
@@ -72,6 +74,64 @@ trait CodeType: Debug {
     }
 }
 
+// taken from https://docs.oracle.com/javase/specs/ section 3.9
+static KEYWORDS: Lazy<HashSet<String>> = Lazy::new(|| {
+    let kwlist = vec![
+        "abstract",
+        "continue",
+        "for",
+        "new",
+        "switch",
+        "assert",
+        "default",
+        "if",
+        "package",
+        "synchronized",
+        "boolean",
+        "do",
+        "goto",
+        "private",
+        "this",
+        "break",
+        "double",
+        "implements",
+        "protected",
+        "throw",
+        "byte",
+        "else",
+        "import",
+        "public",
+        "throws",
+        "case",
+        "enum",
+        "instanceof",
+        "return",
+        "transient",
+        "catch",
+        "extends",
+        "int",
+        "short",
+        "try",
+        "char",
+        "final",
+        "interface",
+        "static",
+        "void",
+        "class",
+        "finally",
+        "long",
+        "strictfp",
+        "volatile",
+        "const",
+        "float",
+        "native",
+        "super",
+        "while",
+        "_",
+    ];
+    HashSet::from_iter(kwlist.into_iter().map(|s| s.to_string()))
+});
+
 // config options to customize the generated Java.
 #[derive(Debug, Default, Clone, Serialize, Deserialize)]
 pub struct Config {
@@ -237,6 +297,14 @@ impl<'a> TypeRenderer<'a> {
     }
 }
 
+fn fixup_keyword(name: String) -> String {
+    if KEYWORDS.contains(&name) {
+        format!("_{name}")
+    } else {
+        name
+    }
+}
+
 #[derive(Clone)]
 pub struct JavaCodeOracle;
 
@@ -249,9 +317,11 @@ impl JavaCodeOracle {
     fn class_name(&self, ci: &ComponentInterface, nm: &str) -> String {
         let name = nm.to_string().to_upper_camel_case();
         // fixup errors.
-        ci.is_name_used_as_error(nm)
-            .then(|| self.convert_error_suffix(&name))
-            .unwrap_or(name)
+        fixup_keyword(
+            ci.is_name_used_as_error(nm)
+                .then(|| self.convert_error_suffix(&name))
+                .unwrap_or(name),
+        )
     }
 
     fn convert_error_suffix(&self, nm: &str) -> String {
@@ -263,18 +333,22 @@ impl JavaCodeOracle {
 
     /// Get the idiomatic Java rendering of a function name.
     fn fn_name(&self, nm: &str) -> String {
-        nm.to_string().to_lower_camel_case()
+        fixup_keyword(nm.to_string().to_lower_camel_case())
     }
 
-    /// Get the idiomatic Java rendering of a variable name. Java variable names can't be escaped
-    /// and reserved words will cause breakage.
+    /// Get the idiomatic Java rendering of a variable name.
     pub fn var_name(&self, nm: &str) -> String {
+        fixup_keyword(self.var_name_raw(nm))
+    }
+
+    /// `var_name` without the reserved word alteration.  Useful for using in `@Structure.FieldOrder`.
+    pub fn var_name_raw(&self, nm: &str) -> String {
         nm.to_string().to_lower_camel_case()
     }
 
     /// Get the idiomatic setter name for a variable.
     pub fn setter(&self, nm: &str) -> String {
-        format!("set{}", nm.to_string().to_upper_camel_case())
+        format!("set{}", fixup_keyword(nm.to_string().to_upper_camel_case()))
     }
 
     /// Get the idiomatic Java rendering of an individual enum variant.
@@ -292,10 +366,11 @@ impl JavaCodeOracle {
         format!("Uniffi{}", nm.to_upper_camel_case())
     }
 
-    fn ffi_type_label_by_value(&self, ffi_type: &FfiType) -> String {
+    fn ffi_type_label_by_value(&self, ffi_type: &FfiType, prefer_primitive: bool) -> String {
         match ffi_type {
             FfiType::RustBuffer(_) => format!("{}.ByValue", self.ffi_type_label(ffi_type)),
             FfiType::Struct(name) => format!("{}.UniffiByValue", self.ffi_struct_name(name)),
+            _ if prefer_primitive => self.ffi_type_primitive(ffi_type),
             _ => self.ffi_type_label(ffi_type),
         }
     }
@@ -310,7 +385,7 @@ impl JavaCodeOracle {
             // function pointer better and allows for `null` as a default value.
             // Everything is nullable in Java by default.
             FfiType::Callback(name) => self.ffi_callback_name(name).to_string(),
-            _ => self.ffi_type_label_by_value(ffi_type),
+            _ => self.ffi_type_label_by_value(ffi_type, true),
         }
     }
 
@@ -337,12 +412,11 @@ impl JavaCodeOracle {
 
     fn ffi_type_label_by_reference(&self, ffi_type: &FfiType) -> String {
         match ffi_type {
+            FfiType::Int32 | FfiType::UInt32 => "IntByReference".to_string(),
             FfiType::Int8
             | FfiType::UInt8
             | FfiType::Int16
             | FfiType::UInt16
-            | FfiType::Int32
-            | FfiType::UInt32
             | FfiType::Int64
             | FfiType::UInt64
             | FfiType::Float32
@@ -379,6 +453,32 @@ impl JavaCodeOracle {
         }
     }
 
+    /// Generate primitive types where possible. Useful where we don't need or can't have boxed versions (ie structs).
+    fn ffi_type_primitive(&self, ffi_type: &FfiType) -> String {
+        match ffi_type {
+            // Note that unsigned integers in Java are currently experimental, but java.nio.ByteBuffer does not
+            // support them yet. Thus, we use the signed variants to represent both signed and unsigned
+            // types from the component API.
+            FfiType::Int8 | FfiType::UInt8 => "byte".to_string(),
+            FfiType::Int16 | FfiType::UInt16 => "short".to_string(),
+            FfiType::Int32 | FfiType::UInt32 => "int".to_string(),
+            FfiType::Int64 | FfiType::UInt64 => "long".to_string(),
+            FfiType::Float32 => "float".to_string(),
+            FfiType::Float64 => "double".to_string(),
+            FfiType::Handle => "long".to_string(),
+            FfiType::RustArcPtr(_) => "Pointer".to_string(),
+            FfiType::RustBuffer(maybe_suffix) => {
+                format!("RustBuffer{}", maybe_suffix.as_deref().unwrap_or_default())
+            }
+            FfiType::RustCallStatus => "UniffiRustCallStatus.ByValue".to_string(),
+            FfiType::ForeignBytes => "ForeignBytes.ByValue".to_string(),
+            FfiType::Callback(name) => self.ffi_callback_name(name),
+            FfiType::Struct(name) => self.ffi_struct_name(name),
+            FfiType::Reference(inner) => self.ffi_type_label_by_reference(inner),
+            FfiType::VoidPointer => "Pointer".to_string(),
+        }
+    }
+
     /// Get the name of the interface and class name for an object.
     ///
     /// If we support callback interfaces, the interface name is the object name, and the class name is derived from that.
@@ -422,7 +522,7 @@ impl AsCodeType for Type {
             Type::Float64 => Box::new(primitives::Float64CodeType),
             Type::Boolean => Box::new(primitives::BooleanCodeType),
             Type::String => Box::new(primitives::StringCodeType),
-            Type::Bytes => unimplemented!(), //Box::new(primitives::BytesCodeType),
+            Type::Bytes => Box::new(primitives::BytesCodeType),
 
             Type::Timestamp => Box::new(miscellany::TimestampCodeType),
             Type::Duration => Box::new(miscellany::DurationCodeType),
@@ -432,9 +532,9 @@ impl AsCodeType for Type {
                 Box::new(object::ObjectCodeType::new(name.clone(), *imp))
             }
             Type::Record { name, .. } => Box::new(record::RecordCodeType::new(name.clone())),
-            Type::CallbackInterface { name, .. } => {
-                unimplemented!() //Box::new(callback_interface::CallbackInterfaceCodeType::new(name))
-            }
+            Type::CallbackInterface { name, .. } => Box::new(
+                callback_interface::CallbackInterfaceCodeType::new(name.clone()),
+            ),
             Type::Optional { inner_type } => {
                 Box::new(compounds::OptionalCodeType::new((**inner_type).clone()))
             }
@@ -493,6 +593,11 @@ impl AsCodeType for &'_ uniffi_bindgen::interface::Record {
         self.as_type().as_codetype()
     }
 }
+impl AsCodeType for &'_ uniffi_bindgen::interface::CallbackInterface {
+    fn as_codetype(&self) -> Box<dyn CodeType> {
+        self.as_type().as_codetype()
+    }
+}
 
 mod filters {
     use super::*;
@@ -593,7 +698,7 @@ mod filters {
     }
 
     pub fn ffi_type_name_by_value(type_: &FfiType) -> Result<String, askama::Error> {
-        Ok(JavaCodeOracle.ffi_type_label_by_value(type_))
+        Ok(JavaCodeOracle.ffi_type_label_by_value(type_, false))
     }
 
     pub fn ffi_type_name_for_ffi_struct(type_: &FfiType) -> Result<String, askama::Error> {
@@ -619,6 +724,11 @@ mod filters {
         Ok(JavaCodeOracle.var_name(nm))
     }
 
+    /// Get the idiomatic Java rendering of a variable name, without altering reserved words.
+    pub fn var_name_raw(nm: &str) -> Result<String, askama::Error> {
+        Ok(JavaCodeOracle.var_name_raw(nm))
+    }
+
     /// Get the idiomatic Java setter method name.
     pub fn setter(nm: &str) -> Result<String, askama::Error> {
         Ok(JavaCodeOracle.setter(nm))
@@ -651,13 +761,35 @@ mod filters {
         Ok(JavaCodeOracle.object_names(ci, obj))
     }
 
+    pub fn async_inner_return_type(
+        callable: impl Callable,
+        ci: &ComponentInterface,
+    ) -> Result<String, askama::Error> {
+        callable
+            .return_type()
+            .map_or(Ok("Void".to_string()), |t| type_name(&t, ci))
+    }
+
+    pub fn async_return_type(
+        callable: impl Callable,
+        ci: &ComponentInterface,
+    ) -> Result<String, askama::Error> {
+        let is_async = callable.is_async();
+        let inner_type = async_inner_return_type(callable, ci)?;
+        if is_async {
+            Ok(format!("CompletableFuture<{inner_type}>"))
+        } else {
+            Ok(inner_type)
+        }
+    }
+
     pub fn async_poll(
         callable: impl Callable,
         ci: &ComponentInterface,
     ) -> Result<String, askama::Error> {
         let ffi_func = callable.ffi_rust_future_poll(ci);
         Ok(format!(
-            "{{ future, callback, continuation -> UniffiLib.INSTANCE.{ffi_func}(future, callback, continuation) }}"
+            "(future, callback, continuation) -> UniffiLib.INSTANCE.{ffi_func}(future, callback, continuation)"
         ))
     }
 
@@ -675,21 +807,20 @@ mod filters {
             }) => {
                 // Need to convert the RustBuffer from our package to the RustBuffer of the external package
                 let suffix = JavaCodeOracle.class_name(ci, &name);
+                // TODO(murph): make this Java
                 format!("{call}.let {{ RustBuffer{suffix}.create(it.capacity.toULong(), it.len.toULong(), it.data) }}")
             }
             _ => call,
         };
-        Ok(format!("{{ future, continuation -> {call} }}"))
+        Ok(format!("(future, continuation) -> {call}"))
     }
 
     pub fn async_free(
         callable: impl Callable,
         ci: &ComponentInterface,
     ) -> Result<String, askama::Error> {
         let ffi_func = callable.ffi_rust_future_free(ci);
-        Ok(format!(
-            "{{ future -> UniffiLib.INSTANCE.{ffi_func}(future) }}"
-        ))
+        Ok(format!("(future) -> UniffiLib.INSTANCE.{ffi_func}(future)"))
     }
 
     /// Remove the "`" chars we put around function/variable names