fix ci and docs

README.md

@@ -48,22 +48,26 @@ The currently supported AbiImpls are:
 
 By default, we test the following pairings:
 
+* rustc_calls_rustc
+* cc_calls_cc
 * rustc_calls_cc
 * cc_calls_rustc
-* cc_calls_cc
 
 In theory other implementations aren't *too bad* to add. You just need to:
 
 * Add an implementation of abis::AbiImpl
-    * Specify the name, language, and source-file extension
-    * Specify supported calling conventions
-    * Specify how to generate a caller from a signature
-    * Specify how to generate a callee from a signature
+    * Specify the language and source-file extension
+    * Specify how to generate source for a caller from a signature
+    * Specify how to generate source for a callee from a signature
     * Specify how to compile a source file to a static lib
 * Register it in the `abi_impls` map in `fn main`
 * (Optional) Register what you want it paired with by default in `DEFAULT_TEST_PAIRS`
     * i.e. (ABI_IMPL_YOU, ABI_IMPL_CC) will have the harness test you calling into C
 
+The bulk of the work is specifying how to generate source code, which can be done
+incrementally by return UnimplementedError to indicate unsupported features. This
+is totally fine, all the backends have places where they give up!
+
 See the Test Harness section below for details on how to use it.
 
 
@@ -82,96 +86,37 @@ Windows Conventions:
 * cdecl
 * fastcall
 * stdcall
-* ~~vectorcall~~ (code is there, but disabled due to linking issues)
-
-Any test which specifies the "All" will implicitly combinatorically generate every known convention.
-"Nonsensical" situations like stdcall on linux are the responsibility of the AbiImpls to identify and disable.
+* vectorcall
 
 
 ## Types
 
-The test format support for the following types/concepts:
-
-* fixed-width integer types (uint8_t and friends)
-* float/double
-* bool
-* structs
-* c-like enums
-* c-like untagged unions
-* rust-like tagged unions
-* opaque pointers (void\*)
-* pass-by-ref (still checks the pointee's layout, and not the address)
-* arrays (including multi-dimensional arrays, although C often requires arrays to be wrapped in pass-by-ref)
+The abi-cafe typesystem is [defined by kdl-script](https://github.com/Gankra/abi-cafe/blob/main/kdl-script/README.md#types), see those docs for details, but, we basically support most of the types you could define/use in core Rust.
 
 
 # Adding Tests
 
-There are two kinds of tests: `.kdl` and `.procgen.kdl`
+Tests are specified as [kdl-script](https://github.com/Gankra/abi-cafe/blob/main/kdl-script/README.md) files, which are basically C header files, but with a custom syntax that avoids us tying our hands to any particular language/semantics. The syntax will feel fairly familiar to Rust programmers.
 
-The latter is sugar for the former, where you just define a type with the same name of the file (so `MetersU32.procgen.kdl` is expected to define a type named `MetersU32`), and we generate a battery of types/functions that stress it out.
+The default suite of tests can be [found in `/include/tests/`](https://github.com/Gankra/abi-cafe/tree/main/include/tests), which is statically embedded in abi-cafe's binary. You don't need to register the test anywhere, we will just try to parse every file in the tests directory.
 
-Tests are specified as [ron](https://github.com/ron-rs/ron) files in the test/ directory, because it's more compact than JSON, has comments, and is more reliable with large integers. Because C is in some sense the "lingua franca" of FFI that everyone has to deal with, we prefer using C types in these definitions.
-
-You don't need to register the test anywhere, we will just try to parse every file in that directory.
-
-The "default" workflow is to handwrite a ron file, and the testing framework will handle generating the actual code implementating that interface (example: structs.ron). Generated impls will be output to the generated_impls dir for debugging. Build artifacts will get dumped in target/temp/ if you want to debug those too.
-
-Example:
-
-```rust
-Test(
-    // name of this set of tests
-    name: "examples",
-    // generate tests for the following function signatures
-    funcs: [
-        (
-            // base function/subtest name (but also subtest name)
-            name: "some_prims",
-            // what calling conventions to generate this test for
-            // (All = do them all, a good default)
-            conventions: [All],
-            // args
-            inputs: [
-               Int(c_int32_t(5)),
-               Int(c_uint64_t(0x123_abc)),
-            ]
-            // return value
-            output: Some(Bool(true)),
-        ),
-        (
-            name: "some_structs",
-            conventions: [All],
-            inputs: [
-               Int(c_int32_t(5)),
-               // Struct decls are implicit in usage.
-               // All structs with the same name must match!
-               Struct("MyStruct", [
-                  Int(c_uint8_t(0xf1)),
-                  Float(c_double(1234.23)),
-               ]),
-               Struct("MyStruct", [
-                  Int(c_uint8_t(0x1)),
-                  Float(c_double(0.23)),
-               ]),
-            ],
-            // no return (void)
-            output: None,
-        ),
-    ]
-)
-```
+There are two kinds of tests: `.kdl` ("[normal](https://github.com/Gankra/abi-cafe/tree/main/include/tests/normal)") and `.procgen.kdl` ("[procgen](https://github.com/Gankra/abi-cafe/tree/main/include/tests/procgen)").
 
-However, you have two "power user" options available:
+The latter is sugar for the former, where you just define a type with the same name of the file (so `MetersU32.procgen.kdl` is expected to define a type named `MetersU32`), and we generate a battery of types/functions that stress it out.
 
-* Generate the ron itself with generate_procedural_tests in main.rs (example: ui128.ron). This is good for bruteforcing a bunch of different combinations if you just want to make sure a type/feature generally works in many different situations.
+Suggested Examples:
 
-* Use the "Handwritten" convention and manually provide the implementations (example: opaque_example.ron). This lets you basically do *anything* without the testing framework having to understand your calling convention or type/feature. Manual impls go in handwritten_impls and use the same naming/structure as generated_impls.
+* [simple.kdl](https://github.com/Gankra/abi-cafe/blob/main/include/tests/normal/simple.kdl) - a little example of a "normal" test with explicitly defined functions to test
+* [SimpleStruct.procgen.kdl](https://github.com/Gankra/abi-cafe/blob/main/include/tests/procgen/struct/SimpleStruct.procgen.kdl) - similar to simple.kdl, but procgen
+* [MetersU32.procgen.kdl](https://github.com/Gankra/abi-cafe/blob/main/include/tests/procgen/pun/MetersU32.procgen.kdl) - an example of a ["pun type"](https://github.com/Gankra/abi-cafe/tree/main/kdl-script#pun-types), where different languages use different definitions
+* [IntrusiveList.procgen.kdl](https://github.com/Gankra/abi-cafe/blob/main/include/tests/procgen/fancy/IntrusiveList.procgen.kdl) - an example of how we can procgen tests for self-referential types and tagged unions
+* [i8.procgen.kdl](https://github.com/Gankra/abi-cafe/blob/main/include/tests/procgen/primitive/i8.procgen.kdl) - ok this one isn't instructive it's just funny that it can be a blank file because i8 is builtin so all the info needed is in the filename
 
 
 
 # The Test Harness
 
-Implementation details of dylib test harness are split up between main.rs and the contents of the top-level harness/ directory. The contents of harness/ include:
+Implementation details of dylib test harness are split up between [src/harness/run.rs](https://github.com/Gankra/abi-cafe/blob/main/src/harness/run.rs) and the contents of the top-level [/include/harness/](https://github.com/Gankra/abi-cafe/blob/main/include/harness/). The contents of /include/harness/ are:
 
 * "headers" for the testing framework for each language
 * harness.rs, which defines the entry-point for the test and sets up all the global callbacks/pointers. This is linked with the callee and caller to create the final dylib.
@@ -180,50 +125,50 @@ Ideally you shouldn't have to worry about *how* the callbacks work, so I'll just
 
 ```C
 // Caller Side
-uint64_t basic_val(struct MyStruct arg0, int32_t arg1);
+uint64_t basic_val(MyStruct arg0, int32_t arg1);
 
 // The test harness will invoke your test through this symbol!
 void do_test(void) {
     // Initialize and report the inputs
-    struct MyStruct arg0 = { 241, 1234.23 };
-    WRITE(CALLER_INPUTS, (char*)&arg0.field0, (uint32_t)sizeof(arg0.field0));
-    WRITE(CALLER_INPUTS, (char*)&arg0.field1, (uint32_t)sizeof(arg0.field1));
-    FINISHED_VAL(CALLER_INPUTS);
+    MyStruct arg0 = { 241, 1234.23 };
+    write_field(CALLER_INPUTS, arg0.field0);
+    write_filed(CALLER_INPUTS, arg0.field1);
+    finished_val(CALLER_INPUTS);
 
     int32_t arg1 = 5;
-    WRITE(CALLER_INPUTS, (char*)&arg1, (uint32_t)sizeof(arg1));
-    FINISHED_VAL(CALLER_INPUTS);
+    write_field(CALLER_INPUTS, arg1);
+    finished_val(CALLER_INPUTS);
 
     // Do the call
     uint64_t output = basic_val(arg0, arg1);
 
     // Report the output
-    WRITE(CALLER_OUTPUTS, (char*)&output, (uint32_t)sizeof(output));
-    FINISHED_VAL(CALLER_OUTPUTS);
+    write_field(CALLER_OUTPUTS, output);
+    finished_val(CALLER_OUTPUTS);
 
     // Declare that the test is complete on our side
-    FINISHED_FUNC(CALLER_INPUTS, CALLER_OUTPUTS);
+    finished_func(CALLER_INPUTS, CALLER_OUTPUTS);
 }
 ```
 
 ```C
 // Callee Side
-uint64_t basic_val(struct MyStruct arg0, int32_t arg1) {
+uint64_t basic_val(MyStruct arg0, int32_t arg1) {
     // Report the inputs
-    WRITE(CALLEE_INPUTS, (char*)&arg0.field0, (uint32_t)sizeof(arg0.field0));
-    WRITE(CALLEE_INPUTS, (char*)&arg0.field1, (uint32_t)sizeof(arg0.field1));
-    FINISHED_VAL(CALLEE_INPUTS);
+    write_field(CALLEE_INPUTS, arg0.field0);
+    write_field(CALLEE_INPUTS, arg0.field1);
+    finished_val(CALLEE_INPUTS);
 
-    WRITE(CALLEE_INPUTS, (char*)&arg1, (uint32_t)sizeof(arg1));
-    FINISHED_VAL(CALLEE_INPUTS);
+    write_field(CALLEE_INPUTS, arg1);
+    finished_val(CALLEE_INPUTS);
 
     // Initialize and report the output
     uint64_t output = 17;
-    WRITE(CALLEE_OUTPUTS, (char*)&output, (uint32_t)sizeof(output));
-    FINISHED_VAL(CALLEE_OUTPUTS);
+    write_field(CALLEE_OUTPUTS, output);
+    finished_val(CALLEE_OUTPUTS);
 
     // Declare that the test is complete on our side
-    FINISHED_FUNC(CALLEE_INPUTS, CALLEE_OUTPUTS);
+    finished_func(CALLEE_INPUTS, CALLEE_OUTPUTS);
 
     // Actually return
     return output;
@@ -232,13 +177,14 @@ uint64_t basic_val(struct MyStruct arg0, int32_t arg1) {
 
 The high level idea is that each side:
 
-* Uses WRITE to report individual fields of values (to avoid padding)
-* Uses FINISHED_VAL to specify that all fields for a value have been written
-* Uses FINISHED_FUNC to specify that the current function is done (the caller will usually contain many subtests, FINISHED_FUNC delimits those)
+* Uses write_field to report individual fields of values (to avoid padding)
+* Uses finished_val to specify that all fields for a value have been written
+* Uses finished_func to specify that the current function is done (the caller will usually contain many subtests, FINISHED_FUNC delimits those)
 
 There are 4 buffers: CALLER_INPUTS, CALLER_OUTPUTS, CALLEE_INPUTS, CALLEE_OUTPUTS. Each side should only use its two buffers.
 
-The signatures of the callbacks are:
+The signatures of the callbacks are as follows, but each language wraps these
+in functions/macros to keep the codegen readable:
 
 * `WRITE(Buffer buffer, char* input, uint32_t size_of_input)`
 * `FINISH_VAL(Buffer buffer)`

include/tests/normal/simple.kdl

@@ -0,0 +1,22 @@
+// This is a simple little test case to demonstrate a basic normal
+// abi-cafe test that includes some custom types and functions
+
+struct "Point3" {
+    x "f32"
+    y "f32"
+    z "f32"
+}
+
+fn "print" {
+    inputs { _ "Point3"; }
+}
+
+fn "scale" {
+    inputs { _ "Point3"; factor "f32"; }
+    output { "Point3"; }
+}
+
+fn "add" {
+    inputs { _ "Point3"; _ "Point3"; }
+    outputs { _ "Point3"; }
+}

include/tests/normal/types.kdl → include/tests/normal/typemess.kdl

@@ -20,17 +20,17 @@ enum "ErrorCode" {
 
 tagged "OptionI32" {
     None
-    Some { _0 "i32"; }
+    Some { _ "i32"; }
 }
 
 tagged "MyResult" {
-    Ok { _0 "[u32;3]"; }
-    Err { _0 "ErrorCode"; }
+    Ok { _ "[u32;3]"; }
+    Err { _ "ErrorCode"; }
 }
 
 tagged "MyDeepResult" {
-    Ok { _0 "MyResult"; }
-    Err { _0 "OptionI32"; }
+    Ok { _ "MyResult"; }
+    Err { _ "OptionI32"; }
     FileNotFound { x "bool"; y "Simple"; }
 }
 

src/abis/c.rs

@@ -64,9 +64,6 @@ enum Platform {
 }
 
 impl AbiImpl for CcAbiImpl {
-    fn name(&self) -> &'static str {
-        self.mode
-    }
     fn lang(&self) -> &'static str {
         "c"
     }

src/abis/mod.rs

@@ -267,7 +267,6 @@ impl std::fmt::Display for WriteImpl {
 
 /// ABI is probably a bad name for this... it's like, a language/compiler impl. idk.
 pub trait AbiImpl {
-    fn name(&self) -> &'static str;
     fn lang(&self) -> &'static str;
     fn src_ext(&self) -> &'static str;
     fn pun_env(&self) -> Arc<PunEnv>;

src/abis/rust.rs

@@ -60,9 +60,6 @@ enum Platform {
 }
 
 impl AbiImpl for RustcAbiImpl {
-    fn name(&self) -> &'static str {
-        "rustc"
-    }
     fn lang(&self) -> &'static str {
         "rust"
     }

src/abis/rust/declare.rs

@@ -259,7 +259,7 @@ impl RustcAbiImpl {
                 if let Some(real_tyname) = state.borrowed_tynames.get(real) {
                     writeln!(f, "type {name}<'a> = {real_tyname};\n")?;
                 } else {
-                    let real_tyname = &state.tynames[&real];
+                    let real_tyname = &state.tynames[real];
                     writeln!(f, "type {name} = {real_tyname};\n")?;
                 }
             }

src/abis/vals.rs

@@ -15,15 +15,13 @@ pub struct ValueTree {
 
 #[derive(Debug, Clone)]
 pub struct FuncValues {
-    pub func_name: String,
     pub args: Vec<ArgValues>,
 }
 
 #[derive(Debug, Clone)]
 pub struct ArgValues {
     pub arg_name: String,
     pub ty: TyIdx,
-    pub is_input: bool,
     pub vals: Vec<Value>,
 }
 
@@ -115,25 +113,22 @@ impl ValueTree {
             .all_funcs()
             .map(|func_idx| {
                 let func = types.realize_func(func_idx);
-                let func_name = func.name.to_string();
                 let args = func
                     .inputs
                     .iter()
-                    .map(|arg| (true, arg))
-                    .chain(func.outputs.iter().map(|arg| (false, arg)))
-                    .map(|(is_input, arg)| {
+                    .chain(&func.outputs)
+                    .map(|arg| {
                         let mut vals = vec![];
                         let arg_name = arg.name.to_string();
                         generators.build_values(types, arg.ty, &mut vals, arg_name.clone())?;
                         Ok(ArgValues {
                             ty: arg.ty,
                             arg_name,
-                            is_input,
                             vals,
                         })
                     })
                     .collect::<Result<Vec<_>, GenerateError>>()?;
-                Ok(FuncValues { func_name, args })
+                Ok(FuncValues { args })
             })
             .collect::<Result<Vec<_>, GenerateError>>()?;
 

src/fivemat.rs

@@ -77,6 +77,9 @@ mod test {
     fn fivemat_basic() -> std::fmt::Result {
         let mut out = String::new();
         let mut f = Fivemat::new(&mut out, "    ");
+        let ident_inner = "Inner";
+        let ident_x = "x";
+        let ty_f64 = "f64";
         writeln!(&mut f)?;
         {
             writeln!(&mut f, "struct MyStruct {{")?;
@@ -88,10 +91,10 @@ mod test {
             }
             {
                 write!(&mut f, "field2: ")?;
-                writeln!(&mut f, "{} {{", "Inner")?;
+                writeln!(&mut f, "{} {{", ident_inner)?;
                 f.add_indent(1);
                 {
-                    writeln!(&mut f, "{}: {},", "x", "f64")?;
+                    writeln!(&mut f, "{}: {},", ident_x, ty_f64)?;
                     writeln!(&mut f, "y: f32,")?;
                 }
                 f.sub_indent(1);