Reorganize incubation repo to allow hosting multiple projects (#10)

WICG · May 31, 2023 · efe0ad6 · efe0ad6
1 parent 606987a
commit efe0ad6
Show file tree

Hide file tree

Showing 21 changed files with 146 additions and 110 deletions.
diff --git a/README.md b/README.md
@@ -1,88 +1,36 @@
-# ab-worker-prototype
+# Client A/B testing
 
-A proof of concept implementation of the performant client-side A/B testing specification [as outlined in `EXPLAINER.md`](/EXPLAINER.md), using a CloudFlare Edge Worker.
+Client A/B testing refers to the method of performing experimentation related
+changes to a web application at runtime, typically within a browser.
+This method of experimentation is popular in the industry as it is easier to
+deploy, requires minimal to no engineering bandwidth for creating experiments
+and is accessible to non-engineering personnel as well.
 
-## Architecture
+This incubation's objective is to devise methods of conducting the same outcome
+with all its benefits offered, but without the performance penalties.
 
-![architecture-diagram](/assets/images/arch.png)
+##  Goals
 
-To implement a proof of concept, we will use a CloudFlare Worker to apply the `PRE_UA` transforms. Alternatively, this could be any compute node that can act as a proxy — a connect middleware for an Express.js frontend application, a proxy or an Edge at the boundaries of an Origin, or a CDN with compute capabilities (like CF workers). The proxy should be able to intercept a request to the origin, minimally parse and modify the HTML and apply the transformations.
+ * Standardize A/B transformation operations to a spec.
+ * Enable application of experimentation changes at the Origin,
+   an intermediary CDN/Edge/Proxy or the Browser.
+ * Minimize or eliminate performance metrics degradation caused by
+   application of changes in the browser through performant methods and
+   through exploration of browser-native implementations.
 
-Internally, CloudFlare production is [said to](https://blog.cloudflare.com/html-parsing-1/) use [lol-html](https://github.com/cloudflare/lol-html), a low-latency HTML parser that supports streaming. A non-CloudFlare version of this prototype will have to employ a similar mechanism with same performance characteristics for this to function at scale.
+## Explainers and presentations
 
-For the A/B configuration provider, we will make a pseudo-API out of GitHub gists — this would allow us to test multiple sites and variations without redeploying the CloudFlare worker. This will also mimic the real world dependency of an API call federation off the Edge/CDN/Server. We will use stale-while-revalidate (SWR) policy for the configuration fetch, which should be acceptable for most websites’ real A/B tests.
+ * [Explainer and supported operations](specs/EXPLAINER.md)
+ * [WICG: A/B testing: updates - March 2023](https://docs.google.com/presentation/d/1WX-E63jL7ZwGf_jNszhfkdxsvzlXLdJdPMSTxK3X0A0/edit?usp=sharing)
+ * [PerfWG: Client-side A/B testing - March 2022](https://docs.google.com/presentation/d/1-cxHITwVtWJ5x3ev0__XzDtDtJn2cB9CAgN9Mkia3Ag/edit?usp=sharing)
+ * [Incubation proposal](https://github.com/WICG/proposals/issues/54)
 
-In addition to that, the prototype edge also implements the following essential A/B testing functions:
- * Figure out if an A/B test configuration is in place for the requested URI, via consulting the pseudo API.
- * Randomly select the request into an experiment based on the configuration, and set a cookie to make the selection sticky.
+## Directory structure
 
-## Sequence diagram
+ * `specs/` folder ontains the latest specification and explainers.
+ * `sdks/` contains sdks, libraries, etc.
+ * `prototypes/` hosts current prototypes, demos and sdks.
 
-![Sequence diagram](/assets/images/sequence.svg)
+## Contributing
 
-## Demo
-
-For a demo, we're going to A/B test ToDoMVC, that is built using React as a client-side SPA.
-
-The following A/B test configuration consists of transformations that create a randomized experiment for 50% of the traffic:
-  * Change background-color of the page to _`beige`_. Since this is best done on the static markup, we'd like to do have this transformation applied before UA.
-  * Change heading (H1) to "_a/b test h1_".
-  * Change the placeholder text of the text box to "_What would you like to do today?_"
-  * Color the first todo item _`red`_.
-
-In addition, we'll also inject correct `<base>` tag to make relative URLs work, and `<link rel=canonical>` tag to avoid getting our prototype indexed as a duplicate.
-
-```javascript
-const experimentConfigJson = {
-  "control": {
-    "url": "https://todomvc.com/examples/react",
-    "cache": {
-    }
-  },
-  "variants": [
-    {
-      "weight": 0.5,
-      "url": "https://todomvc.com/examples/react",
-      "transformations": [
-        [1 /* PRE_UA */, "head", 3 /* OP_PREPEND */,
-          "<base href=\"https://todomvc.com/examples/react/\" target=\"_blank\">"],
-        [1 /* PRE_UA */, "head", 3 /* OP_PREPEND */,
-          "<link rel=\"canonical\" href=\"https://todomvc.com/examples/react/\" />"]
-      ]
-    },
-    {
-      "weight": 0.5,
-      "url": "https://todomvc.com/examples/react",
-      "transformations": [
-        [1 /* PRE_UA */, "head", 3 /* OP_PREPEND */,
-          "<base href=\"https://todomvc.com/examples/react/\" target=\"_blank\">"],
-        [1 /* PRE_UA */, "head", 3 /* OP_PREPEND */,
-          "<link rel=\"canonical\" href=\"https://todomvc.com/examples/react/\" />"],
-        [1 /* PRE_UA */, "head", 4 /* OP_APPEND */,
-          "<style>body{background:beige!important}</style>"],
-        [2 /* ON_UA */, "h1", 0 /* OP_CUSTOM_JS */,
-          "$.innerHTML=\"a/b test h1\";"],
-        [2 /* ON_UA */, ".new-todo", 0 /* OP_CUSTOM_JS */,
-          "$.placeholder=\"What would you like to do today?\""],
-        [2 /* ON_UA */, ".todo-list>li", 0 /* OP_CUSTOM_JS */,
-          "$.style.color=\"red\""]
-      ]
-    }
-  ]
-}
-```
-
-The prototype deployed on CloudFlare Edge can be accessed at:
-  * [Control](https://ab-worker.alexnj.workers.dev/?experiment=todomvc-v01.json&force=0)
-  * [Experiment](https://ab-worker.alexnj.workers.dev/?experiment=todomvc-v01.json&force=1)
-  * [Select one of them at random](https://ab-worker.alexnj.workers.dev/?experiment=todomvc-v01.json)
-
-Configuration for running this experiment is [hosted as a gist](https://gist.github.com/alexnj/4c8d9198d16b238e4c7040250f052284#file-todomvc-v01-json).
-
-### Performance comparison
-
-#### WebPageTest
-
-![Filmstrip](/assets/images/filmstrip.png)
-
-[Comparison with more stats here](https://webpagetest.org/video/compare.php?tests=220128_AiDcKT_e71a33f6dd31af1157630f95377bbb4c,220128_BiDcH4_8af7080f26f4bc2f030ccce7e2695045).
+Please see [How to Contribute](CONTRIBUTING.md).
diff --git a/.gitignore → prototypes/cf-worker-prototype/.gitignore b/.gitignore → prototypes/cf-worker-prototype/.gitignore
diff --git a/LICENSE → prototypes/cf-worker-prototype/LICENSE b/LICENSE → prototypes/cf-worker-prototype/LICENSE
diff --git a/prototypes/cf-worker-prototype/README.md b/prototypes/cf-worker-prototype/README.md
@@ -0,0 +1,88 @@
+# ab-worker-prototype
+
+A proof of concept implementation of the performant client-side A/B testing specification [as outlined in `EXPLAINER.md`](/EXPLAINER.md), using a CloudFlare Edge Worker.
+
+## Architecture
+
+![architecture-diagram](assets/images/arch.png)
+
+To implement a proof of concept, we will use a CloudFlare Worker to apply the `PRE_UA` transforms. Alternatively, this could be any compute node that can act as a proxy — a connect middleware for an Express.js frontend application, a proxy or an Edge at the boundaries of an Origin, or a CDN with compute capabilities (like CF workers). The proxy should be able to intercept a request to the origin, minimally parse and modify the HTML and apply the transformations.
+
+Internally, CloudFlare production is [said to](https://blog.cloudflare.com/html-parsing-1/) use [lol-html](https://github.com/cloudflare/lol-html), a low-latency HTML parser that supports streaming. A non-CloudFlare version of this prototype will have to employ a similar mechanism with same performance characteristics for this to function at scale.
+
+For the A/B configuration provider, we will make a pseudo-API out of GitHub gists — this would allow us to test multiple sites and variations without redeploying the CloudFlare worker. This will also mimic the real world dependency of an API call federation off the Edge/CDN/Server. We will use stale-while-revalidate (SWR) policy for the configuration fetch, which should be acceptable for most websites’ real A/B tests.
+
+In addition to that, the prototype edge also implements the following essential A/B testing functions:
+ * Figure out if an A/B test configuration is in place for the requested URI, via consulting the pseudo API.
+ * Randomly select the request into an experiment based on the configuration, and set a cookie to make the selection sticky.
+
+## Sequence diagram
+
+![Sequence diagram](assets/images/sequence.svg)
+
+## Demo
+
+For a demo, we're going to A/B test ToDoMVC, that is built using React as a client-side SPA.
+
+The following A/B test configuration consists of transformations that create a randomized experiment for 50% of the traffic:
+  * Change background-color of the page to _`beige`_. Since this is best done on the static markup, we'd like to do have this transformation applied before UA.
+  * Change heading (H1) to "_a/b test h1_".
+  * Change the placeholder text of the text box to "_What would you like to do today?_"
+  * Color the first todo item _`red`_.
+
+In addition, we'll also inject correct `<base>` tag to make relative URLs work, and `<link rel=canonical>` tag to avoid getting our prototype indexed as a duplicate.
+
+```javascript
+const experimentConfigJson = {
+  "control": {
+    "url": "https://todomvc.com/examples/react",
+    "cache": {
+    }
+  },
+  "variants": [
+    {
+      "weight": 0.5,
+      "url": "https://todomvc.com/examples/react",
+      "transformations": [
+        [1 /* PRE_UA */, "head", 3 /* OP_PREPEND */,
+          "<base href=\"https://todomvc.com/examples/react/\" target=\"_blank\">"],
+        [1 /* PRE_UA */, "head", 3 /* OP_PREPEND */,
+          "<link rel=\"canonical\" href=\"https://todomvc.com/examples/react/\" />"]
+      ]
+    },
+    {
+      "weight": 0.5,
+      "url": "https://todomvc.com/examples/react",
+      "transformations": [
+        [1 /* PRE_UA */, "head", 3 /* OP_PREPEND */,
+          "<base href=\"https://todomvc.com/examples/react/\" target=\"_blank\">"],
+        [1 /* PRE_UA */, "head", 3 /* OP_PREPEND */,
+          "<link rel=\"canonical\" href=\"https://todomvc.com/examples/react/\" />"],
+        [1 /* PRE_UA */, "head", 4 /* OP_APPEND */,
+          "<style>body{background:beige!important}</style>"],
+        [2 /* ON_UA */, "h1", 0 /* OP_CUSTOM_JS */,
+          "$.innerHTML=\"a/b test h1\";"],
+        [2 /* ON_UA */, ".new-todo", 0 /* OP_CUSTOM_JS */,
+          "$.placeholder=\"What would you like to do today?\""],
+        [2 /* ON_UA */, ".todo-list>li", 0 /* OP_CUSTOM_JS */,
+          "$.style.color=\"red\""]
+      ]
+    }
+  ]
+}
+```
+
+The prototype deployed on CloudFlare Edge can be accessed at:
+  * [Control](https://ab-worker.alexnj.workers.dev/?experiment=todomvc-v01.json&force=0)
+  * [Experiment](https://ab-worker.alexnj.workers.dev/?experiment=todomvc-v01.json&force=1)
+  * [Select one of them at random](https://ab-worker.alexnj.workers.dev/?experiment=todomvc-v01.json)
+
+Configuration for running this experiment is [hosted as a gist](https://gist.github.com/alexnj/4c8d9198d16b238e4c7040250f052284#file-todomvc-v01-json).
+
+### Performance comparison
+
+#### WebPageTest
+
+![Filmstrip](assets/images/filmstrip.png)
+
+[Comparison with more stats here](https://webpagetest.org/video/compare.php?tests=220128_AiDcKT_e71a33f6dd31af1157630f95377bbb4c,220128_BiDcH4_8af7080f26f4bc2f030ccce7e2695045).
diff --git a/assets/images/arch.png → ...f-worker-prototype/assets/images/arch.png b/assets/images/arch.png → ...f-worker-prototype/assets/images/arch.png
diff --git a/assets/images/filmstrip.png → ...ker-prototype/assets/images/filmstrip.png b/assets/images/filmstrip.png → ...ker-prototype/assets/images/filmstrip.png
diff --git a/assets/images/sequence.svg → ...rker-prototype/assets/images/sequence.svg b/assets/images/sequence.svg → ...rker-prototype/assets/images/sequence.svg
diff --git a/package-lock.json → ...pes/cf-worker-prototype/package-lock.json b/package-lock.json → ...pes/cf-worker-prototype/package-lock.json
diff --git a/package.json → prototypes/cf-worker-prototype/package.json b/package.json → prototypes/cf-worker-prototype/package.json
diff --git a/src/client/index.ts → ...s/cf-worker-prototype/src/client/index.ts b/src/client/index.ts → ...s/cf-worker-prototype/src/client/index.ts
diff --git a/src/config.ts → prototypes/cf-worker-prototype/src/config.ts b/src/config.ts → prototypes/cf-worker-prototype/src/config.ts
diff --git a/src/index.ts → prototypes/cf-worker-prototype/src/index.ts b/src/index.ts → prototypes/cf-worker-prototype/src/index.ts
diff --git a/src/lib/ab-random-variant.ts → ...er-prototype/src/lib/ab-random-variant.ts b/src/lib/ab-random-variant.ts → ...er-prototype/src/lib/ab-random-variant.ts
diff --git a/src/lib/ab-variant-selection.ts → ...prototype/src/lib/ab-variant-selection.ts b/src/lib/ab-variant-selection.ts → ...prototype/src/lib/ab-variant-selection.ts
diff --git a/src/lib/constants.ts → .../cf-worker-prototype/src/lib/constants.ts b/src/lib/constants.ts → .../cf-worker-prototype/src/lib/constants.ts
diff --git a/src/lib/pre-ua-transform-applicator.ts → ...pe/src/lib/pre-ua-transform-applicator.ts b/src/lib/pre-ua-transform-applicator.ts → ...pe/src/lib/pre-ua-transform-applicator.ts
diff --git a/src/lib/transformer.ts → ...f-worker-prototype/src/lib/transformer.ts b/src/lib/transformer.ts → ...f-worker-prototype/src/lib/transformer.ts
diff --git a/src/lib/ua-transform-applicator.ts → ...totype/src/lib/ua-transform-applicator.ts b/src/lib/ua-transform-applicator.ts → ...totype/src/lib/ua-transform-applicator.ts
diff --git a/tsconfig.json → prototypes/cf-worker-prototype/tsconfig.json b/tsconfig.json → prototypes/cf-worker-prototype/tsconfig.json
diff --git a/wrangler.toml → prototypes/cf-worker-prototype/wrangler.toml b/wrangler.toml → prototypes/cf-worker-prototype/wrangler.toml