BREAKING - Uplift to use cht-form web component (#245)

4.0

.eslintignore

@@ -2,3 +2,4 @@ node_modules
 out
 dist
 ext
+build

.eslintrc

@@ -1,7 +1,8 @@
 {
   "extends": "@medic",
   "env": {
-    "node": true
+    "node": true,
+    "jquery": true
   },
   "parserOptions": {
     "ecmaVersion": 11

.github/workflows/run-tests.yml

@@ -26,4 +26,4 @@ jobs:
         node-version: ${{ matrix.node-version }}
     - run: sudo apt install -y xsltproc
     - run: npm ci --legacy-peer-deps
-    - run: npm run travis
+    - run: npm run build-ci

CONTRIBUTING.md

@@ -14,8 +14,9 @@ The Community Health Toolkit is powered by people like you. Your contributions h
   - Include tests for any new/updated logic.
     - Include the issue/PR number in the test title. This provides context for future debugging if the test ever regresses.
   - Update the project's version number in the [package.json](./package.json) by running `npm version <major|minor|patch>`
+  - Update the [release-notes.md](./release-notes.md) with a description of your change.
 - Before you submit a pull request, please make sure your contribution passes all tests. Test failures need to be addressed before we can merge your contribution.
-  - You can run `npm run travis` to build the project, lint the source code, and execute the tests.
+  - You can run `npm run build-ci` to build the project, lint the source code, and execute the tests.
 - Provide detail about the issue you are solving in the pull request description. Note: If your pull request addresses a specific issue, please reference it using medic/<repo>#<issue number>
 - Our CI will automatically schedule a build; monitor the build to ensure it passes.
 - Your PR will be reviewed by one of the repository's maintainers. Most PRs have at least one change requested before they're merged so don't be offended if your change doesn't get accepted on the first try!
@@ -26,3 +27,53 @@ All maintainers and contributors are required to act according to our [Code of C
 
 #### License
 The software is provided under AGPL-3.0. Contributions to this project are accepted under the same license.
+
+## Architecture
+
+```mermaid
+flowchart TB
+    cht-core-bundle
+    subgraph browser
+        form-host
+        cht-form
+        form-host --> cht-form
+    end
+
+    subgraph tests[Config Tests]
+        harness[Harness]
+    end
+
+    harness -->|tasks/targets/contact-summary| cht-core-bundle
+    harness -->|forms| form-host
+```
+
+The test harness leverages subsets of the actual cht-core code to recreate an approximation of the production environment.
+
+- `Harness` - The external interface exposed to users via cht-conf-test-harness for writing config tests
+- `form-host` - Browser-side shim for integrating with cht-form
+- `cht-form` - Renders ODK forms. Built from cht-core/webapp.
+- `cht-core-bundle` - A bundle of code from cht-core used for calculating tasks, targets, and contact summaries. Also used to convert forms to xforms.
+
+## Maintaining CHT Artifacts
+
+### Adding new cht-core version
+
+Code for each CHT version is stored in [`cht-bundles`](./cht-bundles). To add a new version:
+
+1. Create a new folder in `cht-bundles` with the name of the version you want to add (e.g. `cht-core-5-0`).
+    1. Inside this folder create `bundle.js` and `xsl-paths.js` files following the pattern of the other versions.
+    1. Update the contents of these files to point to the (non-existent) `build` directory of the cht-core version you want to add. This build directory will be created/populated later.
+1. Update the [`all-chts-bundle.js`](./cht-bundles/all-chts-bundle.js) file to include the new version.
+    1. Note that the key given here will represent the value consumers should set in the `coreVersion` field of their harness configuration.
+1. Update the [`build.sh`](./build.sh) script to include the new version in the `cht_versions` map. Include the exact commit hash to use for the version. 
+1. Run `npm run build` to create the new artifacts.
+1. Update the [`harness.defaults.json` config](./test/collateral/harness.defaults.json) so that `coreVersion` points to the new version.
+    1. Run `npm run test` to ensure the new artifacts are working as expected.
+1. Update the compatibility matrix in the [`README.md`](./README.md) to include the new version.
+1. Do not forget to commit the newly generated contents of `./dist`!
+
+### Updating existing cht-core artifacts
+
+It is not necessary to rebuild the cht-core artifacts for every change to the cht-conf-test-harness code. Many changes to the harness logic should be passive in regard to the cht-core integration and running `npm run build` should be sufficient to rebuild only the required artifacts.
+
+By default, the build script will not re-build the cht-core artifacts in the `dist/cht-core-x-x` directories if they already exist. To rebuild a particular version, delete the `dist/cht-core-x-x` directory before running the build script. To force a rebuild of all versions, you can run the `build.sh` script with the `--force` flag. Rebuilding the existing cht-core artifacts should only be necessary if changes to the harness logic require changes to the cht-core integration (e.g. pulling in additional logic from the cht-core code).

README.md

@@ -1,7 +1,21 @@
 # Test Harness for CHT Configuration
 
+## Usage
+
 View the documentation [here](http://docs.communityhealthtoolkit.org/cht-conf-test-harness/).
 
+### Compatibility
+
+The cht-conf-test-harness is targeted at specific versions of the CHT Core Framework. This allows you to run your tests and validate your config against a specific version of the CHT before deploying it to a live environment.
+
+The following table shows the compatibility between the test harness and the CHT Core Framework (including the `coreVersion` values you can use in your harness configuration):
+
+| cht-conf-test-harness | CHT Core Framework | Supported `coreVersion` values                |
+|-----------------------|--------------------|-----------------------------------------------|
+| 2.x                   | 3.9.x+             | `3.9`, `3.10`, `3.11`, `3.12`, `3.13`, `3.14` |
+| 3.x                   | 4.0.x-4.5.x        | `4.0`                                         |
+| 4.x                   | 4.6.x+             | `4.6`                                         |
+
 ## Contributing
 
 At Medic we are the technical steward of the [Community Health Toolkit](https://communityhealthtoolkit.org) (CHT). We welcome and appreciate contributions, and support new developers to use the tools whenever possible. If you have an idea or a question we'd love to hear from you! The easiest ways to get in touch are by raising issues in the [Github repo](https://github.com/medic/cht-conf-test-harness/issues) or [joining our Slack channel](https://communityhealthtoolkit.org/slack). For more info check out our [contributor guidelines](CONTRIBUTING.md).

build.sh

@@ -1,4 +1,8 @@
 #!/usr/bin/env bash
+declare -A cht_versions=(
+  ["cht-core-4-6"]="4.6.0"
+)
+
 exit_on_error() {
   exit_code=$?
   if [[ $exit_code -ne 0 ]]; then
@@ -10,40 +14,47 @@ exit_on_error() {
 set -e
 trap exit_on_error EXIT
 
+if [ "$1" == "--force" ]; then
+  FORCE=1
+fi
+
 npm ci --legacy-peer-deps
-rm -Rf dist build
-node ./compile-ddocs.js
-
-mkdir -p ext/xsl
-mkdir -p ext/enketo-transformer/xsl
-cp ./node_modules/cht-core-4-0/api/src/xsl/openrosa2html5form.xsl ext/xsl
-cp ./node_modules/cht-core-4-0/api/src/enketo-transformer/xsl/* ext/enketo-transformer/xsl
-
-dirs=($(find node_modules/cht-* -maxdepth 0 -type d))
-for dir in "${dirs[@]}"; do
-  (cd "$dir"/webapp && npm ci --legacy-peer-deps --production)
-  (cd "$dir"/api && npm ci --legacy-peer-deps --production)
-  (cd "$dir"/shared-libs/calendar-interval && npm ci --legacy-peer-deps)
-  (cd "$dir"/shared-libs/rules-engine && npm ci --legacy-peer-deps)
-  (cd "$dir"/shared-libs/phone-number && npm ci --legacy-peer-deps --production)
-
-  # patch the daterangepicker for responsiveness
-  # https://github.com/dangrossman/bootstrap-daterangepicker/pull/437
-  (cd "$dir" && patch -f webapp/node_modules/bootstrap-daterangepicker/daterangepicker.js < webapp/patches/bootstrap-daterangepicker.patch)
+rm -Rf build
+
+if [[ 1 == "$FORCE" ]]; then
+  rm -Rf dist
+else
+  for item in `ls dist | grep -v cht-core`; do
+    rm -rf dist/"$item"
+  done
+fi
+
+for version in "${!cht_versions[@]}"; do
+  if [[ ! 1 == "$FORCE" ]] && [ -d dist/"$version" ]; then
+    printf "\033[0;32m== SKIPPING $version ==\n"
+    continue
+  fi
 
-  # 210 patch to disable db-object-widget
-  (cd "$dir" && patch -f webapp/src/js/enketo/widgets.js < ../../patches/210-disable-db-object-widgets.patch)
+  git clone https://github.com/medic/cht-core.git build/"$version"
+  (cd build/"$version" && git reset --hard "${cht_versions[$version]}")
+  (cd build/"$version" && git clean -df)
 
-  # patch enketo to always mark the /inputs group as relevant
-  (cd "$dir" && patch -f webapp/node_modules/enketo-core/src/js/form.js < webapp/patches/enketo-inputs-always-relevant_form.patch)
-  (cd "$dir" && patch -f webapp/node_modules/enketo-core/src/js/relevant.js < webapp/patches/enketo-inputs-always-relevant_relevant.patch)
+  (cd build/"$version" && npm ci)
+
+  node ./compile-ddocs.js "$version"
+
+  (cd build/"$version"/api && npm ci --legacy-peer-deps --production)
+  (cd build/"$version" && patch -f api/src/services/generate-xform.js < ../../patches/generate-xform.patch)
+  # 210 patch to disable db-object-widget
+  (cd build/"$version" && patch -f webapp/src/js/enketo/widgets.js < ../../patches/210-disable-db-object-widgets.patch)
+  (cd build/"$version" && npm run build-cht-form)
 
-  # patch enketo to fix repeat name collision bug - this should be removed when upgrading to a new version of enketo-core
-  # https://github.com/enketo/enketo-core/issues/815
-  (cd "$dir" && patch -f webapp/node_modules/enketo-core/src/js/calculate.js < webapp/patches/enketo-repeat-name-collision.patch)
+  mkdir -p dist/"$version"/xsl
+  mkdir -p dist/"$version"/enketo-transformer/xsl
+  cp ./build/"$version"/api/src/xsl/openrosa2html5form.xsl dist/"$version"/xsl
+  cp ./build/"$version"/api/src/enketo-transformer/xsl/* dist/"$version"/enketo-transformer/xsl
 
-  # patch messageformat to add a default plural function for languages not yet supported by make-plural #5705
-  (cd "$dir" && patch -f webapp/node_modules/messageformat/lib/plurals.js < webapp/patches/messageformat-default-plurals.patch)
+  npx webpack --config cht-bundles/webpack.config.cht-core.js --env.cht="$version"
 done
 
 npx webpack

cht-bundles/all-chts-bundle.js

@@ -0,0 +1,3 @@
+module.exports = {
+  '4.6': require('../dist/cht-core-4-6/cht-core-bundle.dev'),
+};

cht-bundles/cht-core-4-6/bundle.js

@@ -0,0 +1,11 @@
+module.exports = {
+  ddocs: require('../../build/cht-core-4-6-ddocs.json'),
+  RegistrationUtils: require('../../build/cht-core-4-6/shared-libs/registration-utils'),
+  CalendarInterval: require('../../build/cht-core-4-6/shared-libs/calendar-interval'),
+  RulesEngineCore: require('../../build/cht-core-4-6/shared-libs/rules-engine'),
+  RulesEmitter: require('../../build/cht-core-4-6/shared-libs/rules-engine/src/rules-emitter'),
+  nootils: require('../../build/cht-core-4-6/node_modules/cht-nootils'),
+  Lineage: require('../../build/cht-core-4-6/shared-libs/lineage'),
+  ChtScriptApi: require('../../build/cht-core-4-6/shared-libs/cht-script-api'),
+  convertFormXmlToXFormModel: require('../../build/cht-core-4-6/api/src/services/generate-xform.js').generate,
+};

cht-bundles/cht-core-4-6/xsl-paths.js

@@ -0,0 +1,6 @@
+const path = require('path');
+
+module.exports = {
+  FORM_STYLESHEET: path.join(__dirname, '../dist/cht-core-4-6/xsl/openrosa2html5form.xsl'),
+  MODEL_STYLESHEET: path.join(__dirname, '../dist/cht-core-4-6/enketo-transformer/xsl/openrosa2xmlmodel.xsl'),
+};

cht-bundles/webpack.config.cht-core.js

@@ -0,0 +1,62 @@
+const path = require('path');
+const WebpackCleanConsolePlugin = require('webpack-clean-console-plugin');
+module.exports = env => [
+  {
+    entry: `./cht-bundles/${env.cht}/bundle.js`,
+    output: {
+      path: path.join(__dirname,'../', 'dist', env.cht),
+      filename: `cht-core-bundle.dev.js`,
+      library: {
+        type: 'commonjs',
+      }
+    },
+    resolve: {
+      alias: {
+        // inside cht-core/api/src/services/generate-xform.js
+        '../xsl/xsl-paths': path.join(__dirname, env.cht, 'xsl-paths.js'),
+      },
+    },
+    target: 'node',
+    mode: 'development',
+    devtool: 'source-map',
+    plugins: [
+      new WebpackCleanConsolePlugin({ include: ['debug'] }),
+    ],
+  },
+  {
+    entry: [
+      `./build/${env.cht}/build/cht-form/main.js`,
+      `./build/${env.cht}/build/cht-form/polyfills.js`,
+      `./build/${env.cht}/build/cht-form/runtime.js`,
+      `./build/${env.cht}/build/cht-form/scripts.js`,
+      `./build/${env.cht}/build/cht-form/styles.css`,
+    ],
+    output: {
+      filename: 'cht-form.js',
+      path: path.join(__dirname, '../', 'dist', env.cht),
+      assetModuleFilename: '[name][ext]'
+    },
+    resolve: {
+      alias: {
+        '/fonts/NotoSans-Regular.ttf': './fonts/NotoSans-Regular.ttf',
+        '/fonts/NotoSans-Bold.ttf': './fonts/NotoSans-Bold.ttf',
+        '/fonts/enketo-icons-v2.woff': './fonts/enketo-icons-v2.woff',
+        '/fonts/enketo-icons-v2.ttf': './fonts/enketo-icons-v2.ttf',
+        '/fonts/enketo-icons-v2.svg': './fonts/enketo-icons-v2.svg',
+      },
+    },
+    module: {
+      rules: [
+        {
+          test: /\.(svg|ttf|woff)$/i,
+          type: 'asset/resource'
+        },
+        {
+          test: /\.css$/i,
+          use: ['style-loader', 'css-loader'],
+        },
+      ],
+    },
+    optimization: { minimize: false },
+  }
+];

compile-ddocs.js

@@ -6,23 +6,19 @@ if (!fs.existsSync('build')) {
   fs.mkdirSync('build');
 }
 
-const coreVersions = fs.readdirSync('node_modules')
-  .filter(dir => dir.startsWith('cht-core-'));
+const coreVersion = process.argv[2];
+const outputPath = path.resolve(`./build/${coreVersion}-ddocs.json`);
+const ddocFolderPath = [
+  `build/${coreVersion}/ddocs/medic-client/`,
+  `build/${coreVersion}/ddocs/medic-db/medic-client/`,
+].find(fs.existsSync);
+console.log(`Compiling ddocs for ${coreVersion} to ${outputPath}`);
+compile(ddocFolderPath, function(error, doc) {
+  if (error) {
+    console.error(error);
+    throw error;
+  }
 
-for (const coreVersion of coreVersions) {
-  const outputPath = path.resolve(`./build/${coreVersion}-ddocs.json`);
-  const ddocFolderPath = [
-    `node_modules/${coreVersion}/ddocs/medic-client/`,
-    `node_modules/${coreVersion}/ddocs/medic-db/medic-client/`,
-  ].find(fs.existsSync);
-  console.log(`Compiling ddocs for ${coreVersion} to ${outputPath}`);
-  compile(ddocFolderPath, function(error, doc) {
-    if (error) {
-      console.error(error);
-      throw error;
-    }
-
-    fs.writeFileSync(outputPath, JSON.stringify([doc], null, 2));
-    console.log(`ddocs compiled to ${outputPath}`);
-  });
-}
+  fs.writeFileSync(outputPath, JSON.stringify([doc], null, 2));
+  console.log(`ddocs compiled to ${outputPath}`);
+});