Add utility for creating a new monorepo package (#3581)

## Explanation

Manually a new monorepo package can be a tedious, even frustrating
process. To spare us from that suffering, this PR adds a CLI that
automates most of the job for us, creatively titled `create-package`.

Most of the context / explanation for what this is and how it works can
be found in a new section within the [contributor
documentation](https://github.com/MetaMask/core/blob/create-package/docs/contributing.md#adding-new-packages).
In addition, here's the output of `yarn create-package --help`:

```text
create-package <cmd> [args]

Commands:
  create-package new      Create a new monorepo
                          package. Handles a lot
                          of the boilerplate for
                          you.
  create-package default  Create a new monorepo
                          package with default
                          values.

Options:
  -h, --help  Show help                  [boolean]
```

### Use of Babel

This PR introduces a new directory, `./scripts/create-package`, and
100%-coverage tests for this directory. Due to some kind of ESM-related
incompatibility between version `^27.0.0` of `jest` / `ts-jest` and
TypeScript `4.0.0` / `5.0.0`, these tests use `babel-jest` instead of
`ts-jest`. This required pulling in a suite of Babel development
dependencies. Local testing indicates that we can switch back to
`ts-jest` we've bumped the monorepo to `jest@^29`.

## References

N/A

.eslintrc.js

@@ -9,6 +9,7 @@ module.exports = {
     'docs',
     'coverage',
     'merged-packages',
+    'package-template',
   ],
   overrides: [
     {

.github/workflows/lint-build-test.yml

@@ -115,6 +115,7 @@ jobs:
           node-version: ${{ matrix.node-version }}
           cache: yarn
       - run: yarn --immutable
+      - run: yarn test:scripts
       - run: yarn workspace ${{ matrix.package-name }} run test
       - name: Require clean working directory
         shell: bash

.gitignore

@@ -18,6 +18,7 @@ preview-build-message.txt
 packages/*/coverage
 packages/*/dist
 packages/*/docs
+scripts/coverage
 
 # yarn v3 (w/o zero-install)
 # See: https://yarnpkg.com/getting-started/qa#which-files-should-be-gitignored

.prettierrc.js

@@ -1,6 +1,9 @@
-// All of these are defaults except singleQuote, but we specify them
-// for explicitness
+/**
+ * @type {import('prettier').Options}
+ */
 module.exports = {
+  // All of these are defaults except singleQuote, but we specify them
+  // for explicitness
   quoteProps: 'as-needed',
   singleQuote: true,
   tabWidth: 2,

babel.config.js

@@ -0,0 +1,9 @@
+// We use Babel for our tests in scripts/.
+module.exports = {
+  env: {
+    test: {
+      presets: ['@babel/preset-typescript'],
+      plugins: ['@babel/plugin-transform-modules-commonjs'],
+    },
+  },
+};

docs/contributing.md

@@ -159,6 +159,50 @@ To use a preview build for a package within a project, you need to override the
 
 4. Run `yarn install`.
 
+## Adding new packages
+
+> If you're migrating an existing package to the monorepo, please see [the package migration documentation](./package-migration-process-guide.md).
+> You may be able to make use of `create-package` when migrating your package, but there's a lot more to it.
+
+Manually a new monorepo package can be a tedious, even frustrating process. To spare us from that
+suffering, we have created a CLI that automates most of the job for us, creatively titled
+[`create-package`](../scripts/create-package/). To create a new monorepo package, follow these steps:
+
+1. Create a new package using `yarn create-package`.
+   - Use the `--help` flag for usage information.
+   - Once this is done, you can find a package with your chosen name in `/packages`.
+2. Make sure your license is correct.
+   - By default, `create-package` gives your package an MIT license.
+   - If your desired license is _not_ MIT, then you must update your `LICENSE` file and the
+     `license` field of `package.json`.
+3. Add your dependencies.
+   - Do this as normal using `yarn`.
+   - Remember, if you are adding other monorepo packages as dependents, don't forget to add them
+     to the `references` array in your package's `tsconfig.json` and `tsconfig.build.json`.
+
+And that's it!
+
+### Contributing to `create-package`
+
+Along with this documentation, `create-package` is intended to be the source of truth for the
+process of adding new packages to the monorepo. Consequently, to change that process, you will want
+to change `create-package`.
+
+The `create-package` directory contains a [template package](../scripts/create-package/package-template/). The CLI is not aware of the contents of the template, only that its files have
+[placeholder values](../scripts/create-package/constants.ts). When a new package is created, the template files are read from disk, the
+placeholder values are replaced with real ones, and the updated files are added to a new directory
+in `/packages`. To modify the template package:
+
+- If you need to add or modify any files or folders, just go ahead and make your changes in
+  [`/scripts/create-package/package-template`](../scripts/create-package/package-template/).
+  The CLI will read whatever's in that directory and write it to disk.
+- If you need to add or modify any placeholders, make sure that your desired values are added to
+  both the relevant file(s) and
+  [`/scripts/create-package/constants.ts`](../scripts/create-package/constants.ts).
+  Then, update the implementation of the CLI accordingly.
+- As with placeholders, updating the monorepo files that the CLI interacts with begins by updating
+  [`/scripts/create-package/constants.ts`](../scripts/create-package/constants.ts).
+
 ## Releasing
 
 The [`create-release-branch`](https://github.com/MetaMask/create-release-branch) tool and [`action-publish-release`](https://github.com/MetaMask/action-publish-release) GitHub action are used to automate the release process.

jest.config.scripts.js

@@ -0,0 +1,82 @@
+/*
+ * For a detailed explanation regarding each configuration property and type check, visit:
+ * https://jestjs.io/docs/configuration
+ *
+ * NOTE:
+ * This config uses `babel-jest` due to ESM- / TypeScript-related incompatibilities with our
+ * current version (`^27`) of `jest` and `ts-jest`. We can switch to `ts-jest` once we have
+ * migrated our Jest dependencies to version `>=29`.
+ */
+
+module.exports = {
+  // Indicates whether the coverage information should be collected while executing the test
+  collectCoverage: true,
+
+  // An array of glob patterns indicating a set of files for which coverage information should be collected
+  collectCoverageFrom: ['<rootDir>/scripts/**/*.ts'],
+
+  // The directory where Jest should output its coverage files
+  coverageDirectory: '<rootDir>/scripts/coverage',
+
+  // An array of regexp pattern strings used to skip coverage collection
+  coveragePathIgnorePatterns: ['/package-template/'],
+
+  // Indicates which provider should be used to instrument code for coverage
+  coverageProvider: 'babel',
+
+  // A list of reporter names that Jest uses when writing coverage reports
+  coverageReporters: ['text', 'html', 'json-summary'],
+
+  // An object that configures minimum threshold enforcement for coverage results
+  // <rootDir> does not work here.
+  coverageThreshold: {
+    './scripts/create-package/**/*.ts': {
+      branches: 100,
+      functions: 100,
+      lines: 100,
+      statements: 100,
+    },
+  },
+
+  // A map from regular expressions to module names or to arrays of module names that allow to stub out resources with a single module
+  // This ensures that Babel can resolve ESM exports correctly.
+  moduleNameMapper: {
+    '^@metamask/utils/(.+)$': [
+      '<rootDir>/node_modules/@metamask/utils/dist/$1.js',
+    ],
+  },
+
+  // Disabled due to use of 'transform' below.
+  // // A preset that is used as a base for Jest's configuration
+  // preset: 'ts-jest',
+
+  // "restoreMocks" restores all mocks created using jest.spyOn to their
+  // original implementations, between each test. It does not affect mocked
+  // modules.
+  restoreMocks: true,
+
+  // The test environment that will be used for testing
+  testEnvironment: 'node',
+
+  // Options that will be passed to the testEnvironment
+  testEnvironmentOptions: {
+    customExportConditions: ['node', 'node-addons'],
+  },
+
+  // The glob patterns Jest uses to detect test files
+  testMatch: [
+    '<rootDir>/scripts/**/__tests__/**/*.[jt]s?(x)',
+    '<rootDir>/scripts/**/?(*.)+(spec|test).[tj]s?(x)',
+  ],
+
+  // An array of regexp pattern strings that are matched against all test paths, matched tests are skipped
+  testPathIgnorePatterns: ['/package-template/'],
+
+  // Default timeout of a test in milliseconds.
+  testTimeout: 5000,
+
+  // A map from regular expressions to paths to transformers
+  transform: {
+    '\\.[jt]sx?$': 'babel-jest',
+  },
+};

package.json

@@ -18,6 +18,7 @@
     "build:watch": "yarn run build --watch",
     "changelog:validate": "yarn workspaces foreach --parallel --interlaced --verbose run changelog:validate",
     "child-workspace-package-names-as-json": "ts-node scripts/child-workspace-package-names-as-json.ts",
+    "create-package": "ts-node scripts/create-package",
     "lint": "yarn lint:eslint && yarn lint:misc --check && yarn constraints",
     "lint:eslint": "eslint . --cache --ext js,ts",
     "lint:fix": "yarn lint:eslint --fix && yarn constraints --fix && yarn lint:misc --write && yarn",
@@ -26,8 +27,10 @@
     "prepare-preview-builds": "./scripts/prepare-preview-builds.sh",
     "publish-previews": "yarn workspaces foreach --parallel --verbose run publish:preview",
     "setup": "yarn install",
-    "test": "yarn test:verbose --silent --collectCoverage=false --reporters=jest-silent-reporter",
+    "test": "yarn test:scripts --silent --collectCoverage=false --reporters=jest-silent-reporter && yarn test:packages",
     "test:clean": "yarn workspaces foreach --parallel --verbose run test:clean && yarn test",
+    "test:packages": "yarn test:verbose --silent --collectCoverage=false --reporters=jest-silent-reporter",
+    "test:scripts": "yarn jest --config ./jest.config.scripts.js --silent",
     "test:verbose": "yarn workspaces foreach --parallel --verbose run test:verbose",
     "update-readme-content": "ts-node scripts/update-readme-content.ts"
   },
@@ -38,6 +41,9 @@
     "@metamask/rpc-methods@^0.38.1-flask.1": "patch:@metamask/rpc-methods@npm%3A0.38.1-flask.1#./.yarn/patches/@metamask-rpc-methods-npm-0.38.1-flask.1-081e1eb5b3.patch"
   },
   "devDependencies": {
+    "@babel/core": "^7.23.5",
+    "@babel/plugin-transform-modules-commonjs": "^7.23.3",
+    "@babel/preset-typescript": "^7.23.3",
     "@lavamoat/allow-scripts": "^2.3.1",
     "@metamask/create-release-branch": "^2.0.1",
     "@metamask/eslint-config": "^12.2.0",
@@ -47,9 +53,11 @@
     "@metamask/eth-json-rpc-provider": "^2.3.0",
     "@metamask/json-rpc-engine": "^7.3.0",
     "@metamask/utils": "^8.2.0",
+    "@types/jest": "^27.4.1",
     "@types/node": "^16.18.54",
     "@typescript-eslint/eslint-plugin": "^5.62.0",
     "@typescript-eslint/parser": "^5.62.0",
+    "babel-jest": "^27.5.1",
     "bn.js": "^5.2.1",
     "eslint": "^8.44.0",
     "eslint-config-prettier": "^8.5.0",
@@ -63,15 +71,18 @@
     "eth-block-tracker": "^8.0.0",
     "execa": "^5.0.0",
     "isomorphic-fetch": "^3.0.0",
+    "jest": "^27.5.1",
     "jest-silent-reporter": "^0.5.0",
     "nock": "^13.3.1",
     "prettier": "^2.7.1",
     "prettier-plugin-packagejson": "^2.4.5",
     "rimraf": "^3.0.2",
     "simple-git-hooks": "^2.8.0",
+    "ts-jest": "^27.1.4",
     "ts-node": "^10.9.1",
     "typescript": "~4.8.4",
-    "which": "^3.0.0"
+    "which": "^3.0.0",
+    "yargs": "^17.7.2"
   },
   "packageManager": "[email protected]",
   "engines": {

scripts/create-package/README.md

@@ -0,0 +1,4 @@
+# `create-package`
+
+This directory contains our CLI for creating new monorepo packages.
+For details, please see [the monorepo contributor documentation](../../docs/contributing.md#adding-new-packages).