diff --git a/README.md b/README.md index ca14426..7861dcb 100644 --- a/README.md +++ b/README.md @@ -1,4 +1,190 @@ # veendor -a tool for vendoring your npm dependencies in arbitrary remote storage +A tool for stroing your npm dependencies in arbitraty storage -in development... stay tuned +### Features +Veendor: +* caches your `node_modules` in you-define-where. +* bootstraps your deps **fast**. +* only installs deps that have changed, effectively locking your deps. +* provides multi-layered cache. +* supports caching in git and local directory out-of-the-box. +* supports customizing cache keys calculation. + +### How it works +It calculates SHA-1 of `dependencies` and `devDependencies` in your `package.json`, +then searches for that hash in `backends` (cache providers). +If found, veendor downloads archive and unpacks your `node_modules`. Voila! +If not, veendor looks at previous revisions of your `package.json` and +tries to find older bundles, then installs only deps that have changed. +After that, veendor uploads new bundle to all `backends`. +If older bundles not found, veendor does clean `npm install` and +pushes bundle for future use. + +### Installation and use +Install veendor globally: +``` +npm install -g veendor +``` + +Go to your project and add a config file (`.veendor.js` or `.veendor.json`). +See section about config file below. +Run `veendor install`. +That's all! + +### Config file +Veendor supports configs as nodejs-modules or JSON-files. +Config file contains these sections: + +#### backends +Required. +Define your caches here. `backends` property is an array of objects. +Bundles search/upload will be in order defined here. +Each object has this format: +```js +{ + alias: 'some_name', // required, choose any name you like + backend: 'local', // string or module. See built-in backends and backend API sections + push: true, // optional, defaults to `false`. Should bundles be pushed to this backend + pushMayFail: true // optional, defaults to `false`. + // `veendor install` won't fail if push to backend fails + options: {} // backend-specific options +} +``` + +#### packageHash +Optional, object. +Used to extend cache key calculation. +Right now, only `suffix` property is used. +`suffix` may be string or function that returns string. +Examples: +```js +// Suffix by arch. +// Hashes will look like this: d0d5f10c199f507ea6e1584082feea229d59275b-darwin +packageHash: { + suffix: process.platform +} +``` + +```js +// Suffix by arch and node api version +// d0d5f10c199f507ea6e1584082feea229d59275b-darwin-46 +packageHash: { + suffix: process.platform + '-' + process.versions.modules +} +``` + +```js +// Invalidate every month +// d0d5f10c199f507ea6e1584082feea229d59275b-2017-7 +packageHash: { + suffix: () => { + const date = new Date(); + return date.getFullYear() + '-' + date.getMonth(); + } +} +``` + +#### installDiff +Optional, defaults to `true`. Enables diff installation. + +#### fallbackToNpm +Optional, defaults to `true`. +If true, runs `npm install` when bundle is not found. +Use this if you want to lock deps with veendor. +Should either be environmental-dependent or your backends should be populated manually. + +#### useGitHistory +Optional. +If contains `depth` property with number value, will look at +that amount of git revisions of package.json. +Note that only changes that affect dependencies and devDependencies count. +Example: +```js +useGitHistory: { + depth: 5 +} +``` + +### Built-in backends +#### git-lfs +Stores bundles in git repo. +Accepts these options: +```js +{ + repo: 'git@github.com:you/your-vendors.git', // required. Git remote. + compression: 'xz', // optional, defaults to 'gzip'. Also supports 'bzip2', 'xz'. + defaultBranch: 'braanch' // deafult branch of your repo. Defaults to 'master' +} +``` +Note: while supporting git-lfs is not mandatory for your remote, +it's pretty much required due to future repo size regressions. +Don't forget to set it up — add following to your `.gitattributes`: +``` +.tar.gz filter=lfs diff=lfs merge=lfs -text +``` +(replace `.tar.gz` with your selected compressison format) +[more about git-lfs](git-lfs.github.com) + +#### local +Stores bundles in local directory +Accepts these options: +```js +{ + directory: '/var/cache/veendor', // required. Directory to store bundles in. + compression: 'xz' // optional, defaults to 'gzip'. Also supports 'bzip2', 'xz'. +} +``` + +#### Example config +```js +const path = require('path'); + +module.exports = { + backends: [ + { + alias: 'local', + push: true, + backend: 'local', + options: { + directory: path.resolve(process.env.HOME, '.veendor-local') + } + }, + { + alias: 'github', + push: true, + backend: 'git-lfs', + options: { + repo: 'git@github.com:you/your-vendors.git' + } + } + ], + useGitHistory: { + depth: 5 + } +}; + +``` + +### Backends API +Backend should be an object with these properties: +#### pull(hash, options, cacheDir) => Promise +Should search for bundle with provided hash and +place node_modules into `process.cwd()`. +Promise resolves if succeded, rejects if not. +Options is object called `backend-specific options` earlier. +If backend needs to store some temp data, +veendor provides a clean `cacheDir` +#### push(hash, options, cacheDir) => Promise +Should take node_modules from `process.cwd()` and +upload it to the remote as bundle with `hash`. +`options` and `cacheDir` are same as in `pull`. +Promise resolves if succeded, rejects if not. +#### validateOptions(options) => undefined +Called upon start while validating config. +Should throw error if backend-specific options in config +are invalid. +May mutate options to set default values. +#### keepCache +Boolean, optional, defaults to false. +If your backend needs old calls cache for sake of efficiency, set it to true. +Otherwise, `cacheDir` will be clean before every call. diff --git a/bin/veendor.js b/bin/veendor.js index e82cf06..4c09471 100644 --- a/bin/veendor.js +++ b/bin/veendor.js @@ -1,7 +1,9 @@ -var program = require('commander'); +#!/usr/bin/env node +const program = require('commander'); +const version = require('../package.json').version; program - .version('0.0.0') + .version(version) .description('A tool for vendoring your npm dependencies') .command('calc', 'calculate and print your bundle id') .command('install', 'download and install node_modules') diff --git a/lib/validateConfig.js b/lib/validateConfig.js index 3e65861..91baf3b 100644 --- a/lib/validateConfig.js +++ b/lib/validateConfig.js @@ -71,6 +71,22 @@ function validateBackend(backend, position) { throw new InvalidBackendError(backend.alias, 'validateOptions'); } + if (backend.push === undefined) { + backend.push = false; + } + + if (typeof backend.push !== 'boolean') { + throw new InvalidBackendOptionError(backend.alias, 'push'); + } + + if (backend.pushMayFail === undefined) { + backend.pushMayFail = false; + } + + if (typeof backend.pushMayFail !== 'boolean') { + throw new InvalidBackendOptionError(backend.alias, 'pushMayFail'); + } + backend.backend.validateOptions(backend.options); } @@ -86,6 +102,12 @@ class InvalidBackendError extends Error { } } +class InvalidBackendOptionError extends Error { + constructor(alias, field) { + super(`backend\'s '${alias}' '${field}' option in invalid`); + } +} + class EmptyBackendAliasError extends Error { constructor(position) { super(`backend at position '${position}' lacks or has invalid 'alias' field`); @@ -97,6 +119,7 @@ class InvalidUseGitHistoryError extends Error {} module.exports.EmptyBackendsPropertyError = EmptyBackendsPropertyError; module.exports.InvalidBackendError = InvalidBackendError; +module.exports.InvalidBackendOptionError = InvalidBackendOptionError; module.exports.EmptyBackendAliasError = EmptyBackendAliasError; module.exports.AliasesNotUniqueError = AliasesNotUniqueError; module.exports.InvalidUseGitHistoryError = InvalidUseGitHistoryError; diff --git a/package.json b/package.json index 606aa20..8a7001d 100644 --- a/package.json +++ b/package.json @@ -1,7 +1,7 @@ { "name": "veendor", - "version": "0.0.0", - "description": "a tool for vendoring your npm dependencies into a git repository", + "version": "1.0.1", + "description": "a tool for stroing your npm dependencies in arbitraty storage", "main": "bin/veendor.js", "bin": { "veendor": "./bin/veendor.js" diff --git a/test/unit/validateConfig.test.js b/test/unit/validateConfig.test.js index 0eac347..af51fd8 100644 --- a/test/unit/validateConfig.test.js +++ b/test/unit/validateConfig.test.js @@ -63,6 +63,62 @@ describe('validateConfig', function () { }, validateConfig.EmptyBackendAliasError); }); + it('should check whether backend\'s push options are boolean', () => { + config.backends[0].push = 'test'; + + assert.throws(() => { + validateConfig(config); + }, validateConfig.InvalidBackendOptionError); + + config.backends[0].push = 1; + + assert.throws(() => { + validateConfig(config); + }, validateConfig.InvalidBackendOptionError); + + config.backends[0].push = () => {}; + + assert.throws(() => { + validateConfig(config); + }, validateConfig.InvalidBackendOptionError); + }); + + it('sets backend\'s push options to false', () => { + config.backends[0].push = true; + validateConfig(config); + + assert(config.backends[0].push === true, 'defined option should stay'); + assert(config.backends[1].push === false, 'config.backends[1].push should be `false`'); + }); + + it('should check whether backend\'s push options are boolean', () => { + config.backends[0].pushMayFail = 'test'; + + assert.throws(() => { + validateConfig(config); + }, validateConfig.InvalidBackendOptionError); + + config.backends[0].pushMayFail = 1; + + assert.throws(() => { + validateConfig(config); + }, validateConfig.InvalidBackendOptionError); + + config.backends[0].pushMayFail = () => {}; + + assert.throws(() => { + validateConfig(config); + }, validateConfig.InvalidBackendOptionError); + }); + + it('sets backend\'s pushMayFail options to false', () => { + config.backends[0].pushMayFail = true; + validateConfig(config); + + assert(config.backends[0].pushMayFail === true, 'defined option should stay'); + assert(config.backends[1].pushMayFail === false, 'config.backends[1].push should be `false`'); + }); + it('should check whether backends aliases are unique', () => { config.backends[0].alias = config.backends[1].alias;