Skip to content
This repository has been archived by the owner on Sep 14, 2024. It is now read-only.

Commit

Permalink
Port in README and DESIGN files
Browse files Browse the repository at this point in the history
  • Loading branch information
LPGhatguy committed Nov 2, 2017
1 parent 2e95eb4 commit e481137
Show file tree
Hide file tree
Showing 2 changed files with 237 additions and 1 deletion.
66 changes: 66 additions & 0 deletions DESIGN.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
# TestEZ Design Notes

## Test Flow
Testing using this module is broken down into four steps:

1. Load tests
2. Create test plan
3. Execute test plan
4. Report results

Before executing the test plan, we can also modify it to change semantics separately from the code.

### #1: Load tests
All ModuleScript objects that have a name ending in `.spec` are loaded as tests. On the filesystem, they appear as `.spec.lua` files.

These tests should return a function that describes the actual test using the `describe`, `it`, and `expect` functions. These functions are automatically injected into the test environment when available.

A test suite for addition might look like this:

```lua
return function()
describe("Addition", function()
it("should be commutative", function()
local a, b, c = 5, 8, 11
expect(a + b + c).to.equal(c + b + a)
end)

it("should be associative", function()
local a, b, c = 7, 4, 9
expect((a + b) + c).to.equal(a + (b + c))
end)
end)
end
```

All test assertions should be contained inside the `it` blocks.

### #2: Create test plan
A tree of tests is constructed out of all the `describe` and `it` calls in the tree. No test code is actually run.

This allows us to potentially output a tree of the tests in the system without actually running them. It also gives us a mechanism to run only specific tests.

This step is carried out by `TestPlanner`. It uses `TestPlanBuilder` to hold temporary state relevant only when building the plan, and then returns a `TestPlan` object.

To debug the test plan tree, use `plan:visualize()`

### #3: Execute test plan
A tree of test results is created that mirrors the test plan nodes.

This step is carried out by `TestRunner`. It creates a `TestSession` using the `TestPlan` from the previous step. This object holds state only relevant when building the test results, then returns a `TestResults` object.

To debug the result tree, use `results:visualize()`

### #4: Report results
Reporting is handled by a test reporter object, which is just a table with a `report` method on it. It takes a `TestResults` object and outputs it to standard output, creates a GUI, or tells TestService about the results.

This is pluggable, and could conceivably output in any format.

## System Architecture
The system is broken down in a way that eliminates global state, provides multiple abstraction layers for each operation, and stores plain data in a way that's agnostic to the operations performed upon it.

In both the planning and running phases of tests, a stateless module (`TestPlanner` and `TestRunner`) creates a temporary object to hold state about the operation it's trying to perform. These objects (`TestPlanBuilder` and `TestSession`) allow the code to traverse and build trees without passing around extra state between iterations.

When the modules are done using these builder objects, they call `finalize` on them to receive a `TestPlan` or `TestResults` object. These objects are passed back to the original caller.

`TestBootStrap` packages up use of the stages of testing into a simpler interface that also automatically locates test modules. When more advanced functionality is desired, it's easy to stop using TestBootstrap and instead call the underlying objects directly.
172 changes: 171 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@

TestEZ can run within Roblox itself, as well as inside [Lemur](https://github.com/LPGhatguy/Lemur) for testing on CI systems.

## Usage
## Installation

### Method 1: Installation Script (Roblox)
* Download the latest release from the [GitHub releases page](https://github.com/Roblox/TestEZ/releases).
Expand All @@ -28,5 +28,175 @@ You can use [Lemur](https://github.com/LPGhatguy/Lemur) paired together with a r

This is the best approach when testing Roblox Lua libraries using existing continuous integration systems like Travis-CI. We use this technique to run tests for [Rodux](https://github.com/Roblox/Rodux).

## Usage

## Writing Tests
Create `.spec.lua` files (or Roblox objects with the `.spec` suffix) for each module you want to test. These modules should return a function that in turn calls functions from TestEZ.

A simple module and associated TestEZ spec might look like:

`Greeter.lua`
```lua
local Greeter = {}

function Greeter:greet(person)
return "Hello, " .. person
end

return Greeter
```

`Greeter.spec.lua`
```lua
return function()
local Greeter = require(script.Parent.Greeter)

describe("greet", function()
it("should include the customary English greeting", function()
local greeting = Greeter:greet("X")
expect(greeting:match("Hello")).to.be.ok()
end)

it("should include the person being greeted", function()
local greeting = Greeter:greet("Joe")
expect(greeting:match("Joe")).to.be.ok()
end)
end)
end
```

The functions `describe`, `it`, and `expect` are injected by TestEZ and automatically hook into the current testing context.

Every module is implicitly scoped according to its path, meaning the tree that the above test represents might be:

```
LuaChat
Greeter
greet
[+] should include the customary English greeting
[+] should include the person being greeted
```

## Debugging Tests
Often during development, you'll want to only run the test that's concerned with the specific code you're working on.

TestEZ provides the `SKIP()` and `FOCUS()` functions to either skip or focus the block that the call is contained in.

This mechanism does not work for `it` blocks, where you can instead is `itSKIP` and `itFOCUS`. This is because the code inside `it` blocks is not run until the test is executed.

For example, I might want to run the tests targeting a specific method for my `DateTime` module:

`DateTime.spec.lua`
```lua
return function()
describe("ImportantFeature", function()
FOCUS()

it("does really important things", function()
-- This callback *will* run!
end)
end)

describe("Format", function()
it("formats things", function()
-- This callback will never run!
end)
end)
end
```

***`FOCUS` and `SKIP` are intended exclusively for development; future versions of TeztEZ will be able to detect this when running in a CI system and fail tests!***

## TestEZ API

### `describe(phrase, callback)`
This function creates a new `describe` block. These blocks correspond to the things that are being tested.

Put `it` blocks inside of `describe` blocks to describe what behavior should be correct.

For example:

```lua
describe("This cheese", function()
it("should be moldy", function()
expect(cheese.moldy).to.equal(true)
end)
end)
```

### `it(phrase, callback)`
This function creates a new 'it' block. These blocks correspond to the behaviors that should be expected of the thing you're testing.

For example:

```lua
it("should add 1 and 1", function()
expect(1 + 1).to.equal(2)
end)
```

### `FOCUS()`
When called inside a `describe` block, `FOCUS()` marks that block as *focused*. If there are any focused blocks inside your test tree, *only* focused blocks will be executed, and all other tests will be skipped.

When you're writing a new set of tests as part of a larger codebase, use `FOCUS()` while debugging them to reduce the amount of noise you need to scroll through.

For example:

```lua
describe("Secret Feature X", function()
FOCUS()

it("should do something", function()
end)
end)

describe("Secret Feature Y", function()
it("should do nothing", function()
-- This code will not run!
end)
end)
```

**Note: `FOCUS` will not work inside an `it` block. The bodies of these blocks aren't executed until the tests run, which is too late to change the test plan.**

### `SKIP()`
This function works similarly to `FOCUS()`, except instead of marking a block as *focused*, it will mark a block as *skipped*, which stops any of the test assertions in the block from being executed.

**Note: `SKIP` will not work inside an `it` block. The bodies of these blocks aren't executed until the tests run, which is too late to change the test plan.**

### `expect(value)`
Creates a new `Expectation`, used for testing the properties of the given value.

Expectations are intended to be read like English assertions. These are all true:

```lua
-- Equality
expect(1).to.equal(1)
expect(1).never.to.equal(2)

-- Nil checking
expect(1).to.be.ok()
expect(false).to.be.ok()
expect(nil).never.to.be.ok()

-- Type checking
expect(1).to.be.a("number")
expect(newproxy(true)).to.be.a("userdata")

-- Function throwing
expect(function()
error("nope")
end).to.throw()

expect(function()
-- I don't throw!
end).never.to.throw()
```

## Inspiration and Prior Work
The `describe` and `it` syntax in TestEZ is based on the [Behavior-Driven Development](https://en.wikipedia.org/wiki/Behavior-driven_development) methodology, notably as implemented in RSpec (Ruby), busted (Lua), Mocha (JavaScript), and Ginkgo (Go).

The `expect` syntax is based on Chai, a JavaScript assertion library commonly used with Mocha. Similar expectation systems are also used in RSpec and Ginkgo, with slightly different syntax.

## License
TestEZ is available under the [todo] license. See [LICENSE.md](LICENSE.md) for details.

0 comments on commit e481137

Please sign in to comment.