4.7.0

What is new since 4.6.0?

Thanks to @renehernandez for these awesome improvements to Mock!

Mock parameter filter can use aliases

Using parameter aliases in Mock ParameterFilters was a long requested feature, and it is finally here.

In the example below you can see the alias Last being used in the parameter filter, and then used from two mocks. One using the actual parameter name Tail and the other one using the parameter alias Last:

Describe "Get-Content" {   
    It "Filters on parameter alias" {
        Mock Get-Content -ParameterFilter {$Last -eq 100} { "mock1" }
        
        Get-Content -Path . -Tail 100 | Should -Be "mock1"
        Get-Content -Path . -Last 100 | Should -Be "mock1"
    }
}

Mock aliases executables on Windows without .exe extension

Mocking an external command on Windows will now alias it with and without .exe. Here is an example of mocking notepad.exe.

Describe "Notepad" {   
    It "Aliases notepad" {
        Mock Notepad { "mock1" }

        notepad.exe | Should -Be "mock1"
        notepad | Should -Be "mock1"        
    }
}

Mocking Get-Module no longer fails

Mocking Get-Module failed because the proxy function that PowerShell generates is conflicting with $PSEdition readonly variable. But now the parameter variable is renamed to $_PSEdition and aliased back to PSEdition. This allows you to mock Get-Module as you would with any other function. The only difference is that in ParemeterFilter you now need to use the $_PSEdition` if you wish to filter by it. Here is an example:

Describe "Get-Module" {
    It "Filtering PSEdition" {
        Mock Get-Module -ParameterFilter { $_PSEdition -eq "Desktop" } { "mock1" }
    
        Get-Module -PSEdition Desktop -ListAvailable | 
            Should -Be "mock1"
    }
}

4.7.0-beta1

What is new since 4.6.0?

Thanks to @renehernandez for these awesome improvements to Mock!

Mock parameter filter can use aliases

Using parameter aliases in Mock ParameterFilters was a long requested feature, and it is finally here.

In the example below you can see the alias Last being used in the parameter filter, and then used from two mocks. One using the actual parameter name Tail and the other one using the parameter alias Last:

Describe "Get-Content" {   
    It "Filters on parameter alias" {
        Mock Get-Content -ParameterFilter {$Last -eq 100} { "mock1" }
        
        Get-Content -Path . -Tail 100 | Should -Be "mock1"
        Get-Content -Path . -Last 100 | Should -Be "mock1"
    }
}

Mock aliases executables on Windows without .exe extension

Mocking an external command on Windows will now alias it with and without .exe. Here is an example of mocking notepad.exe.

Describe "Notepad" {   
    It "Aliases notepad" {
        Mock Notepad { "mock1" }

        notepad.exe | Should -Be "mock1"
        notepad | Should -Be "mock1"        
    }
}

Mocking Get-Module no longer fails

Mocking Get-Module failed because the proxy function that PowerShell generates is conflicting with $PSEdition readonly variable. But now the parameter variable is renamed to $_PSEdition and aliased back to PSEdition. This allows you to mock Get-Module as you would with any other function. The only difference is that in ParemeterFilter you now need to use the $_PSEdition` if you wish to filter by it. Here is an example:

Describe "Get-Module" {
    It "Filtering PSEdition" {
        Mock Get-Module -ParameterFilter { $_PSEdition -eq "Desktop" } { "mock1" }
    
        Get-Module -PSEdition Desktop -ListAvailable | 
            Should -Be "mock1"
    }
}

4.6.0

What is new since 4.5.0?

Should -HaveParameter

A new assertion operator -HaveParameter was added that allows you to check function parameters, and their properties like this:

Get-Command "Invoke-WebRequest" | Should -HaveParameter Uri -Mandatory

function f ([String] $Value) { }
Get-Command f | Should -HaveParameter Value -Type String
Get-Command f | Should -Not -HaveParameter Name

Other minor fixes

See the full changelog for more info.

5.0.0-alpha2

Pester v5 - alpha2

🍌 Scoping of Describe & It

This change is not really that new, it works the same as in alpha1, but there I did not describe it 😅 And now it is useful to understand the difference from v4, which in turns make understanding the mocking described below easier. So here we go:

Execution order

In v4 the execution of Describe, BeforeAll and AfterAll is out of order. Running this code in v4 and v5 yields different results:

Describe "d" {
    Write-Host Running Describe

    BeforeAll {
        Write-Host Running BeforeAll
    }

    It "i" {
        Write-Host Running It
    }

    AfterAll {
        Write-Host Running AfterAll
    }
    Write-Host Leaving Describe
}

# v4
Describing d
Running BeforeAll
Running Describe
Running It
  [+] i 46ms
Leaving Describe
Running AfterAll

# v5
Describing d
Running Describe
Running BeforeAll
Running It
Running AfterAll
    [+] i 28ms
Leaving Describe

As you can see above, the BeforeAll and AfterAll blocks run outside of the Describe in which they are defined. This is slightly surprising and it prevents few scenarios like defining a function inside of Describe and using it in BeforeAll.

In v5 the code runs in the correct order, Describe is entered first and BeforeAll runs right before It is started.

Admittedly this is not a huge change, the issue with blocks being run out of order is not reported often, and in v5 you should be putting all your code in Pester controlled blocks anyway, but it is nice to have things execute in order, because the code is then easier to reason about.

Scopes

Once the blocks are in order and executed closer together, we can start thinking about how they are scoped. In v4 sharing state between the different blocks is hard and you will struggle getting it right, see this example where I change value of variable $v and report what the value is in the next block:

Describe "d" {
    $v = "describe"
    BeforeAll {
        Write-Host "in before all v is: $v"
        $v = "before all"
    }

    BeforeEach {
        Write-Host "in before each v is: $v"
        $v = "before each"
    }

    It "i" {
        Write-Host Write-Host "in it v is: $v"
        $v = "it"
    }

    AfterEach {
        Write-Host "in after each v is: $v"
        $v = "after each"
    }

    AfterAll {
        Write-Host "in after all v is: $v"
        $v = "after all"
    }
    Write-Host "in describe v is: $v"
}

# v4
Describing d
in before all v is:
in before each v is: describe
in it v is: before each
in after each v is: before each
  [+] i 45ms
in describe v is: after each
in after all v is: before all

The v4 output is a bit hard to decipher because the blocks run out of order, but hopefully you can see that:

• BeforeEach each gets value from Describe and not from BeforeAll
• It gets value from BeforeEach, but cannot write into it
• AfterEach does not see the value that It hasset so it gets value from BeforeEach
• Describe gets value from AfterEach, because apparently they run in the same scope and so AfterEach can write the variable
• AfterAll gets value from BeforeAll so they run in the same scope above Describe

If you got lost, don't worry, that is the point.

A curious reader might also try to initialize the $v variable before Describe, and write it after Describe and realize that AfterAll in fact runs in the script scope. This also gets highlighted if you run the snippet above a second time, then BeforeAll will report value of after all, because they both run in the script scope. This is an edge case, but seeing how a previous test run changes values in a block that is visually two scopes deep in the code makes me cringe...

# v5
Describing d
in before all v is: describe
in before each v is: before all
in it v is: before each
in after each v is: it
in after all v is: before all
    [+] i 12ms
in describe v is: after all

In v5 the situation is much clearer. The script blocks execute in order and so the value propagates as you would hopefully expect, but there are few things that need pointing out:

• notice that AfterEach has value from It, this is because BeforeEach, It and AfterEach all run in the same scope. (Personally I think this is super cool and ultra useful. 😁)
• AfterAll has value from BeforeAll because they run one scope above BeforeEach, this is needed to keep tests isolated but still be able to reach values set in BeforeAll from multiple tests.
• Describe has value from AfterAll. Frankly don't have any strong reason for that, I am still figuring out scoping for these. 🙂

BeforeAll & AfterAll failure location

Wanted to write here about how BeforeAll and AfterAll are now associated with the first and last test, but writing this I realized that it does not work properly right now. Failing the one time setup only fails the first test in v5 right now, but it should short circuit every test in that block.

(The $true prevents the test from being pending in v4, in v5 there is no pending yet.)

Describe "d" {
    BeforeAll { throw }
    It "i" { $true }
    It "i" { $true }
}

Describe "d2" {
    It "i2" { $true }
    It "i2" { $true }
    AfterAll { throw }
}

# v4
Describing d
  [-] Error occurred in Describe block 59ms
    RuntimeException: ScriptHalted
    ...stack trace

Describing d2
  [+] i2 46ms
  [+] i2 22ms
  [-] Error occurred in Describe block 8ms
    RuntimeException: ScriptHalted
    ...stack trace

# v5
Describing d
    [-] i 11ms
      RuntimeException: ScriptHalted
      ...stack trace
    [+] i 3ms

Describing d2
    [+] i2 8ms
    [-] i2 12ms
      RuntimeException: ScriptHalted
      ...stack trace

I think I got the behavior almost right. In v4 BeforeAll failure is reported for Describe block. It is a reasonable error message but it is unnecessarily difficult to see that one time setup failed. Failure in AfterAll is reported as an extra test, which for v5 is out of question as it would unnecessarily complicate re-running previous tests, graphical runners etc.

So what I am thinking is making the BeforeEach fail in the test like it does right now, and then automatically fail all the remaining tests. And for AfterAll I would fail the last test, which is where the teardown runs anyway, and give it a more reasonable message which explains that the teardown failed.

What do you think? 🙋‍

🥭 Nested blocks and their setups

This needs a lot of figuring out... and it seems utterly broken right now. So let me just sum up my ideas so someone else can think about it as well.

Right now the setups run just before the first It in the Describe, and they run only for the Its in the current Describe. Here a quick example of a complicated structure on interspersed Describes and Its:

Describe "d" {
    Describe "d.d" {
        It "i.i" { $true }
    }

    BeforeAll {
        Write-Host "before all"
        $a = "parent before all"
    }

    It "i" { Write-Host "first it" }

    Describe "d.d" {
        It "i.i" { $true }
    }

    It "i" { Write-Host "last it" }

    Describe "d.d" {
        It "i.i" { Write-Host "in nested it a is: $a" }
    }

    AfterAll {
        Write-Host "after all"
        $a = "parent after all"
    }
}

# v4
Describing d
before all

  Describing d.d
    [+] i.i 66ms
first it
  [+] i 30ms

  Describing d.d
    [+] i.i 32ms
last it
  [+] i 31ms

  Describing d.d
in nested it a is: parent before all
    [+] i.i 33ms
after all

Describing d

  Describing d.d
      [+] i.i 3ms
before all
first it
    [+] i 9ms

  Describing d.d
      [+] i.i 6ms
last it
after all
    [+] i 13ms

  Describing d.d
in nested it a is: parent after all
      [+] i.i 5ms

As you can see, even though in v5 the setup & teardown run close to the first and last test, they are also run in the Describe scope, which makes the variable $v leak into the child Describes.

The idea here was that this would allow for nesting Describes based on logical relations between the tests, and not based on how the tests are setup. This would allow for organizing Describe in a way that is independent from the test setups, and would possibly allow for multiple options of running the setups like: BeforeEach -It, BeforeEach -It -Recurse, BeforeEach -Describe -Recurse...

But now that I am thinking about it, we can already kinda do that, we cannot prevent a parent BeforeEach from running before every It but that is probably the point of putting it in a parent Describe.

What we cannot do is have It "a" and Describe "b" and have the It "a" setup differently than all the Its inside of that Describe "b", which might be nice but also can be solved by putting It "a" into its own Describe. (Yeah I am also getting lost in this :))

To achieve this separation I would need to change the execution model, because right now I invoke the tests and blocks in order, and just lookup which test / block I am currently running and invoke that in it's own scope. But to accomodate this change I would instead need to now have to maintain separate scopes for It and Describe, or run the blocks out of order - which I deliberatly chose not to to allow simpler migration from v4.

I guess this also has implications for where the BeforeAll and AfterAll blocks get executed, and where the error gets reported, and to make this even more complicated, there are Before*Block and After*Block functions implemented internally which have similar functionality.

Third option is to rec...

5.0.0-alpha1

Pester v5 - alpha1

What is new?

Test discovery

Pester Describe and It are, and always were, just plain PowerShell functions that all connect to one shared internal state. This is a great thing for extensibility, because it allows you to wrap them into foreaches, ifs and your own functions to customize how they work. BUT at the same time it prevents Pester from knowing which Describes and Its there are before execution. This makes test filtering options very limited and inefficient.

To give you an example of how bad it is imagine having 100 test files, each of them does some setup at the start to make sure the tests can run. In 1 of those 100 files is a Describe block with tag "RunThis". Invoking Pester with tag filter "RunThis", means that all 100 files will run, do their setup, and then end because their Describe does not have tag "RunThis". So if every setup took just 100ms, we would run for 10s instead of <1s.

And this only get's worse if we start talking about filtering on It level. Having 1000 tests, and running only 1 of them, still means running setups and teardowns of all 1000 tests, just to be able to run 1 of them. (And I am talking only about time, but of course there is also a lot of wasted computation involved.)

Obviously a better solution is needed, so to make this more efficient, Pester now runs every file TWICE. 😃

On the first pass, let's call it Discovery phase, all Describes are executed, and all Its, Before*s and After*s are saved. This gives back a hierarchical model of all the tests there are without actually executing anything (more on that later). This object is then inspected and filter is evaluated on every It, to see if it ShouldRun. This ShouldRun is then propagated upwards, to the Describe, it's parent Describe and finally to the file level.

Then the second pass, let's call this one Run phase, filters down to only files that have any tests to run, then further checks on every Describe block and It if it should run. Effectively running Before* and After* only where there is an It that will run.

Given the same example as above we would do a first quick pass, and then run just 1 setup out of 100 (or 1000), cutting the execution time down significantly to the time of how long it takes to discover the tests + 1 setup execution.

Now you are probably thinking: But the files still run at least once, and even worse some of them run twice so how it can be faster? So here is the catch: You need to put all your stuff in Pester controlled blocks. Here is an example:

. $PSScriptRoot\Get-Pokemon.ps1

Describe "Get pikachu by Get-Pokemon from the real api" {

    $pikachu = Get-Pokemon -Name pikachu

    It "has correct Name" -Tag IntegrationTest {
        $pikachu.Name | Should -Be "pikachu"
    }

    It "has correct Type" -Tag IntegrationTest {
        $pikachu.Type | Should -Be "electric"
    }

    It "has correct Weight" -Tag IntegrationTest {
        $pikachu.Weight | Should -Be 60
    }

    It "has correct Height" -Tag IntegrationTest {
        $pikachu.Height | Should -Be 4
    }
}

This integration test dot-sources (imports) the SUT on the top, and then in the body of the Describe it makes a call to external web API. Both the dot-sourcing and the call are not controlled by Pester, and would be invoked twice, once on Discovery and once on Run.

To fix this we use a new Pester function Add-Dependency to import the SUT only during Run, and then put the external call to BeforeAll block to run it only when any test in the containing Describe will run. Nothing more is needed.

Add-Dependency $PSScriptRoot\Get-Pokemon.ps1

Describe "Get pikachu by Get-Pokemon from the real api" {

    BeforeAll {
        $pikachu = Get-Pokemon -Name pikachu
    }

    It "has correct Name" -Tag IntegrationTest {
        $pikachu.Name | Should -Be "pikachu"
    }

    It "has correct Type" -Tag IntegrationTest {
        $pikachu.Type | Should -Be "electric"
    }

    It "has correct Weight" -Tag IntegrationTest {
        $pikachu.Weight | Should -Be 60
    }

    It "has correct Height" -Tag IntegrationTest {
        $pikachu.Height | Should -Be 4
    }
}

This makes everything controlled by Pester and we can happily run Discovery on this file without invoking anything extra.

What does this mean for the future?

This opens up a whole slew of new possibilities:

• filtering on test level
• re-running failed tests
• forcing just a single test in whole suite to run by putting a parameter like -DebuggingThis on it
• detecting changes in files and only running what changed
• a proper graphical test runner integration?

Try it out for yourself.

What else?

• The internals changed quite a bit, the result object contains captured errors and standard output, and the whole result is hieararchical. It is also split per file so it's extremely easy to combine runs from multiple suits, you simply put two arrays together.
• Scoping is changed to put the BeforeEach Test and AfterEach blocks into the same scope so variables can be shared amond them easily.
• There is work in progress on per block setups and teardowns.
  ...

All in all I am trying to address or review all the issues in this milestone

Release date?

At the moment a lot of stuff is missing, so much stuff that it's easier to say what partially works:

• Output to screen
• TestDrive
• Filtering based on tags
• PassThru (but has new format)

The other stuff that does not work yet is most notably:

• Mocking
• Code coverage
• Interactive mode
• Passing our own tests
• Gherkin

How can I try it on my own project?

Download the source code and use Pester.psm1 (yes PSM not PSD), to import it. And good luck!

Questions?

Ping me on twitter or #testing

4.5.0

What is new since 4.4.4?

TestRegistry:\

There is now a new automatic drive for your tests in Windows Registry called TestRegistry:\.

Get-ShouldOperator

Learning about all the options Should has just got infinitely simpler. Just call Get-ShouldOperator to get list of all assertions. And Get-ShouldOperator -Name <name> to get info about one of them for example Be.

Set-TestResult

Until now you could only set your tests to incoclusive with Set-TestInconclusive, but now Set-TestInconclusive is deprecated, and replaced with Set-ItResult which allows you to set the result that you choose for example -Skip, and use -Because to describe why you chose to skip that test.

Detailed CodeCoverage

The code coverage got hugely more detailed, and it now even comes in format that allows you to use Code Gutters VSCode extension to see your code coverage directly in your editor.

Other

• Pass scripts as text, not just files
• Optional session state debugging info
• Fixed Set-Variable mocking to put variables into the correct scope
• Improve Gherkin

More info and links to each issue change can be found in the changelog

4.4.4

What is new since 4.4.3

Fixes errors caused by bug in PowerShell 6 on AppVeyor Ubuntu

4.4.3

What is new since 4.4.2?

Signed

The module is now signed so first installation on Windows 10 does not show a huge red error anymore. 🔑 📦 🎉

This is how the installation looks like when installed over Pester 3.4.0 that ships with Windows 10:

The same but with -Verbose:

This is how the installtion used to look like:

Other fixes

• Bug where the module failed to load because id and uname is symlinked from multiple places was fixed, so Pester is now usable on CentOS and can print more detailed info in the TestReport.
• Custom Should operators can define multiple aliases.

4.4.2

What is new since 4.4.1

• Single quotes are no longer added around strings that are expanded in test names. (e.g. from TestCases).
• On non-windows platforms, we try to get environment info.

4.4.1

What is new since 4.4.0

• Fix filtering by tags. In the previous version all non-tagged tests were run all the time.

• Fix parameter name capitalization.