Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Wallet Backend builds and tests on Windows #703

Closed
24 of 27 tasks
KtorZ opened this issue Sep 13, 2019 · 8 comments
Closed
24 of 27 tasks

Wallet Backend builds and tests on Windows #703

KtorZ opened this issue Sep 13, 2019 · 8 comments
Assignees
@rvl
Milestone
Usability & Compa...

Comments

@KtorZ
Copy link
Member

KtorZ commented Sep 13, 2019 •
edited
Loading

Context

cardano-wallet ultimately results in two binaries as detailed in the root .cabal:

  • cardano-wallet-http-bridge
  • cardano-wallet-jormungandr

Both are command-lines and web server doing "wallet backend stuff", but working with different chain producers.

Decision

At the moment, our CI only produces binary artifacts for linux machines yet, most are using Windows and OSX machines so we ought to be able to produce executables for these targets too. Ideally, our test suites should also run on both platform to ensure that code produced works as expected on these platforms.

The focus is given on cardano-wallet-jormungandr for cardano-wallet-http-bridge will be removed soon enough and will not see the lights of production.

Acceptance Criteria

  • cardano-wallet-jormungandr must compile to Windows
  • cardano-wallet-core:test:unit must pass on Windows
  • cardano-wallet-cli:test:unit must pass on Windows
  • cardano-wallet-launcher:test:unit must pass on Windows
  • bech32:test:unit must pass on Windows
  • cardano-wallet-jormungandr:test:unit must pass on Windows
  • cardano-wallet-jormungandr:test:integration must pass on Windows
  • cardano-wallet-core:benchmark:db should pass on Windows

Development Plan

  1. Get all tests working under nix-build for Linux.
  2. Get cross-compiled tests passing under Wine.
  3. Manually test .exe on a Windows 10 VM.

QA

Checking for windows builds and tests in CI

  1. Builds for the master branch are shown in the cardano-wallet jobset.
  2. Ensure that x86_64-pc-mingw32 jobs for the packages listed in the AC are green.
  3. Check that the required job is green.
  4. Check the Constituents of the required job. These should include x86_64-pc-mingw32 jobs for the packages listed in the AC.

Running tests on windows

  1. Download the Windows testing bundle onto a Windows system.
  2. Extract to zip archive to somewhere (e.g. Desktop).
  3. Ensure old data in c:\cardano-wallet-jormungandr is removed.
  4. Use the example batch files launch.bat, cw.bat, jm.bat.
  5. Or you can invoke cardano-wallet-jormungandr.exe yourself.
  6. Run all tests with tests.bat.
  7. Or you can run test suites individually with their .exe files.

Building windows testing bundle locally

  • See Building Wiki for background info.
  • nix-build release.nix -A cardano-wallet-jormungandr-tests-win64 && ls ./result

Running tests with Wine

nix-env -f '<nixpkgs>' -iA pkgs.wineWowPackages.minimal  # Install Wine if you don't already have it.
nix-build release.nix -A x86_64-pc-mingw32.tests.cardano-wallet-core.unit.x86_64-linux  # Build a test suite
wine ./result/cardano-wallet-core-2019.11.7/unit.exe --match ...
@rvl
Copy link
Contributor

rvl commented Sep 16, 2019 •
edited
Loading

How to test cardano-wallet-jormungandr on Windows

  1. If you don't already have it, set up Windows 10 in VirtualBox.
    You can use the Powershell script and instructions at
    iohk-ops/windows/iohk-setup.ps1.
    Just delete everything in the script starting from where it installs GHC.

    Also make sure you have the
    Microsoft Visual C++ Redistributable for Visual Studio 2015
    (or maybe it was 2017, I forget now).

  2. Set up a shared folder between Linux and Windows.
    Let's call it ~/iohk/win_shared or E:.

  3. Under the cardano-wallet repo, run:

    nix-build release.nix -A cardano-wallet-jormungandr-win64
cp -vf result/*.zip ~/iohk/win_shared

    Alternatively, download the prebuilt distribution from Hydra:
    cardano-wallet-jormungandr-win64 | zip download

  4. On Windows, open E: in explorer, then right-click on the zip file
    and select "Extract All...". Extract it to somewhere like
    C:\cardano-wallet-jormungandr.

  5. Open up two command prompt windows (Start→Run cmd.exe) and cd
    to where you extracted the files.

  6. In the first window, type jm [ENTER].

  7. In the second window, type cw [ENTER].

@rvl rvl mentioned this issue Sep 16, 2019
Merged
iohk-bors bot added a commit that referenced this issue Sep 16, 2019
@iohk-bors @rvl
Merge #717
40d731a 
717: Fix windows build r=craigem a=rvl

Relates to #703 

# Overview

- I broke the cross build in 7208a54. This fixes it, and adds a necessary DLL to the packages.
- In the course of writing instructions for testing the windows build, I made a script which packages all the dependencies and configs together in a zip file. This is easy to download off Hydra and quick to test. 

# Comments

- [Link to Hydra build](https://hydra.iohk.io/job/Cardano/cardano-wallet-pr-717/cardano-wallet-jormungandr-win64/latest)
- [zip download](https://hydra.iohk.io/job/Cardano/cardano-wallet-pr-717/cardano-wallet-jormungandr-win64/latest/download-by-type/file/binary-dist)


Co-authored-by: Rodney Lorrimar <[email protected]>
@rvl rvl mentioned this issue Sep 29, 2019
Merged
10 tasks
@KtorZ
Copy link
Member Author

KtorZ commented Oct 15, 2019 •
edited
Loading

@rvl @Disasm

Can someone give a clear albeit concise status update on Windows builds?

In particular, answers to the following questions:

  1. Can we build a Windows executable that runs, without crashing, the wallet server?
  2. Can we run our unit tests on wine or Windows directly?
  3. Can we run our integration scenarios on wine or Windows?
  4. If it exists some build artifacts that are produced for Windows, where can they be found?
  5. If any answer to the questions above is "no", what are the current blockers or items being worked on?

I've got the impression that many things are happening in background but I can't clearly see what because nothing seem to be really tracked and discussions are happening.. Somewhere else. There are PR pending with no descriptions, and it's very hard to follow.

Thank you very much 🙏

@KtorZ KtorZ added this to the Usability & Compatibility milestone Oct 16, 2019
@rvl rvl assigned rvl and unassigned iohk-devops Oct 23, 2019
This was referenced Oct 23, 2019
Merged
Merged
iohk-bors bot added a commit that referenced this issue Oct 24, 2019
@iohk-bors @rvl
Merge #766
850db95 
766: Run tests in nix release build r=rvl a=rvl

Relates to #703 and #704.
Base branch is #864 - this should be merged first.

# Overview

- [x] This enables test suites under the Nix build.
- [x] Fixes test cases that used symlinks.
- [x] `Cardano.Wallet.Api.TypesSpec` is trying to access a file `/build/core/../../specifications/api/swagger.yaml` which is outside of its package. 
- [x] Some tests in `cardano-wallet-jormungandr:test:integration` required `jcli`.
- [x] Some tests in `cardano-wallet-jormungandr:test:integration` required a writable home directory.
- [x] In LauncherSpec the test scripts used `/usr/bin/env`, which is not in the nix-build sandbox, so i made the tests not need external scripts.
- [x] Adds macOS tests to the Hydra required jobs.

[Hydra jobset](https://hydra.iohk.io/jobset/Cardano/cardano-wallet-pr-766)

# Comments

- Symlinks/references to files in other packages won't work with the nix build because when it builds it copies the package source tree to `/build` and builds from there.

# Remaining failures

These windows-specific failures can be fixed in the next PR.

- [ ] `x86_64-pc-mingw32.tests.cardano-wallet-jormungandr.unit.x86_64-linux` expects UTF-8 encoding, but this is not the encoding that Wine is using.
- [ ] Windows cross build got broken. We now require a fix for Haskell.nix.
- [ ] `cardano-wallet-jormungandr.exe` would fail on windows at startup with
  ```
  NUL: openFile: does not exist (No such file or directory)
  ```


Co-authored-by: Rodney Lorrimar <[email protected]>
iohk-bors bot added a commit that referenced this issue Oct 24, 2019
@iohk-bors @rvl
Merge #906
55a7dd2 
906: nix: Update windows jormungandr version and the instructions r=piotr-iohk a=rvl

Relates to #703.

# Overview

Updates the instructions for `nix/jormungandr.nix` to include the windows binary release.


Co-authored-by: Rodney Lorrimar <[email protected]>
This was referenced Oct 25, 2019
Merged
Merged
iohk-bors bot added a commit that referenced this issue Oct 28, 2019
@iohk-bors @rvl
Merge #914
8132207 
914: Fix text locale encoding issues on windows r=KtorZ a=rvl

Relates to #703.

# Overview

- Forces the current windows console into UTF-8 mode.
- UTF-8 is far from standard on windows. But it shouldn't be assumed on any platform.
- This fixes strange characters appearing on windows and test failures with exceptions related to encoding.


Co-authored-by: Rodney Lorrimar <[email protected]>
iohk-bors bot added a commit that referenced this issue Oct 28, 2019
@iohk-bors @rvl
Merge #888
3f4449d 
888: Update Haskell.nix r=KtorZ a=rvl

Relates to #703.

# Overview

- [x] Updates the Haskell.nix version so that we can build for windows again.
- [x] Keeps more or less the same structure as before - i.e. no IFD, no niv. I don't want too much change at once.
- [x] Does however update nixpkgs from iohk fork of 18.09 → upstream 19.03.
- [x] Provides a big bundle of scripts and tests that can be run on Windows.

[Hydra jobset](https://hydra.iohk.io/jobset/Cardano/cardano-wallet-pr-888)
[Hercules jobset](https://hercules-ci.com/github/input-output-hk/cardano-wallet)
[Link to download Windows testing bundle for PR #888](https://hydra.iohk.io/job/Cardano/cardano-wallet-pr-888/cardano-wallet-jormungandr-win64/latest)
[Link to download Windows testing bundle (master branch)](https://hydra.iohk.io/job/Cardano/cardano-wallet/cardano-wallet-jormungandr-win64/latest)

# Comments

@angerman Please help

- [x] Fix failure for missing `hsc2hs`
- [x] Fix a previous mistake that affects cross builds (failing to cross-compile bash for some reason)
- [x] Worked around an [evaluation error](https://hydra.iohk.io/jobset/Cardano/cardano-wallet-pr-888#tabs-errors) because weeder and ghci are no longer available from `pkgs.haskellPackages`.
- [ ] Evaluation is taking a rather long time (5+ hours) in Hydra.
- [ ] Some of the Buildkite pipeline steps don't work with the updated Haskell.nix.

# Next PRs

- [ ] Fix failures in sqlite tests on Windows.
- [ ] Fix failure in launch command on Windows.
- [ ] Fix the UTF-8 text issues on Windows ⇒ #914.
- [ ] Add code which tells the windows command prompt to interpret ANSI color codes.
- [ ] Identify other test failures and fix.


Co-authored-by: Rodney Lorrimar <[email protected]>
iohk-bors bot added a commit that referenced this issue Oct 29, 2019
@iohk-bors @rvl
Merge #888
e6c155d 
888: Update Haskell.nix r=KtorZ a=rvl

Relates to #703.

# Overview

- [x] Updates the Haskell.nix version so that we can build for windows again.
- [x] Keeps more or less the same structure as before - i.e. no IFD, no niv. I don't want too much change at once.
- [x] Does however update nixpkgs from iohk fork of 18.09 → upstream 19.03.
- [x] Provides a big bundle of scripts and tests that can be run on Windows.

[Hydra jobset](https://hydra.iohk.io/jobset/Cardano/cardano-wallet-pr-888)
[Hercules jobset](https://hercules-ci.com/github/input-output-hk/cardano-wallet)
[Link to download Windows testing bundle for PR #888](https://hydra.iohk.io/job/Cardano/cardano-wallet-pr-888/cardano-wallet-jormungandr-win64/latest)
[Link to download Windows testing bundle (master branch)](https://hydra.iohk.io/job/Cardano/cardano-wallet/cardano-wallet-jormungandr-win64/latest)

# Comments

@angerman Please help

- [x] Fix failure for missing `hsc2hs`
- [x] Fix a previous mistake that affects cross builds (failing to cross-compile bash for some reason)
- [x] Worked around an [evaluation error](https://hydra.iohk.io/jobset/Cardano/cardano-wallet-pr-888#tabs-errors) because weeder and ghci are no longer available from `pkgs.haskellPackages`.
- [ ] Evaluation is taking a rather long time (5+ hours) in Hydra.
- [ ] Some of the Buildkite pipeline steps don't work with the updated Haskell.nix.

# Next PRs

- [ ] Fix failures in sqlite tests on Windows.
- [ ] Fix failure in launch command on Windows.
- [ ] Fix the UTF-8 text issues on Windows ⇒ #914.
- [ ] Add code which tells the windows command prompt to interpret ANSI color codes.
- [ ] Identify other test failures and fix.


Co-authored-by: Rodney Lorrimar <[email protected]>
iohk-bors bot added a commit that referenced this issue Nov 4, 2019
@iohk-bors @rvl
@cleverca22 @KtorZ
Merge #888 #952
aaead36 
888: Update Haskell.nix r=KtorZ a=rvl

Relates to #703.

# Overview

- [x] Updates the Haskell.nix version so that we can build for windows again.
- [x] Keeps more or less the same structure as before - i.e. no IFD, no niv. I don't want too much change at once.
- [x] Does however update nixpkgs from iohk fork of 18.09 → upstream 19.03.
- [x] Provides a big bundle of scripts and tests that can be run on Windows.

[Hydra jobset](https://hydra.iohk.io/jobset/Cardano/cardano-wallet-pr-888)
[Hercules jobset](https://hercules-ci.com/github/input-output-hk/cardano-wallet)
[Link to download Windows testing bundle for PR #888](https://hydra.iohk.io/job/Cardano/cardano-wallet-pr-888/cardano-wallet-jormungandr-win64/latest)
[Link to download Windows testing bundle (master branch)](https://hydra.iohk.io/job/Cardano/cardano-wallet/cardano-wallet-jormungandr-win64/latest)

# Comments

@angerman Please help

- [x] Fix failure for missing `hsc2hs`
- [x] Fix a previous mistake that affects cross builds (failing to cross-compile bash for some reason)
- [x] Worked around an [evaluation error](https://hydra.iohk.io/jobset/Cardano/cardano-wallet-pr-888#tabs-errors) because weeder and ghci are no longer available from `pkgs.haskellPackages`.
- [ ] Evaluation is taking a rather long time (5+ hours) in Hydra.
- [ ] Some of the Buildkite pipeline steps don't work with the updated Haskell.nix.
- [ ] Tests for windows are built not not executed. The tests should be run under wine.

# Next PRs

- [ ] Fix failures in sqlite tests on Windows ⇒ #918.
- [ ] Fix failure in launch command on Windows.
- [ ] Fix the UTF-8 text issues on Windows ⇒ #914.
- [ ] Add code which tells the windows command prompt to interpret ANSI color codes.
- [ ] Identify other test failures and fix.


952: review golden tests for legacy transaction and generate witnesses using jcli r=KtorZ a=KtorZ

# Issue Number

<!-- Put here a reference to the issue this PR relates to and which requirements it tackles -->

N/A

# Overview

<!-- Detail in a few bullet points the work accomplished in this PR -->

- [x] I have reviewed the legacy transaction golden tests, now fully using jcli to also generate the witnesses (this feature was missing when the golden tests were initially implemented). 

- [x] I have updated the comment on private key to reflect the real format expected by jcli 

# Comments

<!-- Additional comments or screenshots to attach if any -->

A bit disappointing to some extend since nothing really has changed.. It seems that we were already providing the right output :( 

<!-- 
Don't forget to:

 ✓ Self-review your changes to make sure nothing unexpected slipped through
 ✓ Assign yourself to the PR
 ✓ Assign one or several reviewer(s)
 ✓ Once created, link this PR to its corresponding ticket
 ✓ Acknowledge any changes required to the Wiki
-->


Co-authored-by: Rodney Lorrimar <[email protected]>
Co-authored-by: Michael Bishop <[email protected]>
Co-authored-by: KtorZ <[email protected]>
iohk-bors bot added a commit that referenced this issue Nov 4, 2019
@iohk-bors @rvl @cleverca22
Merge #888
f75f38d 
888: Update Haskell.nix r=KtorZ a=rvl

Relates to #703.

# Overview

- [x] Updates the Haskell.nix version so that we can build for windows again.
- [x] Keeps more or less the same structure as before - i.e. no IFD, no niv. I don't want too much change at once.
- [x] Does however update nixpkgs from iohk fork of 18.09 → upstream 19.03.
- [x] Provides a big bundle of scripts and tests that can be run on Windows.

[Hydra jobset](https://hydra.iohk.io/jobset/Cardano/cardano-wallet-pr-888)
[Hercules jobset](https://hercules-ci.com/github/input-output-hk/cardano-wallet)
[Link to download Windows testing bundle for PR #888](https://hydra.iohk.io/job/Cardano/cardano-wallet-pr-888/cardano-wallet-jormungandr-win64/latest)
[Link to download Windows testing bundle (master branch)](https://hydra.iohk.io/job/Cardano/cardano-wallet/cardano-wallet-jormungandr-win64/latest)

# Comments

@angerman Please help

- [x] Fix failure for missing `hsc2hs`
- [x] Fix a previous mistake that affects cross builds (failing to cross-compile bash for some reason)
- [x] Worked around an [evaluation error](https://hydra.iohk.io/jobset/Cardano/cardano-wallet-pr-888#tabs-errors) because weeder and ghci are no longer available from `pkgs.haskellPackages`.
- [ ] Evaluation is taking a rather long time (5+ hours) in Hydra.
- [ ] Some of the Buildkite pipeline steps don't work with the updated Haskell.nix.
- [ ] Tests for windows are built not not executed. The tests should be run under wine.

# Next PRs

- [ ] Fix failures in sqlite tests on Windows ⇒ #918.
- [ ] Fix failure in launch command on Windows.
- [ ] Fix the UTF-8 text issues on Windows ⇒ #914.
- [ ] Add code which tells the windows command prompt to interpret ANSI color codes.
- [ ] Identify other test failures and fix.


Co-authored-by: Rodney Lorrimar <[email protected]>
Co-authored-by: Michael Bishop <[email protected]>
iohk-bors bot added a commit that referenced this issue Nov 5, 2019
@iohk-bors @KtorZ @rvl
Merge #918
ba35bdc 
918: Fix Sqlite DB on Windows r=rvl a=rvl

Relates to #703.
Based on the branch of #888.

# Overview

The wallet failed with:

    NUL: openFile: does not exist (No such file or directory)

This is due to runMigrationSilent which uses the silently library:

http://hackage.haskell.org/package/silently-1.2.5.1/docs/src/System.IO.Silently.html#line-27

To fix, we swap `runMigrationQuiet` for `runMigrationSilent` on Windows.


Co-authored-by: KtorZ <[email protected]>
Co-authored-by: iohk-bors[bot] <43231472+iohk-bors[bot]@users.noreply.github.com>
Co-authored-by: Rodney Lorrimar <[email protected]>
This was referenced Nov 5, 2019
Merged
Merged
iohk-bors bot added a commit that referenced this issue Nov 5, 2019
@iohk-bors @rvl
Merge #918
6f435df 
918: Fix Sqlite DB on Windows r=rvl a=rvl

Relates to #703.
Based on the branch of #888.

# Overview

The wallet failed with:

    NUL: openFile: does not exist (No such file or directory)

This is due to runMigrationSilent which uses the silently library:

http://hackage.haskell.org/package/silently-1.2.5.1/docs/src/System.IO.Silently.html#line-27

To fix, we swap `runMigrationQuiet` for `runMigrationSilent` on Windows.


Co-authored-by: Rodney Lorrimar <[email protected]>
@rvl rvl mentioned this issue Nov 5, 2019
Merged
iohk-bors bot added a commit that referenced this issue Nov 5, 2019
@iohk-bors @rvl
Merge #918
1fbd206 
918: Fix Sqlite DB on Windows r=rvl a=rvl

Relates to #703.
Based on the branch of #888.

# Overview

The wallet failed with:

    NUL: openFile: does not exist (No such file or directory)

This is due to runMigrationSilent which uses the silently library:

http://hackage.haskell.org/package/silently-1.2.5.1/docs/src/System.IO.Silently.html#line-27

To fix, we swap `runMigrationQuiet` for `runMigrationSilent` on Windows.


Co-authored-by: Rodney Lorrimar <[email protected]>
iohk-bors bot added a commit that referenced this issue Nov 6, 2019
@iohk-bors @rvl
Merge #958
6d2e31d 
958: nix: Add necessary test data files to windows testing bundle r=rvl a=rvl

Relates to #703.

# Overview

- Includes test data in the zip file so that tests can pass.
- Fixes `launch.bat`
- Updates the Haskell.nix revision so that it's on the master branch, not a PR branch.

# Comments

Testing procedure is in #703.


Co-authored-by: Rodney Lorrimar <[email protected]>
iohk-bors bot added a commit that referenced this issue Nov 6, 2019
@iohk-bors @rvl
Merge #957
e28bfe8 
957: cli: ANSI colour for windows 10, and easier utf-8 setup r=rvl a=rvl

Relates to #703.

# Overview

IOHK logging framework uses ANSI control codes for colouring its output (instead of the Win32-specific method). Windows console does not interpret ANSI colour codes unless told to (even then, it's only supported under win 10).

- Enables ANSI colour at startup ([System.Console.ANSI](http://hackage.haskell.org/package/ansi-terminal-0.10.1/docs/System-Console-ANSI.html#v:hSupportsANSIWithoutEmulation)).
- There is a better way of setting UTF-8 on Windows - [code-page package](http://hackage.haskell.org/package/code-page-0.2/docs/System-IO-CodePage.html#v:withCP65001).


Co-authored-by: Rodney Lorrimar <[email protected]>
iohk-bors bot added a commit that referenced this issue Nov 6, 2019
@iohk-bors @rvl
Merge #959
b74a169 
959: LauncherSpec: Let it work on Windows r=rvl a=rvl

Relates to #703.

# Overview

- LauncherSpec used posix commands for testing.
- This substitutes equivalent commands when running on windows.


Co-authored-by: Rodney Lorrimar <[email protected]>
@rvl rvl mentioned this issue Nov 7, 2019
Merged
This was referenced Nov 7, 2019
Merged
Merged
iohk-bors bot added a commit that referenced this issue Nov 7, 2019
@iohk-bors @rvl @KtorZ
Merge #971
977ceff 
971: tests: Fix ServerSpec and SqliteSpec on windows r=KtorZ a=rvl

Relates to #703.

# Overview

Windows unit test fixes.

-  Let ServerSpec error path tests pass.
-  Close temp file so that SqliteSpec passes.

# Comments

Cross-compiled tests are not yet automatically run under Haskell.nix with Wine (cc @angerman @hamishmack). So am testing with:
```
nix-env -f '<nixpkgs>' -iA pkgs.wineWowPackages.minimal
wine $(nix-build release.nix -A x86_64-pc-mingw32.tests.cardano-wallet-core.unit.x86_64-linux)/cardano-wallet-core-2019.11.6/unit.exe
```


Co-authored-by: Rodney Lorrimar <[email protected]>
Co-authored-by: KtorZ <[email protected]>
@KtorZ KtorZ mentioned this issue Nov 7, 2019
Merged
2 tasks
iohk-bors bot added a commit that referenced this issue Nov 8, 2019
@iohk-bors @KtorZ
Merge #981
c328fd9 
981: Make API Golden tests filenames compatible with Windows r=rvl a=KtorZ

# Issue Number

<!-- Put here a reference to the issue this PR relates to and which requirements it tackles -->

#703  

# Overview

<!-- Detail in a few bullet points the work accomplished in this PR -->

- [x] I have revised api golden test filenames generation to only use alpha-num characters
- [x] I have re-generated all api golden tests

# Comments

<!-- Additional comments or screenshots to attach if any -->

```
$ ls
AddressAmountTestnet.json         ApiNetworkTip.json                               ApiTimeReference.json                ApiTWalletId.json              ByronWalletPostData.json
ApiAddressTestnet.json            ApiStakePool.json                                ApiTPassphraseencryption.json        ApiTWalletName.json            PostTransactionDataTestnet.json
ApiBlockReference.json            ApiStakePoolMetrics.json                         ApiTransactionTestnet.json           ApiTWalletPassphraseInfo.json  PostTransactionFeeDataTestnet.json
ApiByronWallet.json               ApiTAddressPoolGap.json                          ApiTSyncProgress.json                ApiTxId.json                   WalletPostData.json
ApiByronWalletMigrationInfo.json  ApiTAddressProxyNetworkDiscriminantTestnet.json  ApiTTxStatus.json                    ApiUtxoStatistics.json         WalletPutData.json
ApiFee.json                       ApiTDirection.json                               ApiTWalletBalance.json               ApiWallet.json                 WalletPutPassphraseData.json
ApiNetworkInformation.json        ApiTHashTx.json                                  ApiTWalletDelegationApiTPoolId.json  ApiWalletPassphrase.json
```

<!-- 
Don't forget to:

 ✓ Self-review your changes to make sure nothing unexpected slipped through
 ✓ Assign yourself to the PR
 ✓ Assign one or several reviewer(s)
 ✓ Once created, link this PR to its corresponding ticket
 ✓ Acknowledge any changes required to the Wiki
-->


Co-authored-by: KtorZ <[email protected]>
iohk-bors bot added a commit that referenced this issue Nov 8, 2019
@iohk-bors @KtorZ
Merge #981
80ddeeb 
981: Make API Golden tests filenames compatible with Windows r=rvl a=KtorZ

# Issue Number

<!-- Put here a reference to the issue this PR relates to and which requirements it tackles -->

#703  

# Overview

<!-- Detail in a few bullet points the work accomplished in this PR -->

- [x] I have revised api golden test filenames generation to only use alpha-num characters
- [x] I have re-generated all api golden tests

# Comments

<!-- Additional comments or screenshots to attach if any -->

```
$ ls
AddressAmountTestnet.json         ApiNetworkTip.json                               ApiTimeReference.json                ApiTWalletId.json              ByronWalletPostData.json
ApiAddressTestnet.json            ApiStakePool.json                                ApiTPassphraseencryption.json        ApiTWalletName.json            PostTransactionDataTestnet.json
ApiBlockReference.json            ApiStakePoolMetrics.json                         ApiTransactionTestnet.json           ApiTWalletPassphraseInfo.json  PostTransactionFeeDataTestnet.json
ApiByronWallet.json               ApiTAddressPoolGap.json                          ApiTSyncProgress.json                ApiTxId.json                   WalletPostData.json
ApiByronWalletMigrationInfo.json  ApiTAddressProxyNetworkDiscriminantTestnet.json  ApiTTxStatus.json                    ApiUtxoStatistics.json         WalletPutData.json
ApiFee.json                       ApiTDirection.json                               ApiTWalletBalance.json               ApiWallet.json                 WalletPutPassphraseData.json
ApiNetworkInformation.json        ApiTHashTx.json                                  ApiTWalletDelegationApiTPoolId.json  ApiWalletPassphrase.json
```

<!-- 
Don't forget to:

 ✓ Self-review your changes to make sure nothing unexpected slipped through
 ✓ Assign yourself to the PR
 ✓ Assign one or several reviewer(s)
 ✓ Once created, link this PR to its corresponding ticket
 ✓ Acknowledge any changes required to the Wiki
-->


Co-authored-by: KtorZ <[email protected]>
@rvl rvl mentioned this issue Nov 8, 2019
Merged
5 tasks
iohk-bors bot added a commit that referenced this issue Nov 8, 2019
@iohk-bors @rvl @KtorZ
Merge #975 #980
15175c9 
975: Fix bench-db on windows r=KtorZ a=rvl

Relates to #703.

# Overview

The database benchmark was failing with:

```
cardano-wallet-core-2019.11.6-bench-db.exe: C:\users\rodney\Temp\benf0f5.db: DeleteFile "\\\\?\\C:\\users\\rodney\\Temp\\benf0f5.db": permission denied (Sharing violation.)
```

And:

```
db.exe: <stdout>: commitBuffer: invalid argument (Invalid argument)
```

It works on windows but not under wine (encoding problem).



980: Move delegation certificate declaration in a separate table r=KtorZ a=KtorZ

# Issue Number

<!-- Put here a reference to the issue this PR relates to and which requirements it tackles -->

#913

# Overview

<!-- Detail in a few bullet points the work accomplished in this PR -->

- [x] I have moved tracking of delegation "certificate" into a separate SQLite table
- [x] I have adjusted `readWalletMetadata` to now pull the delegation status from the right table 
- [x] I have added a `putDelegationCertificate` function to insert a new row in the database when new certificates are discovered
- [x] I have extended the state-machine tests accordingly, with proper tagging

# Comments

<!-- Additional comments or screenshots to attach if any -->

```
Cardano.Wallet.DB.Sqlite
  Sqlite State machine tests
    Sequential
      +++ OK, passed 400 tests:
      65.8% UnsuccessfulReadTxHistory
      57.8% SuccessfulReadCheckpoint
      57.2% CreateThenList
      55.0% TxUnsortedInputs
      54.8% TxUnsortedOutputs
      49.0% CreateWalletTwice
      32.5% ReadTxHistoryAfterDelete
      28.5% PutCheckpointTwice
      27.8% UnsuccessfulReadCheckpoint
      24.8% ReadMetaAfterPutCert
      22.5% CreateThreeWallets
      18.5% RemovePendingTxTwice
      17.5% RolledBackOnce
      14.2% SuccessfulReadPrivateKey
      13.2% SuccessfulReadTxHistory
      12.0% RemoveWalletTwice
    Parallel
      # PENDING: No reason given
  Sqlite State machine (RndState)
    Sequential state machine tests
      +++ OK, passed 400 tests:
      65.8% UnsuccessfulReadTxHistory
      57.8% SuccessfulReadCheckpoint
      57.2% CreateThenList
      55.0% TxUnsortedInputs
      54.8% TxUnsortedOutputs
      49.0% CreateWalletTwice
      32.5% ReadTxHistoryAfterDelete
      28.5% PutCheckpointTwice
      27.8% UnsuccessfulReadCheckpoint
      24.8% ReadMetaAfterPutCert
      22.5% CreateThreeWallets
      18.5% RemovePendingTxTwice
      17.5% RolledBackOnce
      14.2% SuccessfulReadPrivateKey
      13.2% SuccessfulReadTxHistory
      12.0% RemoveWalletTwice
```

<!-- 
Don't forget to:

 ✓ Self-review your changes to make sure nothing unexpected slipped through
 ✓ Assign yourself to the PR
 ✓ Assign one or several reviewer(s)
 ✓ Once created, link this PR to its corresponding ticket
 ✓ Acknowledge any changes required to the Wiki
-->


Co-authored-by: Rodney Lorrimar <[email protected]>
Co-authored-by: KtorZ <[email protected]>
iohk-bors bot added a commit that referenced this issue Nov 11, 2019
@iohk-bors @rvl
Merge #974
078ab8b 
974: tests: Fix Network.Wai.Middleware.LoggingSpec on Windows r=KtorZ a=rvl

Relates to #703.

# Overview

```
  test/unit/Network/Wai/Middleware/LoggingSpec.hs:109:5:
  9) Network.Wai.Middleware.Logging, Logging Middleware, GET, 200, no query
       uncaught exception: IOException of type UserError
       user error (Expected exactly 3 log entries but got 1: [LogObject {loName = "request-0", loMeta = LOMeta {tstamp = 2019-11-05 04:43:05.8652477 UTC, tid = "ThreadId 13863", severity = Info, privacy = Public}, loContent = LogMessage "[GET] /get"}])

  To rerun use: --match "/Network.Wai.Middleware.Logging/Logging Middleware/GET, 200, no query/"

  test/unit/Network/Wai/Middleware/LoggingSpec.hs:117:5:
  10) Network.Wai.Middleware.Logging, Logging Middleware, GET, 200, with query
       uncaught exception: IOException of type UserError
       user error (Expected exactly 3 log entries but got 5: [LogObject {loName = "request-0", loMeta = LOMeta {tstamp = 2019-11-05 04:43:05.8652477 UTC, tid = "ThreadId 13863", severity = Info, privacy = Public}, loContent = LogMessage "200 OK in 0s"},LogObject {loName = "request-0", loMeta = LOMeta {tstamp = 2019-11-05 04:43:05.8652477 UTC, tid = "ThreadId 13863", severity = Debug, privacy = Public}, loContent = LogMessage "14"},LogObject {loName = "request-1", loMeta = LOMeta {tstamp = 2019-11-05 04:43:05.8652477 UTC, tid = "ThreadId 13863", severity = Info, privacy = Public}, loContent = LogMessage "[GET] /get?query=patate"},LogObject {loName = "request-1", loMeta = LOMeta {tstamp = 2019-11-05 04:43:05.8652477 UTC, tid = "ThreadId 13863", severity = Info, privacy = Public}, loContent = LogMessage "200 OK in 0s"},LogObject {loName = "request-1", loMeta = LOMeta {tstamp = 2019-11-05 04:43:05.8652477 UTC, tid = "ThreadId 13863", severity = Debug, privacy = Public}, loContent = LogMessage "14"}])

  To rerun use: --match "/Network.Wai.Middleware.Logging/Logging Middleware/GET, 200, with query/"

  test/unit/Network/Wai/Middleware/LoggingSpec.hs:184:13:
  11) Network.Wai.Middleware.Logging, Logging Middleware, different request ids
       Falsifiable (after 4 tests and 6 shrinks):
         NumberOfRequests 1
       expected: 1
        but got: 48

  To rerun use: --match "/Network.Wai.Middleware.Logging/Logging Middleware/different request ids/"

  test/unit/Network/Wai/Middleware/LoggingSpec.hs:198:13:
  12) Network.Wai.Middleware.Logging, Logging Middleware, correct time measures
       Falsifiable (after 1 test):
         (NumberOfRequests 5,RandomIndex 1)
       expected: 1
        but got: 0

  To rerun use: --match "/Network.Wai.Middleware.Logging/Logging Middleware/correct time measures/"

Randomized with seed 534946986

Finished in 629.7711 seconds
915 examples, 12 failures, 1 pending

C:\Users\win\cw>
```

- This "fixes" the tests on windows.
- "different request ids" test still fails

# Comments

It seems like iohk-monitoring writes its logs asynchronously.
Or there is some other race condition with the TVar containing log messages.


Co-authored-by: Rodney Lorrimar <[email protected]>
iohk-bors bot added a commit that referenced this issue Nov 11, 2019
@iohk-bors @rvl
Merge #974
f8dcd12 
974: tests: Fix Network.Wai.Middleware.LoggingSpec on Windows r=rvl a=rvl

Relates to #703.

# Overview

```
  test/unit/Network/Wai/Middleware/LoggingSpec.hs:109:5:
  9) Network.Wai.Middleware.Logging, Logging Middleware, GET, 200, no query
       uncaught exception: IOException of type UserError
       user error (Expected exactly 3 log entries but got 1: [LogObject {loName = "request-0", loMeta = LOMeta {tstamp = 2019-11-05 04:43:05.8652477 UTC, tid = "ThreadId 13863", severity = Info, privacy = Public}, loContent = LogMessage "[GET] /get"}])

  To rerun use: --match "/Network.Wai.Middleware.Logging/Logging Middleware/GET, 200, no query/"

  test/unit/Network/Wai/Middleware/LoggingSpec.hs:117:5:
  10) Network.Wai.Middleware.Logging, Logging Middleware, GET, 200, with query
       uncaught exception: IOException of type UserError
       user error (Expected exactly 3 log entries but got 5: [LogObject {loName = "request-0", loMeta = LOMeta {tstamp = 2019-11-05 04:43:05.8652477 UTC, tid = "ThreadId 13863", severity = Info, privacy = Public}, loContent = LogMessage "200 OK in 0s"},LogObject {loName = "request-0", loMeta = LOMeta {tstamp = 2019-11-05 04:43:05.8652477 UTC, tid = "ThreadId 13863", severity = Debug, privacy = Public}, loContent = LogMessage "14"},LogObject {loName = "request-1", loMeta = LOMeta {tstamp = 2019-11-05 04:43:05.8652477 UTC, tid = "ThreadId 13863", severity = Info, privacy = Public}, loContent = LogMessage "[GET] /get?query=patate"},LogObject {loName = "request-1", loMeta = LOMeta {tstamp = 2019-11-05 04:43:05.8652477 UTC, tid = "ThreadId 13863", severity = Info, privacy = Public}, loContent = LogMessage "200 OK in 0s"},LogObject {loName = "request-1", loMeta = LOMeta {tstamp = 2019-11-05 04:43:05.8652477 UTC, tid = "ThreadId 13863", severity = Debug, privacy = Public}, loContent = LogMessage "14"}])

  To rerun use: --match "/Network.Wai.Middleware.Logging/Logging Middleware/GET, 200, with query/"

  test/unit/Network/Wai/Middleware/LoggingSpec.hs:184:13:
  11) Network.Wai.Middleware.Logging, Logging Middleware, different request ids
       Falsifiable (after 4 tests and 6 shrinks):
         NumberOfRequests 1
       expected: 1
        but got: 48

  To rerun use: --match "/Network.Wai.Middleware.Logging/Logging Middleware/different request ids/"

  test/unit/Network/Wai/Middleware/LoggingSpec.hs:198:13:
  12) Network.Wai.Middleware.Logging, Logging Middleware, correct time measures
       Falsifiable (after 1 test):
         (NumberOfRequests 5,RandomIndex 1)
       expected: 1
        but got: 0

  To rerun use: --match "/Network.Wai.Middleware.Logging/Logging Middleware/correct time measures/"

Randomized with seed 534946986

Finished in 629.7711 seconds
915 examples, 12 failures, 1 pending

C:\Users\win\cw>
```

- This "fixes" the tests on windows.
- "different request ids" test still fails

# Comments

It seems like iohk-monitoring writes its logs asynchronously.
Or there is some other race condition with the TVar containing log messages.


Co-authored-by: Rodney Lorrimar <[email protected]>
@rvl rvl mentioned this issue Nov 12, 2019
Merged
1 task
iohk-bors bot added a commit that referenced this issue Nov 12, 2019
@iohk-bors @rvl
Merge #986
89bebd4 
986: LauncherSpec: Check that all launched processes do exit r=KtorZ a=rvl

Relates to #703.

# Overview

It appears that the `jormungandr` node backend process was not getting cleaned up on Windows and that actions which are supposed to run concurrently with the backend were not running.

These extra tests check that the [`withCreateProcess`](http://hackage.haskell.org/package/process-1.6.6.0/docs/System-Process.html#v:withCreateProcess) function takes care of terminating the process -- if it is given a chance (i.e. if the process is not killed with -9, or "End Task" on Windows).

It also checks that the concurrent actions run while the backend process is running, and that the backend process is terminated when the other action completes.

Finally, it implements a workaround for the unwanted behaviour of the `process` library on Windows where `waitForProcess` seems to block all concurrent async actions in the thread.

- [x] Adds a test to `cardano-wallet-launcher:test:unit` for process clean up.
- [x] Adjusts commands used so that the tests can be run under Wine.
- [x] Adds an assertion to check that the process is killed if the action does not complete.
- [x] Adds an assertion to check that the the process is killed promptly if the action completes.
- [x] Fixes async blocking issue on Windows.

### Testing under Wine

Use something like this:
```
wine $(nix-build release.nix -A x86_64-pc-mingw32.tests.cardano-wallet-launcher.unit.x86_64-linux -o launcher-unit-windows)/cardano-wallet-launcher-2019.11.7/unit.exe --match "Backend process"
```


Co-authored-by: Rodney Lorrimar <[email protected]>
This was referenced Nov 12, 2019
Merged
Merged
iohk-bors bot added a commit that referenced this issue Nov 13, 2019
@iohk-bors @rvl
Merge #1016 #1017
ae18800 
1016: Unit tests passing on both Windows and Wine r=KtorZ a=rvl

Relates to #703. 

# Overview

Fixes/disables some of the last tests that were failing on Windows for Microsoft reasons.


1017: Update stylish-haskell to latest (v0.9.4.4) r=KtorZ a=rvl

Updates stylish-haskell to the latest.

I mistakenly thought that the latest version in a Stackage LTS would be recent and downgraded it before.

To test:

```
nix-shell --run "stylish-haskell --version"
```


Co-authored-by: Rodney Lorrimar <[email protected]>
iohk-bors bot added a commit that referenced this issue Nov 13, 2019
@iohk-bors @rvl
Merge #1015
b69dcae 
1015: nix: Make a separate testing bundle for windows r=KtorZ a=rvl

Relates to #703.

# Overview

- Moves the test executables and test data out of the windows release zip and into a separate archive.

# Comments

- [release job](https://hydra.iohk.io/job/Cardano/cardano-wallet-pr-1015/cardano-wallet-jormungandr-win64/latest)
- [testing bundle job](https://hydra.iohk.io/job/Cardano/cardano-wallet-pr-1015/cardano-wallet-jormungandr-tests-win64/latest)


Co-authored-by: Rodney Lorrimar <[email protected]>
iohk-bors bot added a commit that referenced this issue Nov 14, 2019
@iohk-bors @rvl
@KtorZ @Anviking
Merge #1011 #1028
6873b98 
1011: Can run Windows integration tests under Wine r=rvl a=rvl

Relates to #703.

# Overview

- [ ] It's possible to execute `cardano-wallet-jormungandr:test:integration` under Wine.

# Comments

The tests run on Windows but there is something weird going on with Wine.

![2019-11-12-14-30-53-wine-integration-tests](https://user-images.githubusercontent.com/1019641/68679174-19e1db00-05ab-11ea-9c62-8a30e2807f77.png)

To build and run the tests under wine, use:
```
wine $(nix-build release.nix -A x86_64-pc-mingw32.tests.cardano-wallet-jormungandr.integration.x86_64-linux -o integration-windows)/cardano-wallet-jormungandr-2019.11.7/integration.exe
```


1028: Fix the initial sleep delay in the chain following r=KtorZ a=Anviking


# Issue Number

#1027


# Overview

<!-- Detail in a few bullet points the work accomplished in this PR -->

- [x] I have changed the initial sleep delay from `0` to `1 s`.


# Comments

- Minor downside: we sleep for the initial delay on startup.

<!-- Additional comments or screenshots to attach if any -->

<!-- 
Don't forget to:

 ✓ Self-review your changes to make sure nothing unexpected slipped through
 ✓ Assign yourself to the PR
 ✓ Assign one or several reviewer(s)
 ✓ Once created, link this PR to its corresponding ticket
 ✓ Acknowledge any changes required to the Wiki
-->


Co-authored-by: Rodney Lorrimar <[email protected]>
Co-authored-by: KtorZ <[email protected]>
Co-authored-by: Johannes Lund <[email protected]>
@KtorZ
Copy link
Member Author

KtorZ commented Nov 20, 2019

@rvl How are we doing with wine tests execution here? Wine doesn't work for me locally for a.. mysterious reason and as far as I can see, the Hydra job doesn't really work either.

@rvl
Copy link
Contributor

rvl commented Nov 21, 2019

We are waiting for execution of wine tests to be put back - input-output-hk/haskell.nix#316. Currently it builds the test suite for windows but does not run it, so it's always green in Hydra.

I don't know what version and configuration of wine you were using. But the version used in the nix build is capable of running the test suite and jormungandr.exe.

@KtorZ
Copy link
Member Author

KtorZ commented Nov 22, 2019

I am using the "default" one from aptitude on Ubuntu 18.04, so it's probably not the most recent version as usual ^^" ...
I'll try getting it installed through nix and see.

@rvl rvl mentioned this issue Dec 3, 2019
Merged
7 tasks
@piotr-iohk piotr-iohk mentioned this issue Dec 4, 2019
Closed
iohk-bors bot added a commit that referenced this issue Dec 4, 2019
@iohk-bors @rvl
Merge #1109
da60f8c 
1109: Test.Utils.Paths: Fix the test data directory for Windows and Nix r=rvl a=rvl

# Issue Number

#703 

# Overview

When running tests under either Windows or Nix, we want the test data directory to be relative to the current directory, rather than an absolute path.

# Comments

Tests pass on Windows:

- [x] `cardano-wallet-core-2019.11.18-test-unit.exe`
- [x] `cardano-wallet-jormungandr-2019.11.18-test-unit.exe`
- [ ] `cardano-wallet-jormungandr-2019.11.18-test-integration.exe` (_running_)
- [x] `cardano-wallet-launcher-2019.11.18-test-unit.exe`
- [x] `cardano-wallet-cli-2019.11.18-test-unit.exe`
- [x] `bech32-2019.11.18-bech32-test.exe`
- [x] `text-class-2019.11.18-test-unit.exe`


Co-authored-by: Rodney Lorrimar <[email protected]>
iohk-bors bot added a commit that referenced this issue Dec 4, 2019
@iohk-bors @rvl
Merge #1109
2c6cab9 
1109: Test.Utils.Paths: Fix the test data directory for Windows and Nix r=KtorZ a=rvl

# Issue Number

#703 

# Overview

When running tests under either Windows or Nix, we want the test data directory to be relative to the current directory, rather than an absolute path.

# Comments

Tests pass on Windows:

- [x] `cardano-wallet-core-2019.11.18-test-unit.exe`
- [x] `cardano-wallet-jormungandr-2019.11.18-test-unit.exe`
- [ ] `cardano-wallet-jormungandr-2019.11.18-test-integration.exe` (_running_)
- [x] `cardano-wallet-launcher-2019.11.18-test-unit.exe`
- [x] `cardano-wallet-cli-2019.11.18-test-unit.exe`
- [x] `bech32-2019.11.18-bech32-test.exe`
- [x] `text-class-2019.11.18-test-unit.exe`


Co-authored-by: Rodney Lorrimar <[email protected]>
@piotr-iohk
Copy link
Contributor

@KtorZ KtorZ added this to QA in cardano-wallet 2 hours ago

Does it mean that tests are running on wine in CI?

@KtorZ
Copy link
Member Author

KtorZ commented Dec 11, 2019

Whoops. My bad, I messed up when setting up the GH board

@piotr-iohk piotr-iohk mentioned this issue Jan 8, 2020
Closed
1 task
@KtorZ
Copy link
Member Author

KtorZ commented Jan 10, 2020

closing this one. Tests are now running in hydra:

unit

integration

Some jobs are still failing but this is tracked by #1115

@KtorZ KtorZ closed this as completed Jan 10, 2020
@piotr-iohk piotr-iohk mentioned this issue Jan 20, 2020
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@rvl rvl

Labels
None yet
Projects
None yet
Milestone
Usability & Compatibility
Development

No branches or pull requests
4 participants
@rvl @KtorZ @iohk-devops @piotr-iohk