diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
index fe7a23e..841c338 100644
--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
@@ -36,14 +36,8 @@ bin/prepare-release.sh 1.2.3
 
 A new draft PR will now have been created, and its branch checked out locally.
 
-### Test the release
-You can use [`wp-env`](https://developer.wordpress.org/block-editor/reference-guides/packages/packages-env/) to run a local WordPress instance with the plugin installed:
-
-```shell
-wp-env start
-```
-
-In your browser, navigate to the URL that `wp-env` wrote to the terminal and make sure that the plugin is working as expected.
+### Run the tests
+You must make sure tests pass before publishing a new release. See [End-to-end tests](tests/README.md) for instructions on running the tests.
 
 ### Add a Changelog
 A Changelog must be added to the `Changelog` section of `README.md`. In the PR description, you can find a link to all the commits since the previous release. You should manually go through the list and identify merged PRs that should be included in the Changelog (i.e. PRs that result in user-facing changes).
diff --git a/tests/README.md b/tests/README.md
new file mode 100644
index 0000000..0261275
--- /dev/null
+++ b/tests/README.md
@@ -0,0 +1,32 @@
+# End-to-end tests
+
+Running theses tests requires having [matrix-oidc-playground](https://github.com/Automattic/matrix-oidc-playground/) running in the same machine as the tests. Make sure to follow the setup instructions there before running the tests.
+
+Once you have matrix-oidc-playground running, simply run:
+
+```shell
+composer test
+```
+
+The tests pass when the output ends with something like:
+
+
+```shell
+JWT token {
+  iss: 'https://localhost:8443/',
+  sub: 'admin',
+  aud: 'oidc-server-plugin-tests',
+  iat: 1695316090,
+  exp: 1695319690,
+  auth_time: 1695316090,
+  nonce: '7926217c4ad37e6db5cc8e6f78a421ed'
+}
+userinfo {
+  scope: 'openid profile',
+  username: 'admin',
+  name: 'admin',
+  nickname: 'admin',
+  picture: 'https://secure.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=96&d=mm&r=g',
+  sub: 'admin'
+}
+```