-
Notifications
You must be signed in to change notification settings - Fork 33
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add quickstart instructions for iree_tests #306
base: main
Are you sure you want to change the base?
Conversation
|
||
# Install required packages | ||
pip install -r iree_tests/requirements.txt | ||
pip install ./common_tools/ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm considering folding common_tools
back into iree_tests
to this extra setup step isn't needed, FYI.
# Install Git LFS | ||
git lfs install |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It helps to run this before cloning the repo IIRC. I'm not sure if installing git lfs will automatically fetch blobs if you had previously cloned/fetched without git lfs.
echo "Setup complete! You can now run IREE tests." | ||
echo "Example: pytest iree_tests -n auto" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We rarely ever run all tests with the same config file. Consider the simple case of CPU:
- https://github.com/nod-ai/SHARK-TestSuite/blob/main/iree_tests/configs/onnx_cpu_llvm_sync.json
- https://github.com/nod-ai/SHARK-TestSuite/blob/main/iree_tests/configs/models_cpu_llvm_task.json
The ONNX tests run single threaded (--device=local-sync') and have lists of XFAIL'd ONNX tests. The model tests run multithreaded (
--device=local-task), enable some baseline optimizations (
--iree-llvmcpu-target-cpu-features=host`), and have lists of XFAIL'd model tests.
I'd put a command here that is expected to work out of the box. Maybe something more like pytest iree_tests -n auto -k "test_sub_"
(see "Run a filtered subset of tests" further down in this file) or pytest iree_tests/onnx -n auto
.
``` | ||
#!/bin/bash |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: add an explicit language when known to code blocks, to help markdown renderers apply syntax highlighting
``` | |
#!/bin/bash | |
```bash | |
#!/bin/bash |
@@ -1,5 +1,39 @@ | |||
# IREE Tests | |||
|
|||
## Quickstart instructions |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This duplicates the setup at https://github.com/nod-ai/SHARK-TestSuite/tree/main/iree_tests#common-venv-setup-with-deps . Want to merge the two?
- Keep it as "quickstart" near the top
- Keep as a single code block for easier copy/paste
- Add a line or two for local build dir and
$PATH
vs python packages - Move the overview text ("A conftest.py file collects test cases from subdirectories, wrapping ...") to the newly labeled "About" section
No description provided.