You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This issue will be used to track compilation failures for migraphx models on CPU and GPU. Compile failures for each model should have a link to an issue with a smaller reproducer in the notes column.
Notes:
migraphx_ORT__bert_base_cased_1 fails on CPU but passes on GPU. Other adjacent models fail for similar reasons on both. Very odd.
not including tests migraphx_sdxl__unet__model, migraphx_ORT__bert_large_uncased_1 because they cause a crash (likely OOM)
Test was run with the following arguments:
Namespace(device='local-task', backend='llvm-cpu', iree_compile_args=None, mode='cl-onnx-iree', torchtolinalg=True, stages=None, skip_stages=None, benchmark=False, load_inputs=False, groups='all', test_filter='migraphx', testsfile=None, tolerance=None, verbose=True, rundirectory='test-run', no_artifacts=False, cleanup='0', report=True, report_file='mi_10_10.md')
up to 5 (not included in total) crash during this stage
compilation
4
preprocessing
0
import_model
1
native_inference
2
construct_inputs
0
setup
0
Test Run Detail
Test was run with the following arguments:
Namespace(device='hip://1', backend='rocm', iree_compile_args=['iree-hip-target=gfx942'], mode='onnx-iree', torchtolinalg=False, stages=None, skip_stages=None, load_inputs=False, groups='all', test_filter='migraphx', tolerance=None, verbose=True, rundirectory='test-run', no_artifacts=False, report=True, report_file='9_3_migraphx.md')
1. Failing to use shape inference torch-mlir passes in torch-to-iree pipeline gives an all dynamic squeeze-dim op. 2. If using torch-lower-to-backend-contract to get the shape information, this crashes during inference with OOB memory access
crashes with "Memory access fault by GPU node-3 (Agent handle: 0x5595fe450840) on address 0x7f1811a56000. Reason: Unknown."
migraphx_pytorch-examples__wlang_gru
Numerics
migraphx_pytorch-examples__wlang_lstm
Numerics
migraphx_torchvision__densenet121i32
PASS
migraphx_torchvision__inceptioni1
PASS
migraphx_torchvision__inceptioni32
PASS
migraphx_torchvision__resnet50i1
PASS
migraphx_torchvision__resnet50i64
PASS
Note: GPU missing sd model (runs out of memory and kills the test). Probably happening during native inference, so it might need some looking into.
Performance data with iree-benchmark-module on GPU
Summary
Stage
Count
Total
30
PASS
13
Numerics
3
results-summary
0
postprocessing
0
benchmark
0
compiled_inference
2
native_inference
1
construct_inputs
0
compilation
8
preprocessing
0
import_model
3
setup
0
Test Run Detail
Test was run with the following arguments:
Namespace(device='local-task', backend='llvm-cpu', iree_compile_args=None, mode='cl-onnx-iree', torchtolinalg=False, stages=None, skip_stages=None, benchmark=True, load_inputs=False, groups='all', test_filter='migraphx', testsfile=None, tolerance=None, verbose=True, rundirectory='test-run', no_artifacts=False, cleanup='0', report=True, report_file='report.md')
This issue will be used to track compilation failures for migraphx models on CPU and GPU. Compile failures for each model should have a link to an issue with a smaller reproducer in the notes column.
Notes:
migraphx_ORT__bert_base_cased_1
fails on CPU but passes on GPU. Other adjacent models fail for similar reasons on both. Very odd.migraphx_sdxl__unet__model
,migraphx_ORT__bert_large_uncased_1
because they cause a crash (likely OOM)CPU Status Table
The Following report was generated with IREE compiler version iree-org/iree@caacf6c
Torch-mlir version llvm/torch-mlir@2665ed3
Passing Summary
TOTAL TESTS = 30
Fail Summary
TOTAL TESTS = 30
Test Run Detail
Test was run with the following arguments:
Namespace(device='local-task', backend='llvm-cpu', iree_compile_args=None, mode='cl-onnx-iree', torchtolinalg=True, stages=None, skip_stages=None, benchmark=False, load_inputs=False, groups='all', test_filter='migraphx', testsfile=None, tolerance=None, verbose=True, rundirectory='test-run', no_artifacts=False, cleanup='0', report=True, report_file='mi_10_10.md')
OLD STATUS (Will update and migrate issues to current table)
GPU Status Table
last generated with pip installed iree tools at version
Summary
Test Run Detail
Test was run with the following arguments:
Namespace(device='hip://1', backend='rocm', iree_compile_args=['iree-hip-target=gfx942'], mode='onnx-iree', torchtolinalg=False, stages=None, skip_stages=None, load_inputs=False, groups='all', test_filter='migraphx', tolerance=None, verbose=True, rundirectory='test-run', no_artifacts=False, report=True, report_file='9_3_migraphx.md')
Note: GPU missing sd model (runs out of memory and kills the test). Probably happening during native inference, so it might need some looking into.
Performance data with iree-benchmark-module on GPU
Summary
Test Run Detail
Test was run with the following arguments:
Namespace(device='local-task', backend='llvm-cpu', iree_compile_args=None, mode='cl-onnx-iree', torchtolinalg=False, stages=None, skip_stages=None, benchmark=True, load_inputs=False, groups='all', test_filter='migraphx', testsfile=None, tolerance=None, verbose=True, rundirectory='test-run', no_artifacts=False, cleanup='0', report=True, report_file='report.md')
The text was updated successfully, but these errors were encountered: