ORTModelUnet config file not found after initial run

[willkara]

Describe the bug
After doing a completely fresh install of Olive and the dependencies, I am trying to run the samples in Olive-main/examples/directml/stable_diffusion_xl using this command: python stable_diffusion_xl.py --model_id=stabilityai/stable-diffusion-xl-refiner-1.0 --optimize

It succesfully runs through some of the training/eval and seems to error out when creating the ONNX pipeline.

To Reproduce

1. Fresh install
2. python stable_diffusion_xl.py --model_id=stabilityai/stable-diffusion-xl-refiner-1.0 --optimize

Expected behavior
Optimization pipelines runs succesfully

Olive config
Vanilla install

Olive logs

...
...
[2024-12-16 11:26:47,590] [INFO] [cache.py:193:load_model] Loading model 08363e27 from cache.
[2024-12-16 11:26:47,998] [INFO] [engine.py:295:run] Saved output model to C:\Users\wkara\PycharmProjects\StableAyymd\Olive-main\examples\directml\stable_diffusion_xl\footprints\text_encoder_2
Unoptimized Model : C:\Users\wkara\PycharmProjects\StableAyymd\Olive-main\examples\directml\stable_diffusion_xl\cache\default_workflow\runs\1d1c4ae8\models\model.onnx
Optimized Model   : C:\Users\wkara\PycharmProjects\StableAyymd\Olive-main\examples\directml\stable_diffusion_xl\cache\default_workflow\runs\08363e27\models\model.onnx

Creating ONNX pipeline...
Traceback (most recent call last):
  File "C:\Users\wkara\PycharmProjects\StableAyymd\Olive-main\examples\directml\stable_diffusion_xl\stable_diffusion_xl.py", line 631, in <module>
    main()
  File "C:\Users\wkara\PycharmProjects\StableAyymd\Olive-main\examples\directml\stable_diffusion_xl\stable_diffusion_xl.py", line 597, in main
    optimize(
  File "C:\Users\wkara\PycharmProjects\StableAyymd\Olive-main\examples\directml\stable_diffusion_xl\stable_diffusion_xl.py", line 420, in optimize
    onnx_pipeline = ORTStableDiffusionXLImg2ImgPipeline(
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\wkara\PycharmProjects\StableAyymd\.venv\Lib\site-packages\optimum\onnxruntime\modeling_diffusion.py", line 114, in __init__
    self.unet = ORTModelUnet(unet_session, self)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\wkara\PycharmProjects\StableAyymd\.venv\Lib\site-packages\optimum\onnxruntime\modeling_diffusion.py", line 536, in __init__
    super().__init__(*args, **kwargs)
  File "C:\Users\wkara\PycharmProjects\StableAyymd\.venv\Lib\site-packages\optimum\onnxruntime\modeling_diffusion.py", line 452, in __init__
    raise ValueError(f"Configuration file for {self.__class__.__name__} not found at {config_file_path}")
ValueError: Configuration file for ORTModelUnet not found at C:\Users\wkara\PycharmProjects\StableAyymd\Olive-main\examples\directml\stable_diffusion_xl\cache\default_workflow\runs\c5dd1953\models\config.json

Other information

• OS: Windows 11 - Python 3.12
• Olive version:main - Commit: https://github.com/microsoft/Olive/commit/bb51b0aeff79a3e3fc9872fbe9b70871b5c0301b
• ONNXRuntime package and version: Did a python setup.py install from main and also pip install -r requirements.txt in the stable_diffusion_xl` directory
• Transformers package version: Please see above

Additional context
Seems to be running fine through everything until it tries to reference the missing config file.

[willkara]

Just tried another on a fresh install and while it seems like it made it further, still running into the same issue.

[2024-12-20 22:55:14,226] [INFO] [engine.py:295:run] Saved output model to C:\Users\wkara\OneDrive\Development\Projects\Olive\examples\directml\stable_diffusion_xl\footprints\text_encoder_2
Unoptimized Model : C:\Users\wkara\OneDrive\Development\Projects\Olive\examples\directml\stable_diffusion_xl\cache\default_workflow\runs\f44775b0\models\model.onnx    
Optimized Model   : C:\Users\wkara\OneDrive\Development\Projects\Olive\examples\directml\stable_diffusion_xl\cache\default_workflow\runs\c7690e9f\models\model.onnx    

Creating ONNX pipeline...
Traceback (most recent call last):
  File "C:\Users\wkara\OneDrive\Development\Projects\Olive\examples\directml\stable_diffusion_xl\stable_diffusion_xl.py", line 631, in <module>
    main()
  File "C:\Users\wkara\OneDrive\Development\Projects\Olive\examples\directml\stable_diffusion_xl\stable_diffusion_xl.py", line 597, in main
    optimize(
  File "C:\Users\wkara\OneDrive\Development\Projects\Olive\examples\directml\stable_diffusion_xl\stable_diffusion_xl.py", line 420, in optimize
    onnx_pipeline = ORTStableDiffusionXLImg2ImgPipeline(
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\wkara\OneDrive\Development\Projects\Olive\.venv\Lib\site-packages\optimum\onnxruntime\modeling_diffusion.py", line 114, in __init__
    self.unet = ORTModelUnet(unet_session, self)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\wkara\OneDrive\Development\Projects\Olive\.venv\Lib\site-packages\optimum\onnxruntime\modeling_diffusion.py", line 536, in __init__
    super().__init__(*args, **kwargs)
  File "C:\Users\wkara\OneDrive\Development\Projects\Olive\.venv\Lib\site-packages\optimum\onnxruntime\modeling_diffusion.py", line 452, in __init__
    raise ValueError(f"Configuration file for {self.__class__.__name__} not found at {config_file_path}")
ValueError: Configuration file for ORTModelUnet not found at C:\Users\wkara\OneDrive\Development\Projects\Olive\examples\directml\stable_diffusion_xl\cache\default_workflow\runs\85c77687\models\config.json