Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pylib: add run_simulator runner #173

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

nmeum
Copy link

@nmeum nmeum commented Jan 17, 2023

This adds a Python module which is suitable for running the benchmarks within a simulator that doesn't not support a GDB stub and allows for a rough performance estimation of said simulator. The module is based on the existing run_native.py module and has the same deficiencies (i.e. inaccurate measurements through the use of the time(1) command). An alternative approach would be modifying the run_native.py module in away that a wrapper could be passed via an additional command-line flag (e.g. ./benchmark_speed.py --target-module run_native --wrapper 'my-fancy-simulator ...') the passed string would then also be inserted into the time(1) command as proposed here.

Usage example of the proposed run_simulator module:

$ ./benchmark_speed.py --target-module run_simulator --simulator 'my-simulator <additional args for the simulator>'

Fixes #106

Otherwise, this function does not handle multiple line output correctly.
For example, if a simulator writes text to standard error for some
reason.
@jeremybennett
Copy link
Collaborator

Hi @nmeum

Thanks for the PR. A useful addition. Could you update the documentation in doc/README.md. I suggest a new section on how to run using your run_simulator module instead of the approach detailed would be sufficient.

Thanks for the contribution.

@nmeum
Copy link
Author

nmeum commented Mar 10, 2023

Hi, good suggestion! I can also add documentation for run_simulator. Do you have any hints where specifically in doc/README.md I should add a section? Essentially, run_simulator is no different then the existing run scripts. Unfortunately, I wasn't able to find any detailed documentation for the existing run scripts in doc/README.md and hence I am unsure where to best add documentation for run_simulator.

I also just pushed an additional improvement to also use multiline regex search in decode_results when looking for the timing information. This is analog to 5d05d18 and necessary when the simulator writes data to stderr itself for some reason.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Queries on Embench benchmarks- How to config my simulator(target)
2 participants