-
Notifications
You must be signed in to change notification settings - Fork 1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add network docs #35
Add network docs #35
Conversation
docs/writing_benchmarks.rst
Outdated
Writing a network tracking benchmark | ||
------------------------------------ | ||
|
||
Functions that require network access ---such as reading a file from S3--- are often a black box, with functions in other libraries (e.g., `h5py`, `fsspec` etc.) managing the access to the remote resources. The runtime performance of such functions is often inherently driven by how these functions utilize the network to access the resources. It is, hence, important that we can profile the network traffic that is being generated to better understand, e.g., the amount of data that is being downloaded and uploaded, the number of requests that are being sent/received, and others. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Functions that require network access ---such as reading a file from S3--- are often a black box, with functions in other libraries (e.g., `h5py`, `fsspec` etc.) managing the access to the remote resources. The runtime performance of such functions is often inherently driven by how these functions utilize the network to access the resources. It is, hence, important that we can profile the network traffic that is being generated to better understand, e.g., the amount of data that is being downloaded and uploaded, the number of requests that are being sent/received, and others. | |
Functions that require network access ---such as reading a file from S3--- are often a black box, with functions in other libraries (e.g., `h5py`, `fsspec` etc.) managing the access to the remote resources. The runtime performance of such functions is often inherently driven by how these functions utilize the network to access the resources. It is, hence, important that we can profile the network traffic that is being generated to better understand the amount of data that is being downloaded and uploaded, the number of requests that are being sent/received, and so on. |
Some minor style suggestions based on rendering on https://nwb-benchmarks--35.org.readthedocs.build/en/35/writing_benchmarks.html#network-tracking-benchmarks As a reminder, should you choose to accept the suggestions, you can batch them from the 'files changes' view on GitHub by selecting 'add suggestion to batch' and then 'commit X suggestions' to result in only a single commit; or if you prefer to stand by original style, just go ahead and merge |
Co-authored-by: Cody Baker <[email protected]>
Co-authored-by: Cody Baker <[email protected]>
Co-authored-by: Cody Baker <[email protected]>
Co-authored-by: Cody Baker <[email protected]>
Co-authored-by: Cody Baker <[email protected]>
Co-authored-by: Cody Baker <[email protected]>
Co-authored-by: Cody Baker <[email protected]>
Fix #24
NetworkTracker
class to track the total time directly, rather than having thenetwork_activity_tracker
track the time separately. This change addresses the following issues: a)NetworkTracker
was missing the total time so results are not the same compared tonetwork_activity_tracker
and 2)NetworkTracker.asv_network_statistics
was not being updated in thenetwork_activity_tracker
so the timing result was not being recorded.NetworkTracker
andnetwork_activity_tracker
to allow the user to optionally set the process ID to track. This will be useful if/when we need to run code we want to profile in a separate process (e.g., when running in node.js)