You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
It seems like it would be better to save the complete value (or truncate to a larger number of digits) to a file, and then truncation to three digits can happen downstream in the presentation layer.
The text was updated successfully, but these errors were encountered:
With the way the code in groupvg2.py is written in the noel branch, I have some concerns that if in the future, we change our method of defining signal in such a way that the signal values are much smaller (say, on the order of 0.01), the code as written (which is rounding signal values to two decimal places) will not work, or will have serious truncation errors. Now maybe that is unlikely, but the fact that the code would fail to work simply because of a hundredfold change in the scale of the signal, suggests we should make the code more robust. For that reason, I think it may make sense to eliminate rounding in run_vg2 up to the point where to_excel is called (where we can control the floating point representation using the float_format argument).
Of course, if there is a logical reason why rounding is being done internally, I am open to hearing it.
Here in the code, the CV values are being truncated at three digits after the decimal point, right after CV is calculated:
vgrampy/groupvg2.py
Line 114 in 3ee9faf
It seems like it would be better to save the complete value (or truncate to a larger number of digits) to a file, and then truncation to three digits can happen downstream in the presentation layer.
The text was updated successfully, but these errors were encountered: