You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The Kubernetes pods used in production each have 16 CPUs (16 sockets with 1 core per socket and 1 thread per core). Using the Python multiprocessing package could potentially speed up the parsing of large submissions by parallelising the loop over data tables:
It looks like an attempt to use multiprocessing.Pool was started in 980dd23 but later removed in 4b1ad68. If successful, the use of multiprocessing could be extended to other parts of the converter code.
The text was updated successfully, but these errors were encountered:
The Kubernetes pods used in production each have 16 CPUs (16 sockets with 1 core per socket and 1 thread per core). Using the Python multiprocessing package could potentially speed up the parsing of large submissions by parallelising the loop over data tables:
hepdata-converter/hepdata_converter/parsers/yaml_parser.py
Line 83 in 3f0330d
It looks like an attempt to use
multiprocessing.Pool
was started in 980dd23 but later removed in 4b1ad68. If successful, the use ofmultiprocessing
could be extended to other parts of the converter code.The text was updated successfully, but these errors were encountered: