Supporting parallel/multi-machine processing

[cayolblake]

@madMAx43v3r

Is it feasible to utilize multiple machines for plotting using available multiple processors per machine? Something like old school Beowulf clusters or alikes.

[cayolblake]

@madMAx43v3r

Is it feasible to utilize multiple machines for plotting using available multiple processors per machine? Something like old school Beowulf clusters or alikes.

@madMAx43v3r am I knocking on a forbidden door here!?

[hajes]

if you have InfiBand between NUMA nodes, it may be interesting. I doubt there will be any benefits on slow ethernet connectivity. My Ryzen 3950X uses 1500MB/s on my NVMe. How do you want to move such data among many machines?

Old DDR3-12800 can do 18GB/s, how do you want to move those amounts of data among many machines?

[madMAx43v3r]

It would only make sense if having 256 GB of RAM on a single machine is not possible. Otherwise you may as well plot individually on the machines.

[smilstea]

@cayolblake I have started investigating plotting K34 for my next 1PB and you can easily get the required tmp2 ramdisk size in something like a dell r720, I did that and am adding some PCIe v3.0 NVME drives in raid0 for tmp1. Doing all ram might be slightly faster if you ran bladebit but to reach 1.5TB of RAM or more needed you would have to spend a lot more money (think like $18k vs $5k) for a plotting machine.
If you absolutely wanted to cluster you could potentially. I believe based on what I have read that something like gluster/glusterfs would work to make a ramdisk cross cluster, but you still have the high speed RDMA to worry about, which can be done with a 25 or 100GBE card which adds a lot of cost. I should also note that gluster dropped support for RDMA recently. It is far cheaper to just use an NVME for tmp1 and ramdisk for tmp2, your plotting times won't suffer that much.