[Requestor enchancement] Make it possible to rent only x out of y cores on a provider and leave the remaining for the rest of the network

[cryptobench]

Let's say you need 2 cores for a specific service and you have a market strategy that selects the absolute cheapest provider on the network within these requirements. It now turns out that a machine with 10 cores is the cheapest one on the network, but you only need 2. Because it's the cheapest then naturally you select that provider, but now the remaining 8 cores are just stuck idling.

There's a lot of issues with this because:

1. The network is not utilised in a proper efficient way
2. The provider now have 8 cores that are not actively billed for because no cpu-time is spent by the requestor. This is effectively decreasing the providers earnings, because he could just as-well have someone pay to use the remaining 8 cores.
3. If I want to create a cloud and rent out instances to my customers as a middleman service, then I could very quickly rent out a very high capacity of the network. How this would work is that if a customer of mine wants to rent 2 cores for some service, then I go ahead and create a demand on the network for 2 cores, but all providers with 2 cores are unavailable because they are already computing something and now I have to pick from the providers with 4 cores available. Even though I have rented 4 cores, then I still only want to give access to 2 cores to my customer, so I will just let the 2 others sit idle and do nothing.
   I could of course provision 2 customers wanting 2 cores to the same provider, but it will be tough for me to limit them from actually only using 2 cores. I can of course investigate doing this, but again let's make Golem as easy as possible for our requestors and provide them the tools out of the box so we don't have to create all these hacky solutions.
4. If we look at how clouds currently provision machines to the customers, then they have 1 big machine with let's say 64 cores, and when someone needs 4 cores then it provisions a VM with 4 cores and now we have 60 cores left to rent out to other customers on this specific machine. Golem should have the same ideology about its providers on the network.
5. I am a requestor and I don't want to pay for example THREE providers when it could be run on a single provider for atleast 1/3rd of the cost. Money is the most important factor of gaining requestors on the network.

TL:DR - I believe the vm ExeUnit should be able to spawn with a specific amount of cores requested by the requestor and not all cores that the provider is advertising on the network. This also means that a provider should be able to create multiple agreements and advertise the cores it has left automatically to the network.

[RoyKas]

The provider can already specify multiple hardware profiles and pricing presets. However currently only one hardware profile can be activated, and there is no system in place to link presets to profiles.

[cryptobench]

That's correct, but this is a manual process. I would like the SDK + Yagna to be updated, so that you specify you want to rent x cores and it doesn't really matter from who. If the provider has 10 cores and you rent 8, then there's two left for the rest of the network and Yagna should be able to construct a hardware profile according to that and advertise that.

[cryptobench]

So similarly to Kubernetes where you specify resource requirements. The kubernetes scheduler will then look for a node with that capacity available and schedule the workload onto that node. The remaining capacity of the node are still then available to the rest of the cluster.

[RoyKas]

But still, it would be even better if pricing could be adjusted for the different hardware profiles. I would like to charge higher pricing for 128 core VM with loads of RAM than for 2 core VM with some leftover memory...

[cryptobench]

That's a valid point. Do you have any vision on how this should work in practice? The first thing that strikes my mind is defining some math that "scales" the pricing up by the amount of resources requested. Next thought is how would this work on out a stats page? Would the mathematical formular be advertised on the network and then one would for example have a slider to adjust the core count and memory they are interested in and then it'll display the pricing accordingly?

Maybe it could be a simpler approach, but I would imagine it's not nice to work with software that requires a lot of manual work, so i'm immediately thinking in an automated approach.

Any thoughts?

[RoyKas]

Preventing too excessive changes to the pricing set-up, it should be possible to use the existing pricing presets. Maybe extending it a bit by interpolation for calculating intermediate core counts. One wouldn't like to first need to set-up 128 presets on a 128 core system... But define pricing for 1 and 128 cores and let ya-provider calculate all 128 presets. All presets would need to be advertised on the network to allow requestors to select the most optimal one. Drawback would be the massive number of offers on the network...

[innocentius]

Is it possible currently to set the max_simutainous_agreement more than 1 right now? Maybe we can have a way to divide our provider into chunks for the network.

[cryptobench]

It's possible to configure but the software itself hasn't been created with having more than 1 in mind, so complications might occur. From what i've noticed on our Discord, then I haven't seen any had success with increasing the value - thus i'm not sure if it works properly or not.

Secondly, the software doesn't deduct those already rented out resources, so the second person might not receive the hardware that his demand specified on the network which effectively means they've gotten scammed.

[innocentius]

For now the only way to control these who only utilize a small portion of the provider's resource is to increase the env_per_hour price or starting price.

But in the current situation where supply >>> demand, things won't do well.

[rafidsanjaya]

I believe it will be good if provider who has a lot of cores can advertise only to requestors who need all of his cores or most of his cores, for renting.