Best Practices for storage network configuration

[rrajendran17]

Related to harvester/harvester#7222

[github-actions]

Name	Link
🔨 Latest commit	af3d934
😎 Deploy Preview	https://67a262a1eb580500af33aee7--harvester-preview.netlify.app

[w13915984028]

LGTM, thanks.

If comment on harvester/harvester#7422 is accepted, then please update the last section accordingly.

since validation added for this in https://github.com/harvester/harvester/pull/7422/files, removing the last line regarding the mgmt network.

[rrajendran17]

I also observed that harvester latest docs still mentions about "instance-manager-r" and "instance-manager-e".
Created this harvester/harvester#7446 to take care of the changes.

[starbops]

LGTM, thank you.

[jillian-maroket]

Review done. @ihcsim is correct in that the list items must be worded as best practices. We must state the recommended action before mentioning other options.

[jillian-maroket]

Suggested change

	- When allocating IP range for storage network using this https://docs.harvesterhci.io/v1.4/advanced/storagenetwork/#configuration-example
	consider allocating sufficient IP addresses keeping in mind the future needs of the cluster.
	When new nodes are added to the cluster or more disks added to the node after storage network configuration and if required number of IPs becomes
	greater than allocated, then it results in Longhorn pods (`instance-manager`, and `backing-image-manager`) not in running state.
	Storage network has to be reconfigured with sufficient IP range to solve this.
	- When configuring an [IP range](https://docs.harvesterhci.io/v1.4/advanced/storagenetwork/#configuration-example) for the storage network, ensure that the allocated IP addresses can service the future needs of the cluster. This is important because Longhorn pods (`instance-manager` and `backing-image-manager`) stop running when new nodes are added to the cluster or more disks are added to a node after the storage network is configured, and when the required number of IPs exceeds the allocation. Resolving the issue involves reconfiguring the storage network with the correct IP range.

[rrajendran17]

Thanks @jillian-maroket. I have updated the PR as suggested.
"when the required number of IPs exceeds the allocation" - I changed allocation to "allocated IPs" to be more clearer.

[jillian-maroket]

Suggested change

	- Users are allowed to configure storage networks using mgmt cluster, but it is always recommended to configure storage network on a non mgmt cluster to
	ensure complete separation of the Longhorn replication traffic from the pod network traffic.
	- Configuring the storage network on the managment (`mgmt`) network will have a negative impact on the network performance, due to resource and
	bandwidth contention as same NIC is shared by both. Use this option only if you could efficiently segregate the two types of traffic or your cluster has
	some NIC constraints.
	- Configure the storage network on a non-`mgmt` cluster network to ensure complete separation of the Longhorn replication traffic from the Kubernetes control plane traffic. Using `mgmt` is possible but not recommended because of the negative impact (resource and bandwidth contention) on the control plane network performance. Use `mgmt` only if your cluster has NIC-related constraints and if you can completely segregate the traffic.

[jillian-maroket]

The second and third items are related so I combined them. In addition, the third point was originally phrased as a limitation rather than a recommendation.

I use mgmt instead of "management network" to refer to the built-in cluster network. This wording decision was based on how @starbops explained the different networks.

[rrajendran17]

Thanks @jillian-maroket for the suggestions.I have incorporated as suggested.

[ihcsim]

LGTM - just one comment about the href.

[ihcsim]

I think we should use relative href like:

docs/docs/networking/harvester-network.md

Line 24 in 03a2ad0

Harvester also introduced storage networking to separate the storage traffic from other cluster-wide workloads. Please refer to [the storage network document](../advanced/storagenetwork.md) for more details.

Otherwise, it will always be pinned to the 1.4 version.