These methods are not in the standard library of the .NET correct?

[Xyncgas]

Seems a performant implementation should become part of the standard library

[Kinematics]

Been doing a little benchmarking of my own for other purposes, and came across this library as an alternative.

For a basic sum of ints (the benchmark does a sum of an array of 1,000,000 ints --- EDIT: Actually, 100,000 ints; just noticed a small typo in the setup), .NET 8 has massively improved LINQ performance of the Sum() function.

In .NET 6, _array.Sum() took 13x as long to complete as a basic for loop, while HPCsharp was roughly 3x as fast as the for loop. For both speed and complexity (a single function call instead of an explicit loop), HPCsharp was a massive improvement.

However, while HPCsharp had some small gains in moving from .NET 6 to .NET 8 (being about 13% faster for the most likely use), the vectorization work done in LINQ obliterated the horrid performance previously seen, and actually made it almost twice as fast as HPCsharp.

So, to a certain extent, this has become part of the standard library. Though that's incomplete, as LINQ still doesn't handle things like uints or shorts, and HPCsharp has tons of algorithms for other needs.

Here's some benchmarks I ran. Most descriptions should be obvious. EnumerableSum uses Enumerable.Sum(_array), while ArraySum does _array.Sum(). Essentially the same thing, just a different syntax. The Vector test was my own very simple implementation of a vector-based algorithm, before I saw the performance of LINQ in .NET 8. It seems to be basically the same as the LINQ rewrite.

The HPCsharp methods are using SumToLongSse, SumSseAndScalar() and SumToLongSsePar(). The parallel method is clearly not a good use case for this scenario.

BenchmarkDotNet v0.13.12, Windows 10 (10.0.19045.3930/22H2/2022Update)
AMD Ryzen 7 3700X, 1 CPU, 16 logical and 8 physical cores
.NET SDK 8.0.200-preview.23624.5
  [Host]   : .NET 6.0.26 (6.0.2623.60508), X64 RyuJIT AVX2
  .NET 6.0 : .NET 6.0.26 (6.0.2623.60508), X64 RyuJIT AVX2
  .NET 8.0 : .NET 8.0.1 (8.0.123.58001), X64 RyuJIT AVX2


| Method              | Job      | Mean       | Ratio | Gen0   | Gen1   | Allocated |
|-------------------- |--------- |-----------:|------:|-------:|-------:|----------:|-
| UsingForLoop        | .NET 6.0 |  31.427 us |  1.00 |      - |      - |         - |
| UsingForLoop        | .NET 8.0 |  33.962 us |  1.08 |      - |      - |         - |
|                     |          |            |       |        |        |           |
| UsingForeachLoop    | .NET 6.0 |  32.576 us |  1.00 |      - |      - |         - |
| UsingForeachLoop    | .NET 8.0 |  34.031 us |  1.04 |      - |      - |         - |
|                     |          |            |       |        |        |           |
| UsingArrayForEach   | .NET 6.0 | 169.861 us |  1.00 |      - |      - |      88 B |
| UsingArrayForEach   | .NET 8.0 | 167.607 us |  0.99 |      - |      - |      88 B |
|                     |          |            |       |        |        |           |
| UsingAggregate      | .NET 6.0 | 486.155 us |  1.00 |      - |      - |      32 B |
| UsingAggregate      | .NET 8.0 | 200.965 us |  0.41 |      - |      - |      32 B |
|                     |          |            |       |        |        |           |
| UsingEnumerableSum  | .NET 6.0 | 414.085 us |  1.00 |      - |      - |      32 B |
| UsingEnumerableSum  | .NET 8.0 |   6.156 us |  0.01 |      - |      - |         - |
|                     |          |            |       |        |        |           |
| UsingArraySum       | .NET 6.0 | 411.671 us |  1.00 |      - |      - |      32 B |
| UsingArraySum       | .NET 8.0 |   6.177 us |  0.02 |      - |      - |         - |
|                     |          |            |       |        |        |           |
| UsingVector         | .NET 6.0 |   9.097 us |  1.00 |      - |      - |      56 B |
| UsingVector         | .NET 8.0 |   6.282 us |  0.69 |      - |      - |      56 B |
|                     |          |            |       |        |        |           |
| UsingHPCSharp       | .NET 6.0 |  12.419 us |  1.00 |      - |      - |         - |
| UsingHPCSharp       | .NET 8.0 |  10.863 us |  0.87 |      - |      - |         - |
|                     |          |            |       |        |        |           |
| UsingHPCSharpScalar | .NET 6.0 |  16.417 us |  1.00 |      - |      - |         - |
| UsingHPCSharpScalar | .NET 8.0 |  12.417 us |  0.76 |      - |      - |         - |
|                     |          |            |       |        |        |           |
| UsingHPCSharpPar    | .NET 6.0 |  30.166 us |  1.00 | 1.3428 | 0.6714 |   11032 B |
| UsingHPCSharpPar    | .NET 8.0 |  29.529 us |  0.98 | 1.2817 | 1.2512 |   10548 B |

If your needs can be handled by LINQ in .NET 8, then definitely just use that. Hopefully HPCsharp can learn from the changes that happened in .NET 8 to further improve its performance in the space that LINQ doesn't cover.

[DragonSpit]

Benchmarks above are spot on and match the following ones on Dell laptop, for int[] of 100,000 elements:

BenchmarkDotNet v0.13.12, Windows 11 (10.0.22621.3007/22H2/2022Update/SunValley2)
12th Gen Intel Core i7-12700H, 1 CPU, 20 logical and 14 physical cores
.NET SDK 8.0.101
[Host] : .NET 8.0.1 (8.0.123.58001), X64 RyuJIT AVX2
DefaultJob : .NET 8.0.1 (8.0.123.58001), X64 RyuJIT AVX2

Method	Mean	Error	StdDev
StdSumRandom	72.648 us	0.4707 us	0.4403 us
StdSumConstant	6.577 us	0.0735 us	0.0687 us
HPCsharpSumRandom	29.147 us	0.1640 us	0.1454 us

These results are from an added BenchmarkDotNet VS2022 solution/project added into BenchmarkDotNetHPCsharp folder of the HPCsharp repository.

The fastest result is from the Standard C# Sum() with input of int[] and int result. However, this function is not safe and can cause arithmetic
overflow to occur. The benchmark is setup to initialize the input array with a constant value, to avoid arithmetic overflow. If an input array
filled with random integer values is used instead, the benchmark does not finish due to a thrown exception. Thus, when using the Standard C# Sum()
care must be used to make sure that input data is such as to not ever cause an arithmetic overflow exception.

StdSumRandom() is setup to use the Standard C# Sum(), casting each integer within the input int[] to long, before summing them up. This avoids the
possibility of arithmetic overflow. This version is safer, but is more than 10X slower.

HPCsharpSumRandom() implements an accelerated function that uses SIMD/SSE instructions on each of the multiple cores used, in such a way as to
avoid arithmetic overflow. It's implementation produces the same result as that of StdSumRandom() implementation. For 100K array HPCsharp provides
over 2X acceleration while providing numerical safety.

For larger arrays HPCsharp provides more acceleration:

For 1,000,000 int[] array size:

BenchmarkDotNet v0.13.12, Windows 11 (10.0.22621.3007/22H2/2022Update/SunValley2)
12th Gen Intel Core i7-12700H, 1 CPU, 20 logical and 14 physical cores
.NET SDK 8.0.101
[Host] : .NET 8.0.1 (8.0.123.58001), X64 RyuJIT AVX2
DefaultJob : .NET 8.0.1 (8.0.123.58001), X64 RyuJIT AVX2

Method	Mean	Error	StdDev
StdSumRandom	744.37 us	5.366 us	5.020 us
StdSumConstant	78.16 us	1.475 us	1.639 us
HPCsharpSumRandom	34.66 us	0.460 us	0.359 us

For 10,000,000 int[] array size:

BenchmarkDotNet v0.13.12, Windows 11 (10.0.22621.3007/22H2/2022Update/SunValley2)
12th Gen Intel Core i7-12700H, 1 CPU, 20 logical and 14 physical cores
.NET SDK 8.0.101
[Host] : .NET 8.0.1 (8.0.123.58001), X64 RyuJIT AVX2
DefaultJob : .NET 8.0.1 (8.0.123.58001), X64 RyuJIT AVX2

Method	Mean	Error	StdDev
StdSumRandom	8,419.2 us	168.02 us	200.01 us
StdSumConstant	2,137.1 us	42.05 us	72.53 us
HPCsharpSumRandom	384.4 us	3.82 us	3.19 us

These benchmarks do show that HPCsharp needs to be improved for smaller array sizes, and there is an issue with its Sum functions for smaller array sizes, where
"Too many cooks in the kitchen" are used even for small arrays. We will work on improving parallel performance for small-ish array sizes, as is done for Parallel
Sort functions.

[DragonSpit]

Agreed. It would be so useful for high performance implementations of standard algorithms, such as the ones in HPCsharp, to be part of the standard C# library. We would welcome contributions in this direction.

[DragonSpit]

Benchmark has been updated to run across 100K, 1M, 10M and 100M array sizes, to show how Standard C# and HPCsharp both struggle to accelerate performance using multi-core for smaller array sizes. HPCsharp has developed a method to not drop performance for small array sizes - implementing "Perform no worse" principle. This issue will get resolved for parallel sum in HPCsharp shortly.

BenchmarkDotNet v0.13.12, Windows 11 (10.0.22621.3007/22H2/2022Update/SunValley2)
12th Gen Intel Core i7-12700H, 1 CPU, 20 logical and 14 physical cores
.NET SDK 8.0.101
[Host] : .NET 8.0.1 (8.0.123.58001), X64 RyuJIT AVX2
DefaultJob : .NET 8.0.1 (8.0.123.58001), X64 RyuJIT AVX2

Method	N	Mean	Error	StdDev	Median
StdSumRandom	100000	72.703 us	0.4440 us	0.3936 us	72.756 us
StdParallelSumRandom	100000	101.309 us	2.0014 us	2.1414 us	100.812 us
StdSumConstant	100000	6.805 us	0.0718 us	0.0671 us	6.821 us
StdParallelSumConstant	100000	37.877 us	0.5398 us	0.5050 us	37.941 us
HPCsharpSumRandom	100000	8.439 us	0.0370 us	0.0328 us	8.429 us
HPCsharpParallelSumRandom	100000	29.126 us	0.2355 us	0.2088 us	29.136 us
StdSumRandom	1000000	681.376 us	4.6970 us	3.9222 us	680.998 us
StdParallelSumRandom	1000000	748.069 us	13.5083 us	22.9381 us	744.469 us
StdSumConstant	1000000	78.704 us	1.5081 us	1.6763 us	78.322 us
StdParallelSumConstant	1000000	229.782 us	4.4075 us	4.5261 us	228.601 us
HPCsharpSumRandom	1000000	88.502 us	1.0652 us	0.9442 us	88.605 us
HPCsharpParallelSumRandom	1000000	34.529 us	0.5266 us	0.4926 us	34.513 us
StdSumRandom	10000000	7,402.298 us	144.8146 us	193.3233 us	7,352.459 us
StdParallelSumRandom	10000000	4,646.317 us	91.7070 us	150.6772 us	4,677.511 us
StdSumConstant	10000000	2,147.984 us	42.5868 us	90.7559 us	2,148.144 us
StdParallelSumConstant	10000000	1,523.485 us	28.6321 us	32.9727 us	1,522.008 us
HPCsharpSumRandom	10000000	2,122.064 us	41.7157 us	62.4380 us	2,127.900 us
HPCsharpParallelSumRandom	10000000	390.209 us	6.3586 us	5.6367 us	389.585 us
StdSumRandom	100000000	74,524.157 us	1,324.8398 us	2,354.9014 us	74,433.533 us
StdParallelSumRandom	100000000	45,052.869 us	2,119.7951 us	6,250.2637 us	43,402.839 us
StdSumConstant	100000000	24,271.598 us	477.9559 us	897.7171 us	24,442.989 us
StdParallelSumConstant	100000000	18,740.032 us	372.7475 us	744.4170 us	18,636.281 us
HPCsharpSumRandom	100000000	24,400.946 us	454.3391 us	446.2218 us	24,395.294 us
HPCsharpParallelSumRandom	100000000	6,451.421 us	128.3394 us	175.6724 us	6,415.620 us