[NOTE: these are simply introductory tests and should not be considered scientifically accurate comparisons of the various platforms]

The second run of the tests used the defaults for the parameters as provided in the original sample with the exception of the “runs” parameter which was increased from 1,000,000 to 10,000,000. The test bed used is detailed here and the results of the first run are here. The net-net is that the calculation is run 100 times and each time contains a loop that runs 10,000,000 times. Each run of the calculation can be run independently of the others as the aggregation/summation is handled in the spreadsheet once the calculations have finished. To provide some protection from anomalies, I ran each test against each platform 10 times with the first run being “cold”.

Results:

The time values on the Y axis are seconds to complete the execution. In this run, a few things jump out:

• As in the first test, the 2-node HPC cluster killed the other two. Again this isn’t surprising, although I’d like to think that as the compute to communication ratio grows, the delta b/t the HPC cluster and Azure will shrink.
• In contrast to the first run, in this run Azure beat the local run by an average of nearly 200 seconds/run. This is consistent with the theory that the communication overhead in the Azure solution is constant per request regardless of the complexity of compute resulting with the communication overhead becoming marginalized as the compute to communicate ratio increases.

I’m currently performing tests that take the runs parameter to 100,000,000 which should separate the three platforms further and it is expected that unlike the first two runs, Azure will be closer to the HPC results than to the local compute results (although the HPC cluster is still expected to outperform Azure).