Randomize per-thread recording order to avoid concurrency hotspots

[jack-berg]

When discussing #8527 in the last java SIG, @trask brought up the point that perhaps the apparent 37% reduction in performance for cardinality=128,temporality=cumulative,instrument=COUNTER_SUM case while the same case with cardinality=1 case improves is not expected, and might be caused test harness itself.

Indeed this turned out to be the case. The multi-thread cases all record to the same set of series in the same order. To mitigate, this PR creates a randomized record order for each distinct series. Recordings will occasionally collide and that's expected and desired, but they won't always collide as a structural side affect.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 90.96%. Comparing base (9b57914) to head (612c7c3).
⚠️ Report is 4 commits behind head on main.

Additional details and impacted files

@@             Coverage Diff              @@
##               main    #8550      +/-   ##
============================================
- Coverage     90.96%   90.96%   -0.01%     
+ Complexity    10206    10205       -1     
============================================
  Files          1013     1013              
  Lines         27166    27166              
  Branches       3182     3182              
============================================
- Hits          24712    24711       -1     
  Misses         1730     1730              
- Partials        724      725       +1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.