[Flaky Test] x-pack/metricbeat/module/autoops_es/cat_shards TestConvertToNodeIndexShardsWithCache

[mauri870]

Flaky Test

• Test Name: TestConvertToNodeIndexShardsWithCache
• Link: https://github.com/elastic/beats/blob/95216ad/x-pack/metricbeat/module/autoops_es/cat_shards/cache_test.go#L703
• Branch: elastic:updatecli_main_bump-elastic-stack-snapshot-main
• Artifact Link: https://buildkite.com/elastic/beats-xpack-metricbeat/builds/29529#019d1b59-d45b-4993-bf37-c783ddaa02bd/L643-L650
• Notes:

Stack Trace

=== Failed
=== FAIL: x-pack/metricbeat/module/autoops_es/cat_shards TestConvertToNodeIndexShardsWithCache (0.00s)
    cache_test.go:703:
        	Error Trace:	/opt/buildkite-agent/builds/bk-agent-prod-gcp-1774280914973550582/elastic/beats-xpack-metricbeat/x-pack/metricbeat/module/autoops_es/cat_shards/cache_test.go:703
        	Error:      	Not equal:
        	            	expected: int(10000)
        	            	actual  : int64(10001)
        	Test:       	TestConvertToNodeIndexShardsWithCache

[github-actions]

tl;dr: this flake is caused by a 1ms wall-clock race in the test path; TimestampDiff is asserted as exactly 10000, but the code computes it using a second time.Now().UnixMilli() call, which intermittently yields 10001.

Recommendation

Make TestConvertToNodeIndexShardsWithCache deterministic by controlling time in code under test (preferred), or relax the assertion to a small bounded range.

• Preferred: inject a nowMillis function in cache.go and stub it in tests.
• Minimal: replace strict equality at cache_test.go:703 with require.InDelta or a range check.

Findings

• x-pack/metricbeat/module/autoops_es/cat_shards/cache_test.go:22-27:
  • initCache() sets cache.NewTimestamp = time.Now().UnixMilli()
  • cache.PreviousTimestamp = cache.NewTimestamp - (previousSeconds * 1000)
• x-pack/metricbeat/module/autoops_es/cat_shards/cache.go:136:
  • convertToNodeIndexShards() overwrites cache.NewTimestamp = time.Now().UnixMilli()
• x-pack/metricbeat/module/autoops_es/cat_shards/cache.go:98-100:
  • TimestampDiff is computed as cache.NewTimestamp - cache.PreviousTimestamp
• x-pack/metricbeat/module/autoops_es/cat_shards/cache_test.go:703:
  • test requires exact 10000 (require.EqualValues(t, 10000, *myIndexNode1.TimestampDiff))

So if 1ms elapses between the two time.Now() calls, the observed value is 10001 and the test flakes.

Verification

I ran targeted repetitions locally:

go test ./x-pack/metricbeat/module/autoops_es/cat_shards -run TestConvertToNodeIndexShardsWithCache -count=200

• Sometimes passes.

go test ./x-pack/metricbeat/module/autoops_es/cat_shards -run TestConvertToNodeIndexShardsWithCache -count=1000

• Reproduced failures:

--- FAIL: TestConvertToNodeIndexShardsWithCache (0.00s)
    cache_test.go:703:
        Error: Not equal:
          expected: int(10000)
          actual  : int64(10001)

Detailed Action Plan

1. In x-pack/metricbeat/module/autoops_es/cat_shards/cache.go, introduce a package-level time source (example: var nowMillis = func() int64 { return time.Now().UnixMilli() }).
2. Replace direct time.Now().UnixMilli() usage in convertToNodeIndexShards() with nowMillis().
3. In x-pack/metricbeat/module/autoops_es/cat_shards/cache_test.go, stub nowMillis to fixed values for initCache() and convertToNodeIndexShards() call path so TimestampDiff is deterministic.
4. Keep the strict 10000 assertion once time is controlled; if not controlling time, change line 703 to tolerance-based assertion.
5. Re-run:
   • go test ./x-pack/metricbeat/module/autoops_es/cat_shards -run TestConvertToNodeIndexShardsWithCache -count=1000
   • go test ./x-pack/metricbeat/module/autoops_es/cat_shards

Related Items

Type	Link	Relevance
Issue	#49620	Current flaky test report
File	x-pack/metricbeat/module/autoops_es/cat_shards/cache_test.go:703	Failing exact-value assertion
File	x-pack/metricbeat/module/autoops_es/cat_shards/cache.go:136	Non-deterministic timestamp assignment
File	x-pack/metricbeat/module/autoops_es/cat_shards/cache.go:98-100	TimestampDiff computation
Commit	078dd4b862306815a7e38c6f03d384f663498641	Latest blame source for affected lines

---

What is this? | From workflow: Issue Triage

Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.

[mauri870]

/ai Since this is a problem only in tests, I'm leaning towards a less invasive solution. Using require.InDelta seems good enough to fix the flakyness, where we know the implementation is correct, but variants in wall clock prevent the test assertion from being accurate. Please test this approach, confirm that it stays stable with a large -count parameter then open a PR.