Skip to content

fix(tests): filter PostHog telemetry from fetch mocks#3353

Closed
la14-1 wants to merge 1 commit intomainfrom
fix-flaky-tests
Closed

fix(tests): filter PostHog telemetry from fetch mocks#3353
la14-1 wants to merge 1 commit intomainfrom
fix-flaky-tests

Conversation

@la14-1
Copy link
Copy Markdown
Member

@la14-1 la14-1 commented Apr 24, 2026

Why: Two tests fail consistently in the full suite (0% pass rate) due to PostHog telemetry fetch calls leaking into globalThis.fetch mocks, corrupting callCount-based response routing.

Summary

  • hetzner-cov.test.ts: "cleans up orphaned primary IPs" test fails because PostHog calls shift mock response order (HTTP 403 returned at wrong time)
  • digitalocean-token.test.ts: "attempts OAuth recovery on 401" test gets callCount=4 instead of 2 (2 extra PostHog calls)
  • Root cause: telemetry.test.ts enables the PostHog singleton which persists across test files in the same bun process
  • Fix: filter non-test URLs in affected fetch mocks so telemetry calls get a benign response without incrementing callCount

Test plan

  • Full suite: 2108 pass, 0 fail (was 2106 pass, 2 fail)
  • Both affected test files pass in isolation
  • Biome lint clean

-- spawn-refactor/test-engineer

@la14-1
Copy link
Copy Markdown
Member Author

la14-1 commented Apr 24, 2026

Note: This PR overlaps with #3341 — both fix the same 2 flaky tests (hetzner-cov.test.ts and digitalocean-token.test.ts) by addressing fetch mock pollution from concurrent test files. #3341 also includes cmdrun-happy-path.test.ts fixes and uses URL-pattern-aware mock routing (more robust). Recommend merging #3341 first; this PR may then be superseded.

-- refactor/pr-maintainer

@la14-1
Copy link
Copy Markdown
Member Author

la14-1 commented Apr 28, 2026

Supersession note: This PR (#3353) is likely superseded by #3358 (root cause fix) + #3365 (more robust defense in depth). If both of those land, this PR can be closed.

Leaving open for now pending human decision on merge order.

-- spawn-refactor/pr-maintainer

Two tests (hetzner-cov and digitalocean-token) failed consistently in the
full suite because telemetry.test.ts enables PostHog telemetry via the
shared module singleton, and those async fetch calls leak into other test
files' globalThis.fetch mocks, incrementing callCount and shifting mock
response order.

Fix: filter non-test URLs in affected mocks so telemetry calls get a
benign response without affecting callCount.

Agent: test-engineer
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
@la14-1
Copy link
Copy Markdown
Member Author

la14-1 commented May 1, 2026

Closing as superseded by #3376, which provides a more comprehensive fix for the same telemetry singleton leak (runtime env-var guard in sendEvent() + SPAWN_TELEMETRY=0 in test preload).

-- refactor/pr-maintainer

@la14-1
Copy link
Copy Markdown
Member Author

la14-1 commented May 1, 2026

Superseded by #3376.

@la14-1 la14-1 closed this May 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants