Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
106 changes: 53 additions & 53 deletions benchmark/REPORT.md
Original file line number Diff line number Diff line change
@@ -1,96 +1,97 @@
# React Testing Framework Benchmark Report

> Generated: Wed, 01 Apr 2026 12:41:50 GMT
> Generated: Sun, 05 Apr 2026 19:06:36 GMT

## Environment

| Property | Value |
| ------------- | ------------- |
| Node.js | v22.5.1 |
| Platform | darwin 25.4.0 |
| CPU | Apple M3 Pro |
| CPU Cores | 12 |
| Total RAM | 18.0 GB |
| Runs/scenario | 7 (trim ±1) |
| Property | Value |
|---|---|
| Node.js | v22.5.1 |
| Platform | darwin 25.4.0 |
| CPU | Apple M3 Pro |
| CPU Cores | 12 |
| Total RAM | 18.0 GB |
| Runs/scenario | 7 (trim ±1) |

## Scenarios

Each scenario runs the **same 9 React tests** across 5 test files:

| Test File | Tests |
| ---------------------- | ------------------------------------------ |
| 'counter.test.jsx' | 1 — stateful counter, event interaction |
| 'hooks.test.jsx' | 2 — custom hook harness + `renderHook` |
| 'lifecycle.test.jsx' | 2 — `rerender`, `unmount` + effect cleanup |
| 'context.test.jsx' | 1 — `createContext` + wrapper injection |
| 'concurrency.test.jsx' | 2 — React 19 `use()` + `useTransition` |
| Test File | Tests |
|---|---|
| 'counter.test.jsx' | 1 — stateful counter, event interaction |
| 'hooks.test.jsx' | 2 — custom hook harness + `renderHook` |
| 'lifecycle.test.jsx' | 2 — `rerender`, `unmount` + effect cleanup |
| 'context.test.jsx' | 1 — `createContext` + wrapper injection |
| 'concurrency.test.jsx' | 2 — React 19 `use()` + `useTransition` |

### Frameworks under test

| Combination | DOM layer | Assertion style |
| --------------------------------- | ------------------------------ | -------------------- |
| poku + @pokujs/react | happy-dom | `assert.strictEqual` |
| poku + @pokujs/react | jsdom | `assert.strictEqual` |
| jest 29 + @testing-library/react | jsdom (jest-environment-jsdom) | `expect().toBe()` |
| vitest 3 + @testing-library/react | jsdom | `expect().toBe()` |
| vitest 3 + @testing-library/react | happy-dom | `expect().toBe()` |
| Combination | DOM layer | Assertion style |
|---|---|---|
| poku + @pokujs/react | happy-dom | `assert.strictEqual` |
| poku + @pokujs/react | jsdom | `assert.strictEqual` |
| jest 29 + @testing-library/react | jsdom (jest-environment-jsdom) | `expect().toBe()` |
| vitest 3 + @testing-library/react | jsdom | `expect().toBe()` |
| vitest 3 + @testing-library/react | happy-dom | `expect().toBe()` |

## Results

| Scenario | Mean | Min | Max | Stdev | Peak RSS | vs poku+happy-dom |
| ------------------ | ------ | ------ | ------ | ------ | -------- | ----------------- |
| poku + happy-dom | 0.560s | 0.515s | 0.600s | 0.033s | 154.3 MB | _(baseline)_ |
| poku + jsdom | 0.444s | 0.429s | 0.451s | 0.008s | 157.1 MB | -21% |
| jest + jsdom | 1.040s | 0.975s | 1.135s | 0.056s | 203.4 MB | +86% |
| vitest + jsdom | 1.193s | 1.129s | 1.269s | 0.057s | 152.3 MB | +113% |
| vitest + happy-dom | 1.041s | 0.990s | 1.126s | 0.047s | 117.1 MB | +86% |

> **Wall-clock time** is measured with `performance.now()` around the child-process spawn.
|--------------------|--------|--------|--------|--------|----------|-------------------|
| poku + happy-dom | 0.115s | 0.112s | 0.109s | 0.004s | 127.4 MB | *(baseline)* |
| poku + jsdom | 0.294s | 0.280s | 0.299s | 0.007s | 172.1 MB | +157% |
| jest + jsdom | 0.843s | 0.768s | 0.879s | 0.041s | 198.8 MB | +636% |
| vitest + jsdom | 0.968s | 0.951s | 0.986s | 0.011s | 147.8 MB | +745% |
| vitest + happy-dom | 0.845s | 0.825s | 0.856s | 0.013s | 115.2 MB | +637% |

> **Poku elapsed time** uses Poku's reported suite `Duration` (ANSI-stripped parse) to avoid teardown-skew artifacts.
> **Jest/Vitest elapsed time** is measured with `performance.now()` around the child-process spawn.
> **Peak RSS** is captured via `/usr/bin/time -l` on macOS (bytes → MB).
> The baseline for relative comparisons is **poku + happy-dom**.

## Analysis

### Overall ranking (mean wall-clock time)

1. **poku + jsdom** — 0.444s
2. **poku + happy-dom** — 0.560s
3. **jest + jsdom** — 1.040s
4. **vitest + happy-dom** — 1.041s
5. **vitest + jsdom** — 1.193s
1. **poku + happy-dom** — 0.115s
2. **poku + jsdom** — 0.294s
3. **jest + jsdom** — 0.843s
4. **vitest + happy-dom** — 0.845s
5. **vitest + jsdom** — 0.968s

### Speed comparison

- poku+happy-dom vs jest+jsdom: jest is **86% slower**
- poku+happy-dom vs vitest+jsdom: vitest is **113% slower**
- poku+happy-dom vs jest+jsdom: jest is **636% slower**
- poku+happy-dom vs vitest+jsdom: vitest is **745% slower**
- jest+jsdom vs vitest+jsdom: vitest is **15% slower** than jest

### DOM adapter impact

- **poku**: happy-dom vs jsdom — jsdom is **-21% faster**
- **poku**: happy-dom vs jsdom — jsdom is **157% slower**
- **vitest**: happy-dom vs jsdom — jsdom is **15% slower**

### Memory footprint

- **vitest + happy-dom**: 117.1 MB peak RSS
- **vitest + jsdom**: 152.3 MB peak RSS
- **poku + happy-dom**: 154.3 MB peak RSS
- **poku + jsdom**: 157.1 MB peak RSS
- **jest + jsdom**: 203.4 MB peak RSS
- **vitest + happy-dom**: 115.2 MB peak RSS
- **poku + happy-dom**: 127.4 MB peak RSS
- **vitest + jsdom**: 147.8 MB peak RSS
- **poku + jsdom**: 172.1 MB peak RSS
- **jest + jsdom**: 198.8 MB peak RSS

### Consistency (lower stdev = more predictable)

- **poku + jsdom**: σ = 0.008s
- **poku + happy-dom**: σ = 0.033s
- **vitest + happy-dom**: σ = 0.047s
- **jest + jsdom**: σ = 0.056s
- **vitest + jsdom**: σ = 0.057s
- **poku + happy-dom**: σ = 0.004s
- **poku + jsdom**: σ = 0.007s
- **vitest + jsdom**: σ = 0.011s
- **vitest + happy-dom**: σ = 0.013s
- **jest + jsdom**: σ = 0.041s

## Key findings

- **Fastest**: poku + jsdom — 0.444s mean
- **Slowest**: vitest + jsdom — 1.193s mean
- **Speed spread**: 169% difference between fastest and slowest
- **Fastest**: poku + happy-dom — 0.115s mean
- **Slowest**: vitest + jsdom — 0.968s mean
- **Speed spread**: 745% difference between fastest and slowest

### Interpretation

Expand All @@ -100,7 +101,6 @@ processes with minimal bootstrap — means cold-start overhead is proportional t
files, not to the framework's own initialization.

**jest** carries the heaviest startup cost due to:

1. Babel transformation of every TSX file on first run (no persistent cache in this benchmark)
2. 'jest-worker' process pool initialisation
3. JSDOM environment setup per test file
Expand Down
4 changes: 2 additions & 2 deletions benchmark/package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

74 changes: 37 additions & 37 deletions benchmark/results.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"timestamp": "2026-04-01T12:41:50.774Z",
"timestamp": "2026-04-05T19:06:36.388Z",
"system": {
"nodeVersion": "v22.5.1",
"platform": "darwin 25.4.0",
Expand All @@ -14,68 +14,68 @@
"results": [
{
"label": "poku + happy-dom",
"mean": 559.9537916000002,
"min": 515.060708,
"max": 599.9880840000001,
"stddev": 33.06234198220535,
"meanRss": 161808384,
"meanUserCpu": 672,
"meanSysCpu": 202,
"mean": 114.5411498,
"min": 111.865375,
"max": 108.566625,
"stddev": 4.164501573153617,
"meanRss": 133624627.2,
"meanUserCpu": 358,
"meanSysCpu": 80,
"runs": 7,
"validRuns": 7,
"failures": 0
},
{
"label": "poku + jsdom",
"mean": 443.6319584000001,
"min": 428.5047089999998,
"max": 450.59829100000024,
"stddev": 7.9857833050707905,
"meanRss": 164767334.4,
"meanUserCpu": 432,
"meanSysCpu": 92,
"mean": 294.0646082,
"min": 280.499125,
"max": 298.546375,
"stddev": 6.855649968732701,
"meanRss": 180456652.8,
"meanUserCpu": 532,
"meanSysCpu": 102,
"runs": 7,
"validRuns": 7,
"failures": 0
},
{
"label": "jest + jsdom",
"mean": 1040.1169332000002,
"min": 975.1540420000001,
"max": 1134.9154159999998,
"stddev": 56.06532298235639,
"meanRss": 213300019.2,
"meanUserCpu": 936,
"meanSysCpu": 228,
"mean": 842.5360584000006,
"min": 768.3345829999998,
"max": 879.3446250000015,
"stddev": 40.57889525259689,
"meanRss": 208506060.8,
"meanUserCpu": 784,
"meanSysCpu": 152,
"runs": 7,
"validRuns": 7,
"failures": 0
},
{
"label": "vitest + jsdom",
"mean": 1192.7835997999996,
"min": 1129.2755830000006,
"max": 1268.7202919999982,
"stddev": 56.83080807294834,
"meanRss": 159652249.6,
"meanUserCpu": 3798,
"meanSysCpu": 1380,
"mean": 967.6274329999997,
"min": 950.946915999999,
"max": 985.529625000001,
"stddev": 11.190019303221952,
"meanRss": 155002470.4,
"meanUserCpu": 3524,
"meanSysCpu": 954,
"runs": 7,
"validRuns": 7,
"failures": 0
},
{
"label": "vitest + happy-dom",
"mean": 1041.3667920000007,
"min": 990.0950420000008,
"max": 1126.3293750000012,
"stddev": 47.26983715919258,
"meanRss": 122814464,
"meanUserCpu": 3346,
"meanSysCpu": 1156,
"mean": 844.6526001999998,
"min": 824.9157500000001,
"max": 856.1071249999986,
"stddev": 13.21280400724632,
"meanRss": 120832000,
"meanUserCpu": 3056,
"meanSysCpu": 900,
"runs": 7,
"validRuns": 7,
"failures": 0
}
]
}
}
Loading
Loading