|
| 1 | +# AGENTS.md - codspeed-go |
| 2 | + |
| 3 | +## Project Overview |
| 4 | + |
| 5 | +codspeed-go is the CodSpeed Go benchmark runner. It patches Go's `testing` package at build time using Go's `-overlay` flag, injecting instrumentation into benchmarks without requiring user code changes. |
| 6 | + |
| 7 | +## Build & Test |
| 8 | + |
| 9 | +```bash |
| 10 | +cargo check # type-check |
| 11 | +cargo test # unit + integration tests (uses rstest + insta) |
| 12 | +cargo nextest run --all # parallel test execution (used in CI) |
| 13 | +cargo clippy # lint |
| 14 | +cargo fmt # format |
| 15 | + |
| 16 | +# Run runner against example project (from example/ directory) |
| 17 | +cargo run -- test -bench . -benchtime 3s ./... |
| 18 | + |
| 19 | +# Run a single integration test |
| 20 | +cargo test test_name_here |
| 21 | +``` |
| 22 | + |
| 23 | +Pre-commit hooks enforce: `go-mod-tidy`, `go-fmt`, `cargo fmt`, `cargo check --all-targets`, `clippy -D warnings`. |
| 24 | + |
| 25 | +CI tests against Go 1.24.x and 1.25.x. Go 1.24 tests require `GOEXPERIMENT=synctest`. |
| 26 | + |
| 27 | +## Architecture |
| 28 | + |
| 29 | +Rust workspace with a single crate: `go-runner/` (`codspeed-go-runner`), edition 2024, toolchain 1.90.0. |
| 30 | + |
| 31 | +**Flow:** `main.rs` parses CLI args → `runner::run()` generates overlay + runs `go test` → Go benchmarks write raw JSON results to `$CODSPEED_PROFILE_FOLDER/raw_results/` → `collect_walltime_results()` aggregates into `results/{pid}.json`. |
| 32 | + |
| 33 | +**Overlay mechanism:** Three files are overlaid into `$GOROOT/src/testing/`: |
| 34 | +- `benchmark.go` — replaces the standard `testing.B` implementation (version-specific: 1.24 or 1.25+) |
| 35 | +- `codspeed.go` — CodSpeed measurement logic, result saving, `codspeed` struct with per-round measurements |
| 36 | +- `instrument-hooks.go` — cgo FFI bindings to the C instrument-hooks library (downloaded at runtime) |
| 37 | + |
| 38 | +The overlay uses `@@PLACEHOLDER@@` strings that the Rust runner substitutes at runtime (`@@INSTRUMENT_HOOKS_DIR@@`, `@@CODSPEED_PROFILE_DIR@@`, `@@GO_RUNNER_VERSION@@`). |
| 39 | + |
| 40 | +**CLI parser:** Custom hand-rolled parser in `cli.rs` because Go uses single-dash flags (`-bench`, `-benchtime`) which clap/structopt don't support. |
| 41 | + |
| 42 | +## Runner Modes |
| 43 | + |
| 44 | +- **walltime** — wall-clock measurement with warmup, multiple rounds. Used on bare metal runners. |
| 45 | +- **simulation** — single iteration under instrumentation (valgrind/callgrind). Used on CodSpeed infrastructure. |
| 46 | +- **memory** — memory profiling mode. |
| 47 | + |
| 48 | +Set via `CODSPEED_RUNNER_MODE` env var (default: `walltime`). |
| 49 | + |
| 50 | +## Integration Tests |
| 51 | + |
| 52 | +Tests in `go-runner/src/integration_tests.rs` use real Go projects from `go-runner/testdata/projects/` (git submodules). Uses `insta` for snapshot testing with redactions for non-deterministic fields (PID, version, stats). Accept new snapshots with `cargo insta review`. |
| 53 | + |
| 54 | +## Key Environment Variables |
| 55 | + |
| 56 | +- `CODSPEED_RUNNER_MODE` — `walltime` (default), `simulation`, or `memory` |
| 57 | +- `CODSPEED_PROFILE_FOLDER` — where results are written (default: `/tmp`) |
| 58 | +- `CODSPEED_LOG` — log level filter (default: `info`) |
| 59 | + |
| 60 | +## Gotchas |
| 61 | + |
| 62 | +- `instrument-hooks.go` requires cgo (`import "C"`). The runner sets `CGO_ENABLED=1` and checks for a C compiler before building. Without this, Go silently excludes the file causing "undefined: InstrumentHooks" errors. |
| 63 | +- The runner uses `$GOROOT/bin/go` directly (not PATH) to avoid infinite recursion with the runner binary intercepting `go test`. |
| 64 | +- The runner sets custom `GOCACHE` and `GOMODCACHE` to temp dirs to avoid cache conflicts. |
| 65 | +- Overlay patches are maintained as `.patch` files alongside the full `.go` files in `go-runner/overlay/`. Use `update-patch.sh` to regenerate. |
| 66 | + |
| 67 | +## Release Process |
| 68 | + |
| 69 | +Update version in `go-runner/Cargo.toml`, generate changelog with `git cliff --tag "v$VERSION" -o CHANGELOG.md`, commit, create annotated tag (`git tag -a`), push with `--follow-tags`. See `RELEASE.md` for details. |
0 commit comments