You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: AGENTS.md
+2Lines changed: 2 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -2,6 +2,8 @@
2
2
3
3
## Regression — iterate until complete
4
4
5
+
Full **`run-regression*.sh`** is a **system / integration** suite for this repo (cluster-backed tiers optional). It is **not** the same as “every test on every application pull request” — see [docs/REGRESSION.md](docs/REGRESSION.md) for PR vs platform scope.
6
+
5
7
Do **not** stop after a single partial test run. Follow **[docs/AGENT-REGRESSION.md](docs/AGENT-REGRESSION.md)**:
6
8
7
9
- Run **`bash scripts/run-regression-agent.sh`** (or **`run-regression-agent-full.sh`** if Results + DB must pass).
Copy file name to clipboardExpand all lines: docs/REGRESSION.md
+16-5Lines changed: 16 additions & 5 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -4,6 +4,17 @@ Single entrypoint: **`scripts/run-regression.sh`** (see `--help`). Run when gree
4
4
5
5
**Cursor / agents:** follow **[AGENT-REGRESSION.md](AGENT-REGRESSION.md)** — run **`scripts/run-regression-agent.sh`** (or **`run-regression-agent-full.sh`**) and **iterate with fixes until exit 0**, not a single partial run. Repo **[AGENTS.md](../AGENTS.md)** points here.
6
6
7
+
## Application PR pipeline vs platform (system) regression
8
+
9
+
Do **not** confuse these:
10
+
11
+
| Scope | What runs | Typical trigger |
12
+
|-------|-----------|-----------------|
13
+
|**Application PR** (`stack-pr-test` on an **app** repo) | Stack-defined tests only — e.g. that app’s Newman/Playwright/Artillery as declared in `stacks/*.yaml`, against the intercept build. | Every PR on the **application** repository (when webhooks/Tekton are wired). |
14
+
|**Platform regression** (`scripts/run-regression*.sh` on **this** repo) |**System / integration** tiers: Phase 1 + orchestrator + shared libs + GUI pytest, Playwright for **management-gui**, real **`stack-dag-verify`** PipelineRun, Newman against **orchestrator** API, optional Tekton Results, optional Kind E2E. |**Manual**, **scheduled**, **pre-release**, or **agent loop** — **not** “automatically on every pull request” unless *you* add GitHub Actions (or similar) to do so. |
15
+
16
+
So: **not all tests run on every PR.** The full regression driver is a **system test** bar for the platform; app PRs run a narrower, stack-scoped test stage.
17
+
7
18
**Streaming / timestamps:** use **`scripts/run-regression-stream.sh`** — same arguments, prefixes each line with `[HH:MM:SS]` and preserves the real exit code (plain `| while read` does not).
8
19
9
20
**Port-forward prep:**[common.sh](../scripts/common.sh) defines **`free_tcp_port`**. Regression (**[run-regression.sh](../scripts/run-regression.sh)**) runs a **prep** step when `kubectl` works: frees **`ORCHESTRATOR_TEST_PORT`** (default **9091**) and **`RESULTS_API_LOCAL_PORT`** (default **8080**). Set **`REGRESSION_FREE_PORTS=0`** to skip that prep. [run-orchestrator-tests.sh](../scripts/run-orchestrator-tests.sh) also frees the orchestrator local port; [verify-results-in-db.sh](../scripts/verify-results-in-db.sh) frees **8080** (or **`RESULTS_API_LOCAL_PORT`**) before forwarding the Results API.
@@ -77,11 +88,11 @@ Environment:
77
88
-`REGRESSION_DAG_VERIFY=auto|skip|yes`
78
89
-`DAG_VERIFY_TIMEOUT` — seconds passed to `verify-dag-phase2.sh``--timeout` (default **300** in regression).
79
90
80
-
## Suggested “complete regression” before doc work
91
+
## Suggested regression cadence (not all on every PR)
81
92
82
-
1.`./scripts/run-regression.sh --local-only`on every commit / PR.
83
-
2.On a cluster with Tekton + orchestrator:**`./scripts/run-regression.sh --cluster --require-dag-verify`**(ensures at least one **Succeeded**`stack-dag-verify`run, not only Newman side-effects).
84
-
3. With Tekton Results: `--with-results-verify` or rely on **auto** when the API exists.
85
-
4.`--kind-e2e` when changing bootstrap, intercepts, or Results integration.
93
+
1.**Often (fast, no cluster):**`./scripts/run-regression.sh --local-only`— Phase 1 + pytest + vitest; good for frequent pushes; safe to wire into lightweight CI.
94
+
2.**System bar (cluster):****`./scripts/run-regression.sh --cluster --require-dag-verify`**when you need a real **Succeeded**`stack-dag-verify`and orchestrator Newman — treat as **integration / system** work: before releases, after big platform changes, on a schedule, or when agents/docs require proof — **not** as “must pass on every GitHub PR” unless you explicitly configure that.
95
+
3.**With Tekton Results:**`--with-results-verify` or rely on **auto** when the API exists.
96
+
4.**Kind E2E:**`--kind-e2e` when changing bootstrap, intercepts, or Results integration (heavy; occasional).
86
97
87
98
After the tiers you care about are green, update [milestones/milestone-8.md](../milestones/milestone-8.md) and related testing docs.
Copy file name to clipboardExpand all lines: docs/TESTING-AND-REGRESSION-OVERVIEW.md
+2Lines changed: 2 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -2,6 +2,8 @@
2
2
3
3
This document is the **story** behind how tekton-dag is verified end-to-end. Use it for **onboarding**, **release checklists**, and **video voiceover**. Operational detail lives in [REGRESSION.md](REGRESSION.md), [AGENT-REGRESSION.md](AGENT-REGRESSION.md), and [scripts/run-regression.sh](../scripts/run-regression.sh).
4
4
5
+
**Scope:** An **application PR** only runs that stack’s declared tests inside **stack-pr-test**. The **`run-regression*.sh`** suites are **platform / system** checks (orchestrator, GUI, real PipelineRuns, Newman, Results). They are **not** all expected to run on every GitHub pull request unless you wire CI that way — see [REGRESSION.md](REGRESSION.md) § *Application PR pipeline vs platform (system) regression*.
6
+
5
7
## Why regression is more than unit tests
6
8
7
9
| Layer | What it proves | Typical command / artifact |
Copy file name to clipboardExpand all lines: docs/demos/README.md
+2Lines changed: 2 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,5 +1,7 @@
1
1
# Demo video toolchain (M8 / M12.2)
2
2
3
+
**Portable copy** for other repos: [`../../documentation-generator/video-framework/`](../../documentation-generator/video-framework/) (+ [`../../documentation-generator/example-demo-bundle/`](../../documentation-generator/example-demo-bundle/)). Canonical scripts stay here; refresh the copy when the framework changes.
4
+
3
5
Regenerate narration, animations, terminal recordings, and composed MP4s:
0 commit comments