Add agentic workflow for automated issue triage#4177
Add agentic workflow for automated issue triage#4177priyankatiwari08 wants to merge 5 commits intodotnet:mainfrom
Conversation
Introduces a GitHub Agentic Workflow (gh-aw) that automatically triages new issues using Copilot. The workflow: - Classifies issue type (Bug/Feature/Question/Task) - Validates environment details for bugs - Assigns area labels based on issue content - Searches for duplicate issues in the repo - Checks for regressions - Posts a single consolidated triage summary comment - Assigns Copilot coding agent for confirmed bugs with complete info Files added: - .github/workflows/issue-triage.md (workflow definition) - .github/workflows/issue-triage.lock.yml (compiled Actions YAML) - .github/aw/actions-lock.json (pinned action versions)
There was a problem hiding this comment.
Pull request overview
This PR adds a GitHub Agentic Workflow (gh-aw) for automated issue triage in the SqlClient repo, using a Copilot-powered agent plus a privileged “safe-outputs” executor job.
Changes:
- Added an agent-authored workflow definition (
issue-triage.md) describing triage logic and safe-output constraints. - Added the compiled/locked GitHub Actions workflow (
issue-triage.lock.yml) generated bygh aw compile. - Added an action lockfile (
.github/aw/actions-lock.json) pinning action SHAs.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 7 comments.
| File | Description |
|---|---|
| .github/workflows/issue-triage.md | Defines the agent prompt, triage steps, and safe-output configuration for labeling/commenting/assignment. |
| .github/workflows/issue-triage.lock.yml | Auto-generated compiled Actions workflow that executes the agent + safe-outputs pipeline. |
| .github/aw/actions-lock.json | Pins action versions used by the compiled workflow. |
…s, fix Bug classification wording, fix Needs More Info label format
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.
Comments suppressed due to low confidence (1)
.github/workflows/issue-triage.md:130
- This file says the workflow always reads the issue, applies labels, and posts one summary comment, but later suggests calling
noopwhen “no action is needed”. As written, it’s ambiguous whethernoopshould replace the labeling/comment steps or be in addition to them, which could lead to either missing triage output or emitting conflicting safe-outputs. Please clarify the control flow (e.g., ifnoopis used, explicitly say to skipadd_labels/add_comment, or removenoopif a comment is always required).
**Finally**: If this is a confirmed code bug with complete environment info,
call `assign_to_agent` to assign Copilot coding agent.
If the issue is spam or no action is needed, call the `noop` tool instead.
| **First**: Call `add_labels` with at most 5 labels total (safe-outputs limit): | ||
| - `:new: Triage Needed` (always) | ||
| - Exactly **one** `Area\*` label from the table above (pick the single best match) |
There was a problem hiding this comment.
The instructions for the triage label appear inconsistent across the repo: this workflow uses :new: Triage Needed, but other repo guidance uses Triage Needed :new: (e.g., .github/prompts/triage-issue.prompt.md and .github/copilot-instructions.md). If the actual label name differs from what’s configured here, add_labels will fail at runtime. Please align the workflow to the canonical label name and/or update the repo guidance/templates so they all match exactly.
| add-labels: | ||
| max: 5 | ||
| assign-to-agent: | ||
| github-token: ${{ secrets.GH_AW_AGENT_TOKEN }} |
There was a problem hiding this comment.
Let's defer this piece to later. We can manually assign to an agent when we know the issue is in a good spot.
| safe-outputs: | ||
| add-comment: | ||
| max: 1 | ||
| hide-older-comments: true |
There was a problem hiding this comment.
cool! this will work nicely when we can get to a spot where the triage agent runs iteratively
There was a problem hiding this comment.
How is this file generated? Is there some step we'll need to take each time we update the markdown to regenerate it?
| <2-4 sentences: what the issue is about, which component is likely affected, | ||
| and severity assessment (P0-P3)> | ||
|
|
||
| ### Next Steps |
There was a problem hiding this comment.
We should add some instructions here to either:
a. not provide guidance to users
or
b. add a header before any guidance that it is machine generated and not human verified
On my test issue, copilot provided my guidance to try managed SNI or to check libssl if on linux. In this case, not unreasonable, but it shouldn't be construed as trusted advice.
Summary
This PR introduces a GitHub Agentic Workflow (gh-aw) that automatically triages new issues using Copilot.
What it does
When a new issue is opened, the workflow:
Area\Async,Area\Connection Pooling,Area\Managed SNI)repo:dotnet/SqlClientfor similar issuesFiles added
1.
.github/workflows/issue-triage.md— Workflow DefinitionThis is the only file that needs to be maintained. It contains:
issues: opened), the AI engine (copilot), and safety limits (max 1 comment, max 5 labels).This file is written in plain Markdown because GitHub Agentic Workflows use an LLM (Copilot) to interpret instructions, unlike traditional YAML workflows that use scripted steps. When changes to the triage logic are needed (e.g. adding a new area label, changing the comment format), only this file is edited, then recompiled.
2.
.github/workflows/issue-triage.lock.yml— Compiled Actions YAML (~61KB)This is the auto-generated GitHub Actions workflow file produced by running
gh aw compile issue-triage.md. It is what GitHub Actions actually executes when triggered. It contains:issue-triage.mdembedded as runtime importsThis file should never be edited manually. It is regenerated every time
issue-triage.mdis compiled. The edit-compile-push cycle is: edit.md→ rungh aw compile→ commit both files → push.3.
.github/aw/actions-lock.json— Pinned Action VersionsThis is a lockfile generated by
gh aw compilethat pins the exact versions of the GitHub Actions used by the agentic workflow runtime. Similar topackage-lock.jsonin Node.js orCargo.lockin Rust, it ensures the workflow uses the same action versions on every run, preventing unexpected behavior from upstream action updates.This file should never be edited manually. It is regenerated automatically during compilation.
Architecture
The workflow uses the safe-outputs pattern:
This ensures the AI agent never has direct write access to the repository.
Safety limits configured
add-comment: max: 1— at most 1 comment per issueadd-labels: max: 5— at most 5 labels per issuehide-older-comments: true— collapses previous triage comments if re-triggeredPrerequisites (post-merge)
The following setup is required by a repo admin after merging:
Set agentic workflow secret:
Set agent assignment secret (required — enables automatic Copilot coding agent assignment for confirmed bugs with complete environment details):
Create missing labels (these don't exist yet on this repo):
Triage Needed :new:— applied to all new issuesNeeds More Info :information_source:— applied when environment details are missingEnsure gh-aw is enabled for the dotnet org / SqlClient repo
Tested on
This workflow was developed and tested on priyankatiwari08/SqlClient-test-prtiwar with multiple test issues covering:
Needs More Infolabel