Skip to content

Add agentic workflow for automated issue triage#4177

Open
priyankatiwari08 wants to merge 5 commits intodotnet:mainfrom
priyankatiwari08:feature/agentic-issue-triage
Open

Add agentic workflow for automated issue triage#4177
priyankatiwari08 wants to merge 5 commits intodotnet:mainfrom
priyankatiwari08:feature/agentic-issue-triage

Conversation

@priyankatiwari08
Copy link
Copy Markdown
Contributor

@priyankatiwari08 priyankatiwari08 commented Apr 10, 2026

Summary

This PR introduces a GitHub Agentic Workflow (gh-aw) that automatically triages new issues using Copilot.

What it does

When a new issue is opened, the workflow:

  1. Classifies issue type — Bug, Feature, Question, or Task
  2. Validates environment details (bugs only) — checks for SqlClient version, .NET target framework, SQL Server version, OS, repro steps, expected/actual behavior
  3. Assigns area labels — matches issue content against 14 area labels (e.g. Area\Async, Area\Connection Pooling, Area\Managed SNI)
  4. Searches for duplicates — searches repo:dotnet/SqlClient for similar issues
  5. Checks for regressions — if the reporter mentions a previously working version
  6. Posts a single consolidated triage summary — one comment with a structured table (issue type, environment, area, duplicates, regression) + analysis + next steps
  7. Assigns Copilot coding agent — for confirmed bugs with complete environment info

Files added

1. .github/workflows/issue-triage.md — Workflow Definition

This is the only file that needs to be maintained. It contains:

  • YAML frontmatter: Defines the trigger (issues: opened), the AI engine (copilot), and safety limits (max 1 comment, max 5 labels).
  • Markdown body: Natural language instructions that tell the Copilot agent how to triage each issue — role definition, analysis steps (classify type, validate environment, match area labels, search duplicates, check regression), area label lookup table, comment template, and action sequence.

This file is written in plain Markdown because GitHub Agentic Workflows use an LLM (Copilot) to interpret instructions, unlike traditional YAML workflows that use scripted steps. When changes to the triage logic are needed (e.g. adding a new area label, changing the comment format), only this file is edited, then recompiled.

2. .github/workflows/issue-triage.lock.yml — Compiled Actions YAML (~61KB)

This is the auto-generated GitHub Actions workflow file produced by running gh aw compile issue-triage.md. It is what GitHub Actions actually executes when triggered. It contains:

  • The full two-job Actions YAML infrastructure (agentic job + safe-output processor job)
  • The instructions from issue-triage.md embedded as runtime imports
  • Safe-outputs enforcement configuration (max limits, allowed actions)
  • MCP server setup (read-only GitHub MCP + write-sink safe-outputs MCP)

This file should never be edited manually. It is regenerated every time issue-triage.md is compiled. The edit-compile-push cycle is: edit .md → run gh aw compile → commit both files → push.

3. .github/aw/actions-lock.json — Pinned Action Versions

This is a lockfile generated by gh aw compile that pins the exact versions of the GitHub Actions used by the agentic workflow runtime. Similar to package-lock.json in Node.js or Cargo.lock in Rust, it ensures the workflow uses the same action versions on every run, preventing unexpected behavior from upstream action updates.

This file should never be edited manually. It is regenerated automatically during compilation.

Architecture

The workflow uses the safe-outputs pattern:

  • Job 1 (Agentic): Copilot agent reads the issue via read-only GitHub MCP, reasons about it, and proposes actions (labels, comment, agent assignment) written to NDJSON files
  • Job 2 (Safe-output Processor): A separate privileged job validates proposed actions against configured limits (max 1 comment, max 5 labels) and executes them

This ensures the AI agent never has direct write access to the repository.

Safety limits configured

  • add-comment: max: 1 — at most 1 comment per issue
  • add-labels: max: 5 — at most 5 labels per issue
  • hide-older-comments: true — collapses previous triage comments if re-triggered

Prerequisites (post-merge)

The following setup is required by a repo admin after merging:

  1. Set agentic workflow secret:

    gh aw secrets set COPILOT_GITHUB_TOKEN --value <token>
    
  2. Set agent assignment secret (required — enables automatic Copilot coding agent assignment for confirmed bugs with complete environment details):

    gh aw secrets set GH_AW_AGENT_TOKEN --value <token>
    
  3. Create missing labels (these don't exist yet on this repo):

    • Triage Needed :new: — applied to all new issues
    • Needs More Info :information_source: — applied when environment details are missing
  4. Ensure gh-aw is enabled for the dotnet org / SqlClient repo

Tested on

This workflow was developed and tested on priyankatiwari08/SqlClient-test-prtiwar with multiple test issues covering:

  • Complete bug reports (all env details present) — correctly triaged with area labels + Copilot assignment
  • Incomplete bug reports (missing OS, .NET TF, SQL Server version) — correctly flagged missing fields + Needs More Info label
  • Various area classifications (Async, Managed SNI, Sql Bulk Copy) — correctly matched

Introduces a GitHub Agentic Workflow (gh-aw) that automatically triages
new issues using Copilot. The workflow:

- Classifies issue type (Bug/Feature/Question/Task)
- Validates environment details for bugs
- Assigns area labels based on issue content
- Searches for duplicate issues in the repo
- Checks for regressions
- Posts a single consolidated triage summary comment
- Assigns Copilot coding agent for confirmed bugs with complete info

Files added:
- .github/workflows/issue-triage.md (workflow definition)
- .github/workflows/issue-triage.lock.yml (compiled Actions YAML)
- .github/aw/actions-lock.json (pinned action versions)
@github-project-automation github-project-automation bot moved this to To triage in SqlClient Board Apr 10, 2026
Copilot AI review requested due to automatic review settings April 10, 2026 08:10
@priyankatiwari08 priyankatiwari08 requested a review from a team as a code owner April 10, 2026 08:10
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds a GitHub Agentic Workflow (gh-aw) for automated issue triage in the SqlClient repo, using a Copilot-powered agent plus a privileged “safe-outputs” executor job.

Changes:

  • Added an agent-authored workflow definition (issue-triage.md) describing triage logic and safe-output constraints.
  • Added the compiled/locked GitHub Actions workflow (issue-triage.lock.yml) generated by gh aw compile.
  • Added an action lockfile (.github/aw/actions-lock.json) pinning action SHAs.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 7 comments.

File Description
.github/workflows/issue-triage.md Defines the agent prompt, triage steps, and safe-output configuration for labeling/commenting/assignment.
.github/workflows/issue-triage.lock.yml Auto-generated compiled Actions workflow that executes the agent + safe-outputs pipeline.
.github/aw/actions-lock.json Pins action versions used by the compiled workflow.

Comment thread .github/workflows/issue-triage.md Outdated
Comment thread .github/workflows/issue-triage.md
Comment thread .github/workflows/issue-triage.md Outdated
Comment thread .github/workflows/issue-triage.md Outdated
Comment thread .github/workflows/issue-triage.lock.yml Outdated
Comment thread .github/workflows/issue-triage.lock.yml
Comment thread .github/workflows/issue-triage.md
Copilot AI review requested due to automatic review settings April 10, 2026 08:50
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 4 comments.

Comment thread .github/workflows/issue-triage.md Outdated
Comment thread .github/workflows/issue-triage.md Outdated
Comment thread .github/workflows/issue-triage.lock.yml Outdated
Comment thread .github/aw/actions-lock.json
…s, fix Bug classification wording, fix Needs More Info label format
@paulmedynski paulmedynski moved this from To triage to In review in SqlClient Board Apr 10, 2026
Copilot AI review requested due to automatic review settings April 13, 2026 11:05
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (1)

.github/workflows/issue-triage.md:130

  • This file says the workflow always reads the issue, applies labels, and posts one summary comment, but later suggests calling noop when “no action is needed”. As written, it’s ambiguous whether noop should replace the labeling/comment steps or be in addition to them, which could lead to either missing triage output or emitting conflicting safe-outputs. Please clarify the control flow (e.g., if noop is used, explicitly say to skip add_labels/add_comment, or remove noop if a comment is always required).
**Finally**: If this is a confirmed code bug with complete environment info,
call `assign_to_agent` to assign Copilot coding agent.

If the issue is spam or no action is needed, call the `noop` tool instead.

Comment on lines +84 to +86
**First**: Call `add_labels` with at most 5 labels total (safe-outputs limit):
- `:new: Triage Needed` (always)
- Exactly **one** `Area\*` label from the table above (pick the single best match)
Copy link

Copilot AI Apr 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The instructions for the triage label appear inconsistent across the repo: this workflow uses :new: Triage Needed, but other repo guidance uses Triage Needed :new: (e.g., .github/prompts/triage-issue.prompt.md and .github/copilot-instructions.md). If the actual label name differs from what’s configured here, add_labels will fail at runtime. Please align the workflow to the canonical label name and/or update the repo guidance/templates so they all match exactly.

Copilot uses AI. Check for mistakes.
add-labels:
max: 5
assign-to-agent:
github-token: ${{ secrets.GH_AW_AGENT_TOKEN }}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's defer this piece to later. We can manually assign to an agent when we know the issue is in a good spot.

safe-outputs:
add-comment:
max: 1
hide-older-comments: true
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cool! this will work nicely when we can get to a spot where the triage agent runs iteratively

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How is this file generated? Is there some step we'll need to take each time we update the markdown to regenerate it?

<2-4 sentences: what the issue is about, which component is likely affected,
and severity assessment (P0-P3)>

### Next Steps
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should add some instructions here to either:
a. not provide guidance to users
or
b. add a header before any guidance that it is machine generated and not human verified

On my test issue, copilot provided my guidance to try managed SNI or to check libssl if on linux. In this case, not unreasonable, but it shouldn't be construed as trusted advice.

Image

@github-project-automation github-project-automation bot moved this from In review to In progress in SqlClient Board Apr 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: In progress

Development

Successfully merging this pull request may close these issues.

4 participants