Skip to content

Quote --filter strings in docs#4994

Open
xtqqczze wants to merge 2 commits intodotnet:mainfrom
xtqqczze:quote-filter
Open

Quote --filter strings in docs#4994
xtqqczze wants to merge 2 commits intodotnet:mainfrom
xtqqczze:quote-filter

Conversation

@xtqqczze
Copy link
Copy Markdown
Contributor

No description provided.

@xtqqczze xtqqczze changed the title Quote --filter strings Quote --filter strings in docs Oct 1, 2025
@xtqqczze

This comment was marked as resolved.

@xtqqczze xtqqczze marked this pull request as ready for review March 27, 2026 19:32
@danmoseley danmoseley requested a review from Copilot April 11, 2026 22:38
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates documentation (and one benchmark project) to quote --filter glob patterns so shell globbing doesn’t unexpectedly expand * before BenchmarkDotNet receives it.

Changes:

  • Add quoting around --filter values in various docs/examples.
  • Update PowerShell.Benchmarks.csproj StartArguments to quote --filter.
Show a summary per file
File Description
src/tools/ResultsComparer/README.md Quotes the --filter example in ResultsComparer usage.
src/benchmarks/real-world/PowerShell.Benchmarks/PowerShell.Benchmarks.csproj Quotes --filter in StartArguments.
src/benchmarks/micro/README.md Quotes --filter in microbenchmark CLI examples.
scripts/BENCHMARKS_LOCAL_README.md Quotes --filter in benchmarks_local.py examples.
docs/microbenchmark-design-guidelines.md Quotes --filter in an ETW profiling example.
docs/crank-to-helix-workflow.md Quotes --filter inside bdnArgs example + description.
docs/benchmarking-workflow-dotnet-runtime.md Quotes many --filter examples, including ones using shell variables.
docs/benchmarkdotnet.md Updates the main filtering docs to use quoted --filter examples and refines the Unix-shell globbing note.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comments suppressed due to low confidence (7)

docs/crank-to-helix-workflow.md:107

  • This bullet repeats the same problematic nested quoting as the example above: the inner single quotes will likely be preserved all the way to BenchmarkDotNet because the tooling splits args without shell-style quote processing. Recommend documenting bdnArgs="--filter *Linq*" instead (outer quotes only).
- `--profile win-x64`: Configures crank to a local Windows x64 build of the runtime, and sets the Helix Queue to a Windows x64 queue.
- `--variable bdnArgs="--filter '*Linq*'"`: Sets arguments to pass to BenchmarkDotNet that will filter it to only Linq benchmarks
- `--profile msft-internal`: Sets the crank agent endpoint to the internal hosted crank agent

docs/benchmarking-workflow-dotnet-runtime.md:260

  • Same issue as above: --filter '$TestToRun*' in the Linux command prevents $TestToRun from expanding. Use double quotes so the variable expands but * is not pathname-expanded, e.g. --filter "$TestToRun*".

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net11.0 --no-restore --no-build -- --filter '$TestToRun*' --anyCategories Libraries Runtime "" --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog "" --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

docs/benchmarking-workflow-dotnet-runtime.md:141

  • --filter '$someFilter' is problematic in both common cases: in cmd.exe the single quotes are literal, and in bash the single quotes prevent $someFilter from expanding. Prefer --filter "$someFilter" (allows variable expansion but avoids glob expansion) or use a placeholder like <someFilter> without $ if it's not meant to expand.
```cmd
dotnet run -c Release -f net11.0 --filter '$someFilter' \
    --coreRun C:\Projects\runtime\artifacts\bin\testhost\net11.0-windows-Release-x64\shared\Microsoft.NETCore.App\11.0.0\CoreRun.exe
**docs/benchmarking-workflow-dotnet-runtime.md:298**
* This Windows section is in a ```cmd block; single quotes are not removed by cmd.exe, so `--filter '$TestToRun*'` will include quotes and may not match. Prefer `--filter *` (no quotes) or `--filter "..."` as needed; if `$TestToRun` is meant to expand, use the appropriate syntax for the documented shell.

Run

dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net11.0 --no-restore --no-build -- --filter '$TestToRun*' --anyCategories Libraries Runtime "" --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog "" --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

docs/benchmarking-workflow-dotnet-runtime.md:488

  • These commands are shown in a ```cmd block but use single-quoted filters. In cmd.exe the quotes are literal; prefer double quotes (e.g. "System.IO.Pipes*") or no quotes for Windows-only examples so the filter value passed to BenchmarkDotNet does not include `'` characters.
C:\Projects\performance\src\benchmarks\micro> dotnet run -c Release -f net11.0 \
    --artifacts "C:\results\before" \
    --coreRun "C:\Projects\runtime\artifacts\bin\testhost\net11.0-windows-Release-x64\shared\Microsoft.NETCore.App\11.0.0\CoreRun.exe" \
    --filter 'System.IO.Pipes*'

docs/benchmarking-workflow-dotnet-runtime.md:547

  • In a ```cmd block, --filter '$YourFilter' will pass the single quotes through on cmd.exe, and on bash it prevents `$YourFilter` expansion if it’s meant to be a variable. Prefer double quotes (or a `` placeholder) depending on whether the text is intended as a real variable.
To profile the benchmarked code and produce an ETW Trace file ([read more](./benchmarkdotnet.md#Profiling)):

```cmd
dotnet run -c Release -f net11.0 --profiler ETW --filter '$YourFilter'

docs/benchmarking-workflow-dotnet-runtime.md:571

  • This sentence suggests running all benchmarks using --filter '*'. In cmd.exe, single quotes are literal, so this can be interpreted as a filter containing quotes and match nothing. Prefer documenting --filter * (Windows) or --filter "*" / --filter '*' explicitly as shell-specific alternatives.

When you identify and fix the regression, you should use [ResultsComparer](../src/tools/ResultsComparer/README.md) to confirm that you have solved the problem. Please remember that if the regression was found in a very common type like `Span<T>` and you are not sure which benchmarks to run, you can run all of them using `--filter '*'`.

  • Files reviewed: 8/8 changed files
  • Comments generated: 8

```cmd
dotnet run -c Release matrix decompress --input D:\results\Performance-Runs.zip --output D:\results\net7.0-preview3
dotnet run -c Release matrix --input D:\results\net7.0-preview3 --base net7.0-preview2 --diff net7.0-preview3 --threshold 10% --noise 2ns --filter System.IO*
dotnet run -c Release matrix --input D:\results\net7.0-preview3 --base net7.0-preview2 --diff net7.0-preview3 --threshold 10% --noise 2ns --filter 'System.IO*'
<ImplicitUsings>enable</ImplicitUsings>
<AllowUnsafeBlocks>true</AllowUnsafeBlocks>
<StartArguments>--filter *</StartArguments>
<StartArguments>--filter '*'</StartArguments>
Comment on lines 31 to 41
To filter the benchmarks using a glob pattern applied to namespace.typeName.methodName ([read more](../../../docs/benchmarkdotnet.md#Filtering-the-Benchmarks)):

```cmd
dotnet run -c Release -f net11.0 --filter *Span*
dotnet run -c Release -f net11.0 --filter '*Span*'
```

To profile the benchmarked code and produce an ETW Trace file ([read more](../../../docs/benchmarkdotnet.md#Profiling)):

```cmd
dotnet run -c Release -f net11.0 --filter $YourFilter --profiler ETW
dotnet run -c Release -f net11.0 --filter '$YourFilter' --profiler ETW
```
Comment on lines 74 to 81
Here is an example command line that runs the MonoJIT RunType from a local runtime for the tests matching `*Span.IndexerBench.CoveredIndex2*`:

`python .\benchmarks_local.py --local-test-repo "<absolute path to runtime folder>/runtime" --run-types MonoJIT --filter *Span.IndexerBench.CoveredIndex2*`
`python .\benchmarks_local.py --local-test-repo "<absolute path to runtime folder>/runtime" --run-types MonoJIT --filter '*Span.IndexerBench.CoveredIndex2*'`

Here is an example command line that runs the MonoInterpreter and MonoJIT RunTypes using commits `dd079f53` and `69702c37` for the tests `*Span.IndexerBench.CoveredIndex2*` with the commits being cloned to the `--repo-storage-path` for building, it also passes `--join` to BenchmarkDotNet so all the reports from a single run will be joined into a single report:

`python .\benchmarks_local.py --commits dd079f53b95519c8398d8b0c6e796aaf7686b99a 69702c372a051580f76defc7ba899dde8fcd2723 --repo-storage-path "<absolute path to where you want to store runtime clones>" --run-types MonoInterpreter MonoJIT --filter *Span.IndexerBench.CoveredIndex2* *WriteReadAsync* --bdn-arguments="--join"`
`python .\benchmarks_local.py --commits dd079f53b95519c8398d8b0c6e796aaf7686b99a 69702c372a051580f76defc7ba899dde8fcd2723 --repo-storage-path "<absolute path to where you want to store runtime clones>" --run-types MonoInterpreter MonoJIT --filter '*Span.IndexerBench.CoveredIndex2*' '*WriteReadAsync*' --bdn-arguments="--join"`


```cmd
dotnet run -c Release -f netcoreapp3.1 --filter *.Reverse --profiler ETW
dotnet run -c Release -f netcoreapp3.1 --filter '*.Reverse' --profiler ETW
Comment on lines 97 to 99
```cmd
crank --config .\helix.yml --scenario micro --profile win-x64 --variable bdnArgs="--filter *Linq*" --profile msft-internal --variable buildNumber="myalias-20230811.1"
crank --config .\helix.yml --scenario micro --profile win-x64 --variable bdnArgs="--filter '*Linq*'" --profile msft-internal --variable buildNumber="myalias-20230811.1"
```
Comment on lines 118 to +127
#### Filtering the Benchmarks

You can filter the benchmarks using `--filter $globPattern` console line argument. The filter is **case insensitive**.
You can filter the benchmarks using `--filter '$globPattern'` console line argument. The filter is **case insensitive**.

The glob patterns are applied to full benchmark name: namespace.typeName.methodName. Examples (all in the `src\benchmarks\micro` folder):

- Run all the benchmarks from BenchmarksGame namespace:

```cmd
dotnet run -c Release -f net9.0 --filter BenchmarksGame*
dotnet run -c Release -f net9.0 --filter 'BenchmarksGame*'

```cmd
dotnet run -c Release -f net11.0 --filter System.Memory*
dotnet run -c Release -f net11.0 --filter 'System.Memory*'
@xtqqczze
Copy link
Copy Markdown
Contributor Author

@copilot apply changes based on the comments in this thread

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants