Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
69 commits
Select commit Hold shift + click to select a range
727782e
Add iOSInnerLoopParser, wire Startup.cs, fix Reporter.cs
davidnguyen-tech Apr 3, 2026
ee23cf0
Add shared Python infrastructure for iOS inner loop
davidnguyen-tech Apr 3, 2026
664f061
Add mauiiosinnerloop scenario directory (pre/test/post.py)
davidnguyen-tech Apr 3, 2026
e20e8a4
Add setup_helix.py for iOS Helix bootstrap
davidnguyen-tech Apr 3, 2026
cb6315e
Add maui_scenarios_ios_innerloop.proj Helix workitem definition
davidnguyen-tech Apr 3, 2026
7c9e0c6
Wire iOS inner loop into CI pipeline YAML and routing
davidnguyen-tech Apr 3, 2026
0d86772
Add physical device support for iOS inner loop
davidnguyen-tech Apr 3, 2026
edab33f
Fix review findings: devicectl commands, UDID regex, edit paths, Xcod…
davidnguyen-tech Apr 3, 2026
9942170
Quality fixes: tempfile.mkstemp, devicectl fallback, report guard
davidnguyen-tech Apr 3, 2026
d21e9b7
[TEMP] Disable all pipeline jobs except iOS Inner Loop
davidnguyen-tech Apr 3, 2026
25a2846
Fix iOS Inner Loop build output capture and logging
davidnguyen-tech Apr 3, 2026
53e647e
Capture and print dotnet build output for iOS Inner Loop
davidnguyen-tech Apr 3, 2026
bb1e6d2
Increase iOS Inner Loop Helix timeout to 2.5 hours
davidnguyen-tech Apr 3, 2026
565f8eb
Bypass Xcode version validation for iOS Inner Loop
davidnguyen-tech Apr 3, 2026
60d8954
Fix iOS simulator RID for Intel x64 Helix machines
davidnguyen-tech Apr 3, 2026
259c10e
Fix FileExistsError when copying traces on incremental iterations
davidnguyen-tech Apr 3, 2026
f62020a
Remove verbose build output from iOS Inner Loop
davidnguyen-tech Apr 3, 2026
62aecd2
Fix ILLink error on Helix machines with older Xcode
davidnguyen-tech Apr 3, 2026
68d6b4c
Re-enable all pipeline jobs after iOS inner loop CI validation
davidnguyen-tech Apr 3, 2026
ba4e0d1
Add install timing, CoreCLR config, and device support for iOS inner …
davidnguyen-tech Apr 3, 2026
60f1106
Add fallback for workload install version skew in pre.py
davidnguyen-tech Apr 3, 2026
995fe94
Add workload install fallback to setup_helix.py
davidnguyen-tech Apr 3, 2026
ae4c34c
Exclude simulator work item from device jobs
davidnguyen-tech Apr 3, 2026
3fbe535
Clean up iOS inner loop: unify ioshelper API, simplify runner and pos…
davidnguyen-tech Apr 4, 2026
92a538a
Fix device job failures: iOSRid propagation and setup_helix exit hand…
davidnguyen-tech Apr 4, 2026
5c092e4
Pass iOSRid as explicit MSBuild property to fix device job dispatch
davidnguyen-tech Apr 4, 2026
edc2ccb
Rename simulator job identifiers for clarity
davidnguyen-tech Apr 4, 2026
d8b4ff1
Switch ioshelper.py from simctl/devicectl to mlaunch for install/launch
davidnguyen-tech Apr 4, 2026
c398150
Use --installsim for granular simulator install timing
davidnguyen-tech Apr 4, 2026
8e89a1f
Re-select Xcode to match iOS SDK's required version after workload in…
davidnguyen-tech Apr 5, 2026
5d80e1f
Resolve booted simulator UDID for mlaunch and remove ValidateXcodeVer…
davidnguyen-tech Apr 5, 2026
6756d03
Run --launchsim as background process to prevent timeout
davidnguyen-tech Apr 5, 2026
803ec21
Add post-build code signing for physical device deployment
davidnguyen-tech Apr 6, 2026
2bcbb64
Make Xcode handling diagnostic-only on Helix — match Device Startup s…
davidnguyen-tech Apr 7, 2026
c2a65e3
Add --skip-manifest-update to workload install retry on Helix
davidnguyen-tech Apr 7, 2026
4253fcd
Resolve sign tool path for Helix device deployment
davidnguyen-tech Apr 7, 2026
bf97725
Remove /usr/local/bin from PATH - shadows venv python
davidnguyen-tech Apr 7, 2026
259573f
Restore active Xcode selection in setup_helix.py — fixes Xcode versio…
davidnguyen-tech Apr 7, 2026
6d7dfc8
Fix sign tool resolution — fall back to login shell execution
davidnguyen-tech Apr 7, 2026
e74cb12
Fix Xcode version parsing — regex greedily captured trailing dot
davidnguyen-tech Apr 7, 2026
15ed711
Use in-box manifest for workload install — avoid Xcode version skew
davidnguyen-tech Apr 7, 2026
189b555
Use in-box manifest on build agent — stop shipping incompatible SDK p…
davidnguyen-tech Apr 8, 2026
482d22e
Revert to --from-rollback-file and make Xcode diagnostic-only
davidnguyen-tech Apr 8, 2026
c8ca867
Add timeout to workload install to prevent NuGet download hangs
davidnguyen-tech Apr 9, 2026
2a6cdf7
Temporarily pin iOS workload to 26.2 for Xcode compatibility
davidnguyen-tech Apr 9, 2026
2f8fd09
Add active Xcode selection and version validation to setup_helix.py
davidnguyen-tech Apr 9, 2026
c29f631
fixup! Add active Xcode selection and version validation to setup_hel…
davidnguyen-tech Apr 9, 2026
9f4ddab
Fix Xcode selection: match workload-required version, not highest
davidnguyen-tech Apr 9, 2026
b859032
fixup! Fix Xcode selection: match workload-required version, not high…
davidnguyen-tech Apr 9, 2026
2c1d0e7
Remove temporary workload pin — restore dynamic NuGet feed resolution
davidnguyen-tech Apr 13, 2026
ece266a
Remove silent workload fallback — fail loud if nightly packs unavailable
davidnguyen-tech Apr 13, 2026
c380cbb
Add manifest patching fallback for missing net10.0 packs
davidnguyen-tech Apr 13, 2026
c64ec2f
Fix review findings: packs type handling, timeouts, diagnostics
davidnguyen-tech Apr 13, 2026
ca4f030
Document Helix manifest-patching dependency in setup_helix.py
davidnguyen-tech Apr 13, 2026
4148296
Fix iOS inner loop local device support
davidnguyen-tech Apr 13, 2026
7ca397b
Improve code quality: deduplicate workload install, fix correctness b…
davidnguyen-tech Apr 13, 2026
28f94ac
Handle missing sign tool gracefully for local device deployment
davidnguyen-tech Apr 13, 2026
afe6351
Add run-local.sh for local iOS inner loop measurements
davidnguyen-tech Apr 14, 2026
625c595
Add SDK-aware Xcode selection for local measurements
davidnguyen-tech Apr 14, 2026
5882cfa
Preserve binlogs in results directory
davidnguyen-tech Apr 14, 2026
b3f8ed6
Copy versions.json to results directory
davidnguyen-tech Apr 14, 2026
e2657fb
Uninstall stale app from physical device before first deploy
davidnguyen-tech Apr 14, 2026
c3d1ab6
Clean stale app/traces dirs before setup to fix XAML duplicate errors
davidnguyen-tech Apr 14, 2026
a0d7d63
Validate cold startup measurement before including in results
davidnguyen-tech Apr 14, 2026
321dcb9
Track ILLink, CompileNativeCode, R2R targets in iOS parser
davidnguyen-tech Apr 14, 2026
bb15078
Fix correctness bugs found during code review
davidnguyen-tech Apr 14, 2026
78cb6b6
Measure .app bundle size in iOS inner loop results
davidnguyen-tech Apr 14, 2026
e2f7b89
Fix watchdog event ordering and find_app_bundle simulator/device pref…
davidnguyen-tech Apr 14, 2026
5844d6a
Add restore timeout to prevent hung NuGet restores from consuming Hel…
davidnguyen-tech Apr 14, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
111 changes: 111 additions & 0 deletions eng/performance/maui_scenarios_ios_innerloop.proj
Original file line number Diff line number Diff line change
@@ -0,0 +1,111 @@
<Project Sdk="Microsoft.DotNet.Helix.Sdk" DefaultTargets="Test">

<Import Project="Scenarios.Common.props" />

<!-- No XHarness needed — iOS inner loop uses xcrun simctl directly for
simulator deploy and does not use the XHarness test infrastructure. -->
Comment on lines +5 to +6
Copy link

Copilot AI Apr 14, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comment says the inner loop scenario uses xcrun simctl directly for simulator deploy, but the implementation uses mlaunch --installsim/--launchsim for deploy/launch (with simctl mainly for boot/terminate/uninstall). Please update the comment to match the actual tooling so future maintainers aren’t misled about how deploy is performed.

Suggested change
<!-- No XHarness needed — iOS inner loop uses xcrun simctl directly for
simulator deploy and does not use the XHarness test infrastructure. -->
<!-- No XHarness needed — iOS inner loop uses mlaunch
(--installsim/--launchsim) for simulator deploy/launch, with xcrun
simctl used for simulator lifecycle tasks such as boot, terminate,
and uninstall. -->

Copilot uses AI. Check for mistakes.

<PropertyGroup>
<!-- Map RuntimeFlavor to the UseMonoRuntime MSBuild property.
Same pattern as maui_scenarios_ios.proj. -->
<_MSBuildArgs Condition="'$(RuntimeFlavor)' == 'mono'">/p:UseMonoRuntime=true</_MSBuildArgs>
<_MSBuildArgs Condition="'$(RuntimeFlavor)' == 'coreclr'">/p:UseMonoRuntime=false</_MSBuildArgs>

<!-- RuntimeIdentifier: simulator by default. Physical device (ios-arm64)
can be set via pipeline parameter when device support is enabled.
Default is iossimulator-x64 because the Mac.iPhone.17.Perf Helix
queue runs on Intel x64 machines (e.g. DNCENGMAC045). -->
<iOSRid Condition="'$(iOSRid)' == ''">iossimulator-x64</iOSRid>
<_MSBuildArgs>$(_MSBuildArgs) /p:RuntimeIdentifier=$(iOSRid)</_MSBuildArgs>
<!-- Device builds: disable automatic code signing during 'dotnet build'.
Signing is handled post-build by ioshelper.sign_app_for_device() using
the provisioning profile and 'sign' tool pre-installed on the Helix
machines (same flow as maui_scenarios_ios.proj device startup). -->
<_MSBuildArgs Condition="'$(iOSRid)' == 'ios-arm64'">$(_MSBuildArgs) /p:EnableCodeSigning=false</_MSBuildArgs>
<!-- Disable managed linker for inner loop builds. Debug configuration
doesn't need linking, and it avoids ILLink errors (MT0180) on Helix
machines with older Xcode versions that lack the required SDK headers. -->
<_MSBuildArgs>$(_MSBuildArgs) /p:MtouchLink=None</_MSBuildArgs>

<RunConfigsString>$(RuntimeFlavor)_Default</RunConfigsString>
<InnerLoopIterations Condition="'$(InnerLoopIterations)' == ''">3</InnerLoopIterations>
</PropertyGroup>

<!-- Remove .NET SDK packs from the Helix correlation payload to reduce
transfer size. The maui-ios workload packs (~1-2 GB) are installed on
the Helix machine by setup_helix.py, so shipping them is unnecessary
and would exceed Helix payload limits. -->
<Target Name="RemoveDotnetFromCorrelationStaging" BeforeTargets="BeforeTest">
<Message Text="Removing Dotnet Packs from Correlation Staging" Importance="high" />
<RemoveDir Directories="$(CorrelationPayloadDirectory)dotnet/packs" />
</Target>

<ItemDefinitionGroup>
<HelixWorkItem>
<!-- iOS builds are slower than Android (AOT compilation, code signing,
native linking) and the workitem also includes workload install +
NuGet restore on the Helix machine. -->
<Timeout>02:30</Timeout>
</HelixWorkItem>
</ItemDefinitionGroup>

<ItemGroup>
<MAUIiOSInnerLoopScenario Include="MAUI iOS Inner Loop">
<ScenarioDirectoryName>mauiiosinnerloop</ScenarioDirectoryName>
<PayloadDirectory>$(ScenariosDir)%(ScenarioDirectoryName)</PayloadDirectory>
</MAUIiOSInnerLoopScenario>
</ItemGroup>

<!-- PreparePayloadWorkItem: runs on the build agent (macOS).
Selects the highest versioned Xcode, then runs pre.py to create the MAUI
app template, strip non-iOS TFMs, and prepare modified source files for
the incremental deploy simulation. NuGet restore is NOT done here —
packages are restored on the Helix machine by setup_helix.py. -->
<ItemGroup>
<PreparePayloadWorkItem Include="@(MAUIiOSInnerLoopScenario)">
<!-- Select highest versioned Xcode_*.app and avoid runner-image symlink
aliases that do not work with the iOS SDK (same pattern as
maui_scenarios_ios.proj PreparePayloadWorkItem). -->
<Command>XCODE_PATH=`find /Applications -maxdepth 1 -type d -name 'Xcode_*.app' | sort -t_ -k2 -V | tail -1` &amp;&amp; echo &quot;Selected Xcode: $XCODE_PATH&quot; &amp;&amp; sudo xcode-select -s &quot;$XCODE_PATH&quot; &amp;&amp; $(Python) pre.py default -f $(PERFLAB_Framework)-ios</Command>
<WorkingDirectory>%(PreparePayloadWorkItem.PayloadDirectory)</WorkingDirectory>
</PreparePayloadWorkItem>
</ItemGroup>

<!-- Environment variables for the macOS Helix machine.
Must be set via shell 'export' in PreCommands because env vars set via
os.environ in a Python script (setup_helix.py) do not persist to
subsequent commands (test.py, post.py) in the Helix shell session. -->
<PropertyGroup>
<_MacEnvVars>export DOTNET_ROOT=$HELIX_CORRELATION_PAYLOAD/dotnet;export DOTNET_CLI_TELEMETRY_OPTOUT=1;export DOTNET_MULTILEVEL_LOOKUP=0;export NUGET_PACKAGES=$HELIX_WORKITEM_ROOT/.packages;export PATH=$HELIX_CORRELATION_PAYLOAD/dotnet:$PATH</_MacEnvVars>
</PropertyGroup>

<!-- Simulator workitem: only when NOT targeting physical devices.
When iOSRid=ios-arm64, _MSBuildArgs bakes in the device RID which
would produce an ARM64 binary incompatible with the simulator. -->
<ItemGroup Condition="'$(iOSRid)' != 'ios-arm64'">
<HelixWorkItem Include="@(MAUIiOSInnerLoopScenario -> 'Inner Loop Simulator - %(Identity)')">
<PreCommands>$(_MacEnvVars);export IOS_RID=$(iOSRid);$(Python) setup_helix.py $(PERFLAB_Framework)-ios &quot;$(_MSBuildArgs)&quot; || exit $?</PreCommands>
<Command>$(Python) test.py iosinnerloop --csproj-path app/MauiiOSInnerLoop.csproj --edit-src &quot;src/MainPage.xaml.cs;src/MainPage.xaml&quot; --edit-dest &quot;app/Pages/MainPage.xaml.cs;app/Pages/MainPage.xaml&quot; --bundle-id com.companyname.mauiiosinnerloop -f $(PERFLAB_Framework)-ios -c Debug --msbuild-args &quot;$(_MSBuildArgs)&quot; --device-type simulator --inner-loop-iterations $(InnerLoopIterations) --scenario-name &quot;%(Identity)&quot; $(ScenarioArgs)</Command>
<PostCommands>export IOS_RID=$(iOSRid);$(Python) post.py</PostCommands>
<DownloadFilesFromResults>output.log</DownloadFilesFromResults>
</HelixWorkItem>
</ItemGroup>

<!-- Physical device workitem: runs on macOS machines with USB-connected
iPhones (Mac.iPhone.17.Perf queue).
Code signing is disabled during 'dotnet build' (EnableCodeSigning=false)
and handled post-build: ioshelper copies embedded.mobileprovision into
the .app and runs the Helix-provided 'sign' tool — same flow as
maui_scenarios_ios.proj device startup scenarios. -->
<ItemGroup Condition="'$(iOSRid)' == 'ios-arm64'">
<HelixWorkItem Include="@(MAUIiOSInnerLoopScenario -> 'Inner Loop Device - %(Identity)')">
<PreCommands>$(_MacEnvVars);export IOS_RID=$(iOSRid);$(Python) setup_helix.py $(PERFLAB_Framework)-ios &quot;$(_MSBuildArgs)&quot; || exit $?</PreCommands>
<Command>$(Python) test.py iosinnerloop --csproj-path app/MauiiOSInnerLoop.csproj --edit-src &quot;src/MainPage.xaml.cs;src/MainPage.xaml&quot; --edit-dest &quot;app/Pages/MainPage.xaml.cs;app/Pages/MainPage.xaml&quot; --bundle-id com.companyname.mauiiosinnerloop -f $(PERFLAB_Framework)-ios -c Debug --msbuild-args &quot;$(_MSBuildArgs)&quot; --device-type device --inner-loop-iterations $(InnerLoopIterations) --scenario-name &quot;%(Identity)&quot; $(ScenarioArgs)</Command>
<PostCommands>export IOS_RID=$(iOSRid);$(Python) post.py</PostCommands>
<DownloadFilesFromResults>output.log</DownloadFilesFromResults>
</HelixWorkItem>
</ItemGroup>

<Import Project="PreparePayloadWorkItems.targets" />

</Project>
Loading