Comments (11)
I have checked on our automated tests in PerfStar and I do not notice any regression without the analyzers on.
As for the run of tests with analyzers, I see insignificant regression (0.4% with previous CI run and 1.4% with 30-days mean) which is on the top edge of the possible variance.
I would say it is an acceptable overhead for the new capabilities. However, not many analyzers are implemented, so it is expected that the overhead would not be high at this moment of development.
TODO: run manual perf tests on bigger repositories.
from msbuild.
Oh - that is almost 14% :-O
Do we have any mid or larger size solution in the batter of repos in PerfStar? It's be nice to be able to catch those slowdowns autmatically.
As for now - we'll likely need a measurement of OC build
- with
MSBUILDLOGPROPERTIESANDITEMSAFTEREVALUATION
env variable set system wide to "1" (this is required by the analyzers currently to provide data for analysis). - with the sole analyzer turned off -
IsEnabled
set tofalse
. So that we can disect the impact of infra and the analyzer.
@AR-May - can you create items and handle those?
It'd be good to test on a version without editorconfig changes (#9811) merged - as that will bring yet another big variable to the perf equation.
It might be interesting to test with #10016 (once it's finlized) - as it'd get the tracing a good test-drive (with ability to compare the results reported by TracingModule and actual 'wall clock time measurements') - FYI @maridematte (you might want to team up with @AR-May on this)
from msbuild.
Do we have any mid or larger size solution in the batter of repos in PerfStar? It's be nice to be able to catch those slowdowns automatically.
They are currently broken; we have an issue to fix them: https://devdiv.visualstudio.com/DevDiv/_workitems/edit/1896972
I will create items and investigate further the source of this perf issue. I agree that's a big difference.
It'd be good to test on a version without editorconfig changes (#9811) merged - as that will bring yet another big variable to the perf equation.
editorconfig changes are not yet merged, so I tested without them.
from msbuild.
FYI @baronfel
from msbuild.
This is an encouraging result! Can you link to the perfstar runs/charts here for posterity?
from msbuild.
from msbuild.
@JanKrivanek checked current main: ran msbuild on OC and console app solutions with and without analyzers.
OC rebuild | with analyzers | without analyzers |
---|---|---|
median | 85053 | 73716 |
mean | 88154 | 77364 |
console app rebuild | with analyzers | without analyzers |
median | 8982 | 8912 |
mean | 8969 | 8915 |
For small solutions overhead is small, but for bigger one it is quite noticeable.
from msbuild.
with MSBUILDLOGPROPERTIESANDITEMSAFTEREVALUATION env variable set system wide to "1"
Done, here it is. Does not explain the difference in ~10 seconds of the build time.
Time Elapsed 00:01:32.61
MSBuild.SharedOutputPathAnalyzer: 00:00:00.0213450
from msbuild.
with the sole analyzer turned off
I turned off the analyzer and ran again with /analyze
and without. Also ran a baseline with commit fc97b2d
Results:
OC rebuild | with analyze | without analyze | baseline |
---|---|---|---|
median | 81445 | 71151 | 72182 |
mean | 82251.14286 | 73447.57143 | 71997 |
It seems like most of the perf overhead lays in the infra part and not in the only analyzer.
The baseline and run without /analyze
are same, difference is withing the variance.
from msbuild.
With @maridematte changes:
Build succeeded.
0 Warning(s)
0 Error(s)
Time Elapsed 00:01:30.07
BuildCheck run times
Infrastructure run times
analyzerAcquisitionTime,00:00:00
analyzerSetDataSourceTime,00:00:00.0000075
newProjectAnalyzersTime,00:00:00.0001385
Analyzer run times
MSBuild.SharedOutputPathAnalyzer,00:00:00.0153347
from msbuild.
Closing this issue - we measured the perf.
Created a new one to investigate (and hopefully fix) the perf issue.
from msbuild.
Related Issues (20)
- [Bug]: dotnet restore throws error NU1202 with dotnet sdk 8.0.300 for some centrally managed packages HOT 2
- [Feature Request]: Expand $(~) to $(HOME) or $(USERPROFILE) depending on OS HOT 4
- Got 'Insufficient system resources exist to complete the requested service' while building Store version of application
- [Feature Request]: Cache RoslynCodeTaskFactory compilations for the lifetime of the Server
- [Bug]: Cannot use Microsoft.Build.Utilities.Core 17.10.4 NuGet package HOT 4
- Correct way to generate sources for an external project (with incompatible `TargetFramework`)?
- [Feature Request]: Localize MSBuild messages with MSBuildInternalMessage task
- [Bug]: BuildCheck: Internal error thrown when reporting results HOT 1
- [Broken Build]: Blazor Integrity issue HOT 1
- [Bug]: NRE in at Microsoft.Build.BackEnd.TargetEntry.<ExecuteTarget>d__44.MoveNext() HOT 4
- Request to backport getResultOutputFile from 8.0.3xx to 8.0.1xx SDK. HOT 3
- [Bug]: CI/CD pipeline broken after update from 17.9 to 17.10: no Appx packages generated HOT 2
- [Bug]: OOM Exception due to infinite loop in FileMatcher HOT 2
- [Feature Request]: The way publish dll only HOT 5
- [Refactoring] Move all Xml*WithLocation to Microsoft.Build.Framework
- [Feature Request]: Reduce the wrapping around SDK Resolver messages when only one resolver is actually invoked HOT 1
- Analyzer for Property read before assigned HOT 2
- [Bug]: Incorrect framework library is used sometimes during build HOT 1
- Q: Need to know BumpAssemblyVersions targets integration, package versions migration HOT 24
- [Bug]: Project file does not exist with LongPaths HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from msbuild.