lt72/jitbench

PRIVATE repo for testing JIT stuff

jitbench's Issues

Show customer events values in lifetime stats

Loading assembly base in dynamic assembly loading

Currently dynamically loading assemblies broke the build as it finds the report base assembly with the error

'Could not load type 'Microsoft.ETWLogAnalyzer.Abstractions.ReportBase' from assembly 'Microsoft.ETWLogAnalyzer.Abstractions

Rename namespaces

The namespaces are called MusicStore. The tool should be a bit more generic

Revise possible events handlers to use in the ETL file parsers

Review the list of possible different actions in Microsoft.Diagnostics.Tracing.Parsers.KernelTraceEventParser and Microsoft.Diagnostics.Tracing.Parsers.ClrTraceEventParser that might be useful for the generation of the matrix.

JitBench build broken

[fix] ASP.NET binaries are being used

Currently the fix branch uses ASP.NET binaries and not the crossgen libraries with the following warning

ASP.NET loaded from bin. This is a bug if you wanted crossgen

This might be a problem if our aim is to make it JIT as much as possible. I suspect the binaries are most likely optimized.

[fix] Duplicate method jitting

Review duplicate method generation; check if it's a bug (i.e. race condition) or if it's an optimistic way to handle race conditions in the JIT.

A perfview dump generated that exemplifies such behavior can be found at:
PerfViewData-5-25-2017-17-15.zip

Overhead from filter using list of int for relevant threads

Local package usage for TraceEvent

We had to modify TraceEvent to surface the ReJit version to add support for tiered jitting. While the change has been merged on the PerfView repo - where TraceEvent lives - the package won't get updated on NuGet until it gets signed.

To work around this a Nuget.config file points to a package with the updated library locally. These should be removed when TraceEvent gets updated on NuGet to version 1.0.42.

Defunct might be a better measure of process finished than stop

The main thread gets switched out after the stop. Defunct marks a point were there's no more process activity.

API doesn't protect timelines against changes

The API returns iterators to the actual elements. Try to see if it's possible to make this safe.

Make Controller.LoadReports private

Report API needs revision

There should be no need for two different API calls. One would allow for less memory consumption (no need to cache, just persist on the run) and better error reporting.

Open overall event interface

Figure how to query the ASP.NET version to use JitBench

Refactor automated scripts to new architecture

PFUtil in Tools is architecture dependent

Must find a way to either fetch it on the run, provide instructions on where to find it

Research quantum length information retrieval

Replace int usage with struct ThreadId for disambiguation

Consider necessary context and replace the lookups for thread based on the struct as threads that are pooled can be reused.

Move report writers out of abstractions

Frozen JitBench Version

Given that JitBench keeps breaking, I've decided to freeze the version of dotnet and JitBench until the testing of fresh CoreCLR/CoreFX and ASP.NET releases becomes priority 0.

Handle how to map the data for BenchView

A lot of data is not relevant and a whole bunch of other data points are not relevant to our (we don't want to generate a visualization per method for example.

[fix] Make the scripts path agnostic

To future proof the current data collection scripts it might be a good idea to clean them up to be path agnostic. This might prove to be useful for future parts of the project.

[fix] Unfreeze preview version to enable most recent build

Given that our current priority is generating a log that jits as much as possible and its collection is automated, the fix branch achieved not performing cross-gen by freezing the versions fetched. However, given that we want automation and it should fetch the latest version of ASP.NET and the CoreCLR, we should find a way to make the libraries fetched from the master channel available to PerfView for the log generation.

Consider reference set for worst case scenario analysis

[fix] Document script usage and dependencies.

Check how to match IO calls for the filter

Some things related to the process are not attributed so might have to add them manually in several passes. See how this can be done.

The current implementation hardcodes providers

Make it a command-line argument of comma separated values.

The AvailableQuantumAccumulatorVisitor uses a placeholder QuantumLength

IO Attribution

Look into how to attribute IO events relevant to dotnet that are not identified to any thread or process we have logged.

Dynamic Filter for customer providers

Determine how to return a filter for the provider events as the DynamicManifestTraceEvent type is internal to the tracing assembly/module.

Unique identifier for methods clash in ETW log

The ETW log collected with pfutil creates a dictionary key clash. Need a really unique identifier.

Fix thread lifetimes

The last event is not necessarily the threadStop

DefaultResult is potentially a bug

Currently we are checking against DefaultResult to check if a result was generated, but for numberical values the default of 0 might be a perfectly reasonable answer

Consider Replacing event time disambiguation with ListWrapper

Clash of events don't happen often enough, so the overhead might not be worth it. Analyze if the overhead and development time are worth pursuing.

Automate whole process of data collection

This should be simple. The only thing missing is the NuGet package restore to be able to run the analysis without opening visual studio. This should be straightforward using MSBuild.

Perfview warms up some of the dll's

Consider wrapping in another script.

Make controller log the report status

Metric Relaxation assumptions

• Quantum efficiency is being calculated as CS_out - CS_in for the active interval times
• Lifetime is taken as Last CS_out - Thread start

Fix dependency warning seen in the package restore

Freezing packages generates warnings. Check this out

Idle time analysis filter missing