jeremymain/gpuprofiler

GPUProfiler - Understand your application and workflow resource requirements

gpuprofiler's Issues

Multi GPU display support for workstation use

Currently capturing all GPUs utilization information but only displaying the first detected GPU.

Add GPU clock rate to advanced display settings

GPU utilization is a relative value in relation to the GPU clock. Correlating the GPU clock with the reported utilization for instances where intermittent non-sustained GPU work does not trigger the boost clock and could be misinterpreted as a requirement for more GPU resource than is actually required for the workload.

GPU BUS ID information

Fred's request:

Add the GPU's BUS ID information to the system state information captured and saved in the GPD file

Display data tooltip when selecting graph data

Looking at graphs is interesting but when you need to know the actual values clicking on the line data will display a tooltip with the utilization and value (where supported).

Add a new resource graph display mode: One graph per resource

By double clicking on the graph, change to
Default mode => minimal view => separate graphs for each resource

Retain last modified sample interval and duration when starting a new profile run

Doing a large number of concurrent VMs in various configurations it became clear that automatically reverting to the default value is not terribly convenient.

Graph only view

double click on the graph output and the window will only display the graph output.
Double clicking again will return the display to the standard view

VRAM clock information

Fred's request:
Capture VRAM clock information

Make the resource plots less visually "busy" for long profile runs

Plotting each and every collected datapoint results in a very busy plot

but I have been experimenting with some improvements to better visualize the overall utilization within the bounds of the visible region.

Sample interval - invalid range detection

When entering an invalid range the UI does not enforce a valid range setting leading to an application exception error.

histogram handling of not utilized (0) and fully utilized (100)

I had noticed that the histogram calculation was not including the 100% utilized values in the 90 ~ 99% bar graph.

Additionally, the 0 values for a resource will not be added to the histogram bucket.
When a resource is not being used rather than showing the 0~10 as 100% probability it will now display nothing as no utilization data exists.

Before: v1.02 ~ v1.03

After:

Compare the CPU utilization (issue with 100%) in the histogram and the GPU utilization as well as the encoder/decoder utilization for the improved 0% handling

Add timestamp collection at profile start and add to output (GPD/CSV) resource utilization inspector

When profiling for long periods of time, users may encounter periods where they notice some performance difference in their normal application usage that may wish to correlate with the data collected during the profiling run.

Having the ability to show in the graph the actual time the data was sampled at would simplify pinpointing when the event occurred.

Adding the label insertion support via hot-key would also be a useful addition to support this end-user assisted workload profiling.

Hide histograms of resources that have no data, not supported or 0 values only

Hide Analysis histograms where no data is available.

In this example both have no non-zero utilization data and should not be displayed until data exists. This applies for all resource utilization data.

[feature request] CLI scriptable versions

Windows and Linux.
This way they can be started and managed administratively via GPOs
With possibility as light weight (no graphical output) persistent daemons.

Add histogram resizing support

Allow the histogram when displayed to be horizontally resized

Multi-GPU profiling and selective display

Fred's request:

Multi-GPU support and selection.
With more than 1 GPU, I must be able to pick the 1 and want to analyze.

In-graph utilization inspector - Not calculating text scaling correctly

For those using text scaling, the utilization inspector is not calculating the vertical spacing between the visible utilization elements correctly.

Remoting protocol delivered frame rate / current framerate

Finding a non-WMI method to capture this information has been elusive.

advanced GPU selection and informations

hello Jeremy, thanks A LOT to build a nice GUI instead of the sh** nvidia-smi

now a lot of options i need are not there:

• multi-GPU support and selection. with more than 1 GPU, i must be able to pick the 1 and want to analysis
• missing GPU infos:

• bus-ID
• driver model : WDDM version or TCC
• GPU boost : enable or disable
• GPU clock : real clock (if boost is enable, then always the max)
• Vram clock
• [don't know if this available ]: a SEPARATE %usage of GPU and Vram... in nvidia-smi both are reported into the single GPU%, so you don't know which one is starving first...

regards,
fred

Remoting protocol detection

Add the current remoting protocol to the GPD data file

Check for GPD file association on first start and ask to register GPUProfiler as the default application

GPUProfiler supports drag-and-drop of GPD files, adding the file association will make viewing previously captured data simpler.

Add GPU Clock information

Fred's request:

GPU clock : real clock (if boost is enable, then always the max)

Make resource utilization histograms more readable

For diverse utilization over a profile period, the resulting histogram scale of 0 ~ 100 is difficult to visually determine the differences between the different values.

Zoom-full after profile stop

Prior versions displayed the entire intended data graph even if the data collection process was stopped early. Now the graph will display the entire graph data on early stop or if the view is zoomed during collection.

Remoting protocol setting detection

Detect the protocol specific settings and store those settings in the GPD file.

Save option state values

Save the current limited number of preferences and reload on next launch.

Option to temporarily bold display graph lines

By using a keyboard accelerator display the lines thicker to aid in situations where fine-details may be lost (during presentations, using a projector, etc.)

candidate would be the 'B' key.

Question: should there be two or three thickness levels ?

Add Memory controller, Bus utilization information

Fred's request:

SEPARATE %usage of GPU and Vram... in nvidia-smi both are reported into the single GPU%, so you don't know which one is starving first...

JJM:
[ nvidia-smi -q ] does list the various utilization data for SM, memory controller, bus, encoder and decoder.

Add video encode / decode utilization metrics (where supported)

To better understand the relationship between GPU (SM) and the video encode/decode engines I will add these metrics. Only GPUs that are supported by NVML will allow this data to be captured.

GPUProfiler_1.04-x64 Crashes on W7x64 with Quadro K2200 (353.82)

GPUProfiler Version: GPUProfiler_1.04-x64

Type: Physical
OS: W7x64 Professional SP1 / German UI
CPU: Intel Xenon E31281v3 @3.70 GHz
RAM: 16 GB

How can we provide more Information to you?
What do you need?

Best Regards

GPU Boost state

Fred's request
Record at profile time of GPU boost is enabled and save within the GPD file

Alternative display "Dark Mode"

adding an alternative display mode with a darker color palette

The biggest challenge for completing this is simply getting the Win32 controls to adhere to the new palette.

Current driver model - WDDM | TCC

Fred's request:
report and store in the GPD file the current (at profile time) the driver model / version:
WDDM version or TCC

prioritize use of NVML over NVAPI where supported

To enable more detailed performance metrics, I will prioritize the use of the NVML API over NVAPI.
There is a limitation in NVML that it only supports x64 build only, therefore the x86 build will lack the ability to use NVML.
Viewing .GPD files will be unaffected by this limitation

List all Labels via list-box, auto zoom to label range

In a list box display all of the labels, their duration and some metrics about each period.

Network utilization option

I have had multiple requests to add network (send/recieve) utilization
Adding it here for tracking.

Time axis zoom via CTRL+mouse wheel with mouse position weighting

Previously when zooming via the mouse wheel, the zoom area was weighting the range equally without respect to the current mouse position. This detracts from the tool usability during the analysis phase but fixed in the next release.

Toggle "Legend" displayed | hidden

Selecting the "L" key, the legend can be displayed or hidden. Useful when being used in the minimal view mode.

Normal:

Legend hidden:

VM agent version

When used within a VM, capture the agent version information and save within the GPD file.

Graph data display state during display mode changes and operational mode changes

Confirm all of the states are correctly reset on particular operations

• Open
• New

However when a file is dropped into the application, retain the current display options

Insert/remove label(s) via mouse operations

Post profiling segmentation of the workload events to classify resource signatures for the operations.

Option to minimize the Window?

Can you please add a minimize button for the Window?

Resource three-state plot display state [ normal | bold | hidden ]

Using the key for each of the resources, toggle between normal, bold and hidden.
This can be used to draw attention to a single or set of utilization data. This is independent of the overall three state "bold" of all profile data

Documentation -> Update wiki information

In case you haven't figured out how to use the application, the wiki will provide some more information about how I use the tool.

Document GPD file format

Define what information is collected and saved in a GPD file.
This would be useful for users that may wish to share profile data but are reluctant to due to not knowing the scope of the collected data.

Insert label during profiling via hotkey

When profiling is being performed, using a global hot key to pop-up a dialog to capture the label, (Ex: "Model load start") and insert that label into the graph output.
Storing of the label data was planned and part of the .GPD file format

Add command line option to pass a label command to the active profiling session

When using GPUProfiler with a batch file, allow calling the GPUProfiler executable with a command line option to simply add a user defined label to the profile timeline during profiling.

This will be useful when using batch files to automate testing of different configurations.
Because it will be a simple command I could either create a small EXE to perform this or add this to the main application.

This is not an implementation sample, just a mockup to illustrate the feature.

Add "monitor" mode

When the tool is being used to simply monitor for demo purposes or for performing an initial investigation where the entire sample term data is not intended to being saved, the mode would allow endless monitoring of the resource states.
When the monitor mode is stopped, an option to save the data would be possible within the current visible range.

Remove requirement to install VC2010 redistibutables (via static linking)

Most people want to use the tool as a portable application, without an installer.
Shifting to static library linking will increase the application size to 2.5MB

Resource utilization probability analysis

Add four bar graphs to show the resource utilization probability