nus-comparch/looppoint

Sampled simulation of multi-threaded applications using LoopPoint methodology

looppoint's Issues

I received an error while executing the command "./run-looppoint.py -p demo-matrix-1 -n 8 --force"

Executing Python script /home/kbtx/looppoint/results/demo-matrix-1-test-passive-8-20220711012007/simulation/wp/sim.scripts.py
[SNIPER] --------------------------------------------------------------------------------
[SNIPER] Sniper using SIFT/trace-driven frontend
[SNIPER] Running full application in DETAILED mode
[SNIPER] --------------------------------------------------------------------------------
[SNIPER] Enabling performance models
[SNIPER] Setting instrumentation mode to DETAILED
[RECORD-TRACE] Using the Pin frontend (sift/recorder)
[SIFT_RECORDER] Running ~/looppoint/tools/sniper/pin_kit/pin -mt -injection child -xyzzy -ifeellucky -follow_execv 1  -t ~/looppoint/tools/sniper/sift/recorder/obj-intel64/sift_recorder -verbose 1 -debug 0 -roi 0 -roi-mpi 0 -f 0 -d 0 -b 0 -o /tmp/tmpKuPmdE/run_benchmarks -e 1 -s 0 -r 1 -pa 0 -rtntrace 1 -stop 0  -sniper:flow 1000  -- ./base.exe 100 8
**NFE: unknown option "-sniper:flow"**
**Error, invalid parameters**

It seems that -sniper:flow shouldn't appear in the commandline, and after which, I got these following messages:

[SNIPER] WARNING: No SIFT connections made yet. Waiting...
[SNIPER] ERROR: Could not establish SIFT connection, aborting! Check benchmark-app*.log for errors.
[SNIPER] End
[SNIPER] Elapsed time: 60.22 seconds

Then, each running command got an end with "Cannot find script file simuserroi"

Running command:
/run-sniper        -n 8        -v -sprogresstrace:10000000 -gtraceinput/timeout=2000 -gscheduler/type=static -cgainestown --trace-args=-sniper:flow 1000 -ssimuserroi --roi-script --trace-args=-control start:address:0x404690:count1:global --trace-args=-control stop:address:0x408018:count6775173:global -gperf_model/fast_forward/oneipc/interval=100 -ggeneral/inst_mode_init=detailed -gperf_model/fast_forward/oneipc/include_memory_latency=true -d /home/kbtx/looppoint/results/demo-matrix-1-test-passive-8-20220711012007/simulation/r1        -- ./base.exe 100 8
[PREPROCESS] matrix-omp
[PREPROCESS] apps/demo/matrix-omp/matrix-omp
[PREPROCESS] /home/kbtx/looppoint/apps/demo/matrix-omp/matrix-omp
[PREPROCESS] symlinking matrix-omp /tmp/tmpGxsc5W/base.exe
[PREPROCESS] apps/demo/matrix-omp/test
[PREPROCESS] /home/kbtx/looppoint/apps/demo/matrix-omp/test
[PREPROCESS] symlinking /home/kbtx/looppoint/apps/demo/matrix-omp/test/matrix-omp.3.cfg /tmp/tmpGxsc5W/matrix-omp.3.cfg
[PREPROCESS] symlinking /home/kbtx/looppoint/apps/demo/matrix-omp/test/matrix-omp.1.cfg /tmp/tmpGxsc5W/matrix-omp.1.cfg
[PREPROCESS] symlinking /home/kbtx/looppoint/apps/demo/matrix-omp/test/matrix-omp.2.cfg /tmp/tmpGxsc5W/matrix-omp.2.cfg
[PREPROCESS] Done
Cannot find script file simuserroi

I wander if the script should be gengrated in the step which reports "unknown option "-sniper:flow" " , and how to fix it?

How to make docker container work?

These are the contents from the Installaion section of README.md:

Installation
Clone the repository and navigate to the base directory
$ git clone https://github.com/nus-comparch/looppoint.git
Follow the below steps to setup and build the artifact once you have the Sniper gitid
Build the docker image (Note that this is a one-time step.)
$ make build
Run the docker image
$ make
Build the demo applications
$ make apps
Download and build Sniper and LoopPoint tools
$ make tools
These steps should automatically download the required versions of SDE/Pin kit and Sniper

By execute sudo make build, I do build a docker image, and by execute sudo make, I can enter the container.
But where should I execute the following commands make apps and make tools?
If I quit the container and execute them directly on my host, it seems that the docker container didn't work. Should I enter the docker container, clone looppoint into it and then execute the last 3 commands?

Issue pulling in PIN tarball with standard Makefile

Hi @alenks,

I've been running and working with LoopPoint and enjoy the tool. However, there seems to be a change to the Intel PIN downloads that isn't picked up by the default Makefile. The download for PIN seems to hang possibly due to some embedded Javascript for the download. I was able to download the tarball using a browser addon like cliget to generate a valid URL string, but this doesn't seem amenable to use with a script.

make[1]: Leaving directory '/net/netscratch/jy/looppoint/tools/src/Drivers'                                                                                   
Downloading Pin                                                                                                                                               
--2022-11-29 13:48:02--  https://software.intel.com/sites/landingpage/pintool/downloads/pin-3.13-98189-g60a6ef199-gcc-linux.tar.gz                           
Resolving software.intel.com (software.intel.com)... 23.46.246.33, 2600:1402:b800:395::b, 2600:1402:b800:38f::b, ...                                         
Connecting to software.intel.com (software.intel.com)|23.46.246.33|:443... connected.                                                                         
HTTP request sent, awaiting response... 

This issue can be replicated outside of the container environment by running the following on any command-line (I used Ubuntu 20.04 and RHEL8):

wget --no-check-certificate https://software.intel.com/sites/landingpage/pintool/downloads/pin-3.13-98189-g60a6ef199-gcc-linux.tar.gz
--2022-11-29 17:50:22--  https://software.intel.com/sites/landingpage/pintool/downloads/pin-3.13-98189-g60a6ef199-gcc-linux.tar.gz
Resolving software.intel.com (software.intel.com)... 23.46.246.33, 2600:1402:b800:3af::b, 2600:1402:b800:395::b
Connecting to software.intel.com (software.intel.com)|23.46.246.33|:443... connected.
HTTP request sent, awaiting response...

Pin error: PRIVILEGED_INS during BBV generation

I have been profiling a program which uses the C++ threads library. The program spawns threads which perform matrix multiplication, which is protected by a mutex. Parent threads spawn children which perform the multiplication. Synchronization is achieved through the use of join().

PFA the code for the program and the cfg file for the looppoint run.
The command to compile code is:
g++ -std=c++11 -pthread contention.cpp -o contention.out

The looppoint command is:
run-looppoint.py -c contention.1.cfg -n 8 --force

The fat pinball seems to be generated successfully, but the pin tool in the SDE kit crashes during the BBV generation phase. The error is:
In:
Thread: 8
PID: 3185
SYSTEM TID: 3195
Exception code: PRIVILEGED_INS
Exception Class: 3
Exception address: 0x7fc34a122c5a
C: [tid:3195] Tool (or Pin) caused signal 11 at PC 0x7fc34a122c5a
Segmentation fault (core dumped)

Thank you for looppoint, its a super useful tool!

contention.tar.gz

Pin stack overflow occurs when running SPEC CPU2017 621.wrf_s.

Hi, thank you for the Looppoint. Recently, I've been trying to use LoopPoint to collect representative regions on SPEC CPU. Following the readme file, I successfully ran SPEC CPU2017 603.bwaves (command: ./speed_bwaves_base.icc bwaves_1 < bwaves_1.in). However, when I run SPEC CPU2017 621.wrf_s, it shows a Pin stack overflow error. Do I need to adjust some settings?

Here is my cfg file:

[Parameters]
program_name: loopiccgo2
input_name: 1
command: ./wrf_s_base.icc

The log of the exception section is as follows:

***  Finished generating whole program pinballs [log_whole]  ***    April 03, 2024 17:28:13

+++  Using whole program pinballs in dir: whole_program.1

***  TRACING: END  ***    April 03, 2024 17:28:13
Running commands:
/mnt/hdd/users/ycchang/code/performance/looppoint/tools/sde-external-9.14.0-2022-10-25-lin/pinplay-scripts/replay.py --pintool=sde-global-looppoint.so  --pintool_options -dcfg -replay:deadlock_timeout 0 -replay:strace -dcfg:out_base_name /mnt/hdd/users/ycchang/code/performance/looppoint/apps/wrf_icc_o2/custom-loopiccgo2-1-test-passive-8-20240403172006/whole_program.1/loopiccgo2.1_2882515 /mnt/hdd/users/ycchang/code/performance/looppoint/apps/wrf_icc_o2/custom-loopiccgo2-1-test-passive-8-20240403172006/whole_program.1/loopiccgo2.1_2882515

......
WRF NUMBER OF TILES =   8
......
wrf: SUCCESS COMPLETE WRF
In: 
Thread: 0
PID: 2889645
SYSTEM TID: 2889645
Exception code: ACCESS_DENIED
Exception Class: 2
Faulty AccessType : 0
Exception address: 0x14923732b008
E: Pin stack overflow in thread 2889645

It also causes subsequent tasks to fail. The complete log file is attached.

Thank you for your project, and I look forward to your response.

looppoint.log.txt

sde error：Operation not permitted

I've built the docker container successfully and then I ran the 'run_looppoint.py' as the following command:
./run-looppoint.py -n 8 -i test -p demo-matrix-1 --force --no-validate
But it failed and gave me the following error information

[LOOPPOINT] Generating fat pinball.
[PREPROCESS] matrix-omp
[PREPROCESS] apps/demo/matrix-omp/matrix-omp
[PREPROCESS] /home/username/code/looppoint-main/apps/demo/matrix-omp/matrix-omp
[PREPROCESS] symlinking matrix-omp /tmp/tmprNBpqK/base.exe
[PREPROCESS] apps/demo/matrix-omp/test
[PREPROCESS] /home/username/code/looppoint-main/apps/demo/matrix-omp/test
[PREPROCESS] symlinking /home/username/code/looppoint-main/apps/demo/matrix-omp/test/matrix-omp.1.cfg /tmp/tmprNBpqK/matrix-omp.1.cfg
[PREPROCESS] symlinking /home/username/code/looppoint-main/apps/demo/matrix-omp/test/matrix-omp.2.cfg /tmp/tmprNBpqK/matrix-omp.2.cfg
[PREPROCESS] symlinking /home/username/code/looppoint-main/apps/demo/matrix-omp/test/matrix-omp.3.cfg /tmp/tmprNBpqK/matrix-omp.3.cfg
[PREPROCESS] symlinking /home/username/code/looppoint-main/apps/demo/matrix-omp/test/custom-matrix-omp-1-test-active-8-20230731074220 /tmp/tmprNBpqK/custom-matrix-omp-1-test-active-8-20230731074220
[PREPROCESS] symlinking /home/username/code/looppoint-main/apps/demo/matrix-omp/test/matrix-omp.1.info.status /tmp/tmprNBpqK/matrix-omp.1.info.status
[PREPROCESS] symlinking /home/username/code/looppoint-main/apps/demo/matrix-omp/test/matrix-omp.1.info.cfg /tmp/tmprNBpqK/matrix-omp.1.info.cfg
[PREPROCESS] symlinking /home/username/code/looppoint-main/apps/demo/matrix-omp/test/pin-log.txt /tmp/tmprNBpqK/pin-log.txt
[PREPROCESS] Done
***  TRACING: START  ***    July 31, 2023 08:28:32
Script version $Revision:1.128$
Script:                    sde_pinpoints.py
Script args:               --delete --mode mt --sdehome=/home/username/code/looppoint-main/tools/sde-external-9.14.0-2022-10-25-lin --cfg /home/username/code/looppoint-main/apps/demo/matrix-omp/test/matrix-omp.1.cfg --log_options -start_address main -log:fat  -log:mp_atomic 0 -log:mp_mode 0 -log:strace -log:basename /home/username/code/looppoint-main/results/demo-matrix-1-test-passive-8-20230731082832/whole_program.1/matrix.1 --replay_options=-replay:strace -l
Program name:              matrix-omp
Input name:                1
Command:                   ./base.exe 100 8
Tracing mode:              mt
Logger options:            -start_address main -log:fat  -log:mp_atomic 0 -log:mp_mode 0 -log:strace -log:basename /home/username/code/looppoint-main/results/demo-matrix-1-test-passive-8-20230731082832/whole_program.1/matrix.1
Replayer options:          -replay:strace
Maxk:                      20
Warmup length:             1500
Prolog length:             0
Slice size (region):       30000000
Epilog length:             0
Dir separator:             .
WP pinball directory:      whole_program.1
Data/lit/pp directory:     matrix-omp.1
Trace file name format:    matrix-omp.1_t0rX_warmup1500_prolog0_region30000000_epilog0
Instance cfg file:         matrix-omp.1.info.cfg
Instance status file:      matrix-omp.1.info.status
Sdehome:                   /home/username/code/looppoint-main/tools/sde-external-9.14.0-2022-10-25-lin
Script path:               /home/username/code/looppoint-main/tools/sde-external-9.14.0-2022-10-25-lin/pinplay-scripts
Working dir:               /tmp/tmprNBpqK
Processor:                 Not defined, using default processor
Number cores/system:       96

PATH: /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/home/username/code/looppoint-main/tools/sde-external-9.14.0-2022-10-25-lin/pinplay-scripts:/home/username/code/looppoint-main/tools/sde-external-9.14.0-2022-10-25-lin/pinplay-scripts/PinPointsHome/Linux/bin

LD_LIBRARY_PATH: /home/username/code/looppoint-main/libs:

+++  Deleting: whole_program.1.relog.*

+++  Deleting: matrix-omp.1*.NATIVE.TIME

+++  Deleting: verify_matrix-omp.1_out.txt

+++  Deleting: matrix-omp.1.info.status

+++  Deleting: PARALLEL.PARAM

+++  Deleting: sniper_results/whole_program.1

+++  Deleting: matrix-omp.1.info.cfg

+++  Deleting: whole_program.1

***  Generating whole program pinballs [log_whole]  ***    July 31, 2023 08:28:32
sde_logger.py "--log_options=-start_address main -log:fat  -log:mp_atomic 0 -log:mp_mode 0 -log:strace -log:basename /home/username/code/looppoint-main/results/demo-matrix-1-test-passive-8-20230731082832/whole_program.1/matrix.1"  --log_file whole_program.1/matrix-omp.1  --global_file global.dat.32218 --cfg /home/username/code/looppoint-main/apps/demo/matrix-omp/test/matrix-omp.1.cfg  "./base.exe 100 8"

sde_log.py --compressed=bzip2  --log_file whole_program.1/matrix-omp.1 --log_options "-log:syminfo -log:pid -start_address main -log:fat  -log:mp_atomic 0 -log:mp_mode 0 -log:strace -log:basename /home/username/code/looppoint-main/results/demo-matrix-1-test-passive-8-20230731082832/whole_program.1/matrix.1"   --global_file global.dat.1801 --cfg /home/username/code/looppoint-main/apps/demo/matrix-omp/test/matrix-omp.1.cfg  -- "./base.exe 100 8"

/home/username/code/looppoint-main/tools/sde-external-9.14.0-2022-10-25-lin/sde -log -xyzzy  -log:mt 1 -log:compressed bzip2 -log:syminfo -log:pid -start_address main -log:fat  -log:mp_atomic 0 -log:mp_mode 0 -log:strace -log:basename /home/username/code/looppoint-main/results/demo-matrix-1-test-passive-8-20230731082832/whole_program.1/matrix.1 -- ./base.exe 100 8
E: Attach to pid 58 failed: Operation not permitted
Command terminated by signal 9
0.00user 0.01system 0:00.01elapsed 94%CPU (0avgtext+0avgdata 8228maxresident)k
0inputs+0outputs (0major+2051minor)pagefaults 0swaps
Command exited with non-zero status 137
0.13user 0.04system 0:00.17elapsed 100%CPU (0avgtext+0avgdata 11256maxresident)k
0inputs+0outputs (0major+9706minor)pagefaults 0swaps

real    0m0.287s
user    0m0.215s
sys     0m0.071s

***  Finished generating whole program pinballs [log_whole]  ***    July 31, 2023 08:28:32

***********************************************************************************************
sde_pinpoints.py ERROR: A problem occurred in phase - Whole program pinball generation [log_whole]

***  TRACING: END  ***    July 31, 2023 08:28:32
[LOOPPOINT] Generating DCFG file.
[LOOPPOINT] Error: Whole-program pinball not found.

It seems like there is something wrong with the sde to attach the specific pid.
Can you give me some advice?