Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

inspxe-cl: 4 Data race problem(s) detected #3

Open
jeffhammond opened this issue Feb 10, 2017 · 6 comments
Open

inspxe-cl: 4 Data race problem(s) detected #3

jeffhammond opened this issue Feb 10, 2017 · 6 comments

Comments

@jeffhammond
Copy link

I followed your directions (thanks!) on identifying races in PENNANT and observe the following in an unmodified version.

Are these false positives?

jrhammon@klondike:~/Work/DOE/APEX/PENNANT-github$ OMP_NUM_THREADS=8 KMP_AFFINITY=scatter,verbose,granularity=fine KMP_SETTINGS=TRUE inspxe-cl -collect=ti2 -result-dir xxx -- ./build/pennant ./test/leblanc/leblanc.pnt | tee yyy
********************
Running PENNANT v0.9
********************


User settings:

   KMP_AFFINITY=scatter,verbose,granularity=fine
   KMP_FOR_TCHECK=1
   KMP_SETTINGS=TRUE
   OMP_NUM_THREADS=8

Effective settings:

   KMP_ABORT_DELAY=0
   KMP_ADAPTIVE_LOCK_PROPS='1,1024'
   KMP_ALIGN_ALLOC=64
   KMP_ALL_THREADPRIVATE=128
   KMP_ALL_THREADS=2147483647
   KMP_ATOMIC_MODE=2
   KMP_BLOCKTIME=200
   KMP_CPUINFO_FILE: value is not defined
   KMP_DETERMINISTIC_REDUCTION=false
   KMP_DISP_NUM_BUFFERS=7
   KMP_DUPLICATE_LIB_OK=false
   KMP_FORCE_REDUCTION: value is not defined
   KMP_FOREIGN_THREADS_THREADPRIVATE=true
   KMP_FORKJOIN_BARRIER='2,2'
   KMP_FORKJOIN_BARRIER_PATTERN='hyper,hyper'
   KMP_FORKJOIN_FRAMES=true
   KMP_FORKJOIN_FRAMES_MODE=3
   KMP_GTID_MODE=3
   KMP_HANDLE_SIGNALS=false
   KMP_HOT_TEAMS_MAX_LEVEL=1
   KMP_HOT_TEAMS_MODE=0
   KMP_INIT_AT_FORK=true
   KMP_INIT_WAIT=2048
   KMP_ITT_PREPARE_DELAY=0
   KMP_LIBRARY=throughput
   KMP_LOCK_KIND=queuing
   KMP_MALLOC_POOL_INCR=1M
   KMP_NEXT_WAIT=1024
   KMP_NUM_LOCKS_IN_BLOCK=1
   KMP_PLAIN_BARRIER='2,2'
   KMP_PLAIN_BARRIER_PATTERN='hyper,hyper'
   KMP_REDUCTION_BARRIER='1,1'
   KMP_REDUCTION_BARRIER_PATTERN='hyper,hyper'
   KMP_SCHEDULE='static,balanced;guided,iterative'
   KMP_SETTINGS=true
   KMP_SPIN_BACKOFF_PARAMS='4096,100'
   KMP_STACKOFFSET=64
   KMP_STACKPAD=0
   KMP_STACKSIZE=4M
   KMP_STORAGE_MAP=false
   KMP_TASKING=2
   KMP_TASK_STEALING_CONSTRAINT=1
   KMP_USER_LEVEL_MWAIT=false
   KMP_VERSION=false
   KMP_WARNINGS=true
   OMP_CANCELLATION=false
   OMP_DEFAULT_DEVICE=0
   OMP_DISPLAY_ENV=false
   OMP_DYNAMIC=false
   OMP_MAX_ACTIVE_LEVELS=2147483647
   OMP_MAX_TASK_PRIORITY=0
   OMP_NESTED=false
   OMP_NUM_THREADS='8'
   OMP_PLACES: value is not defined
   OMP_PROC_BIND='intel'
   OMP_SCHEDULE='static'
   OMP_STACKSIZE=4M
   OMP_THREAD_LIMIT=2147483647
   OMP_WAIT_POLICY=PASSIVE
   KMP_AFFINITY='verbose,warnings,respect,granularity=fine,duplicates,scatter,0,0'

OMP: Info #204: KMP_AFFINITY: decoding x2APIC ids.
OMP: Info #202: KMP_AFFINITY: Affinity capable, using global cpuid leaf 11 info
OMP: Info #154: KMP_AFFINITY: Initial OS proc set respected: {0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15}
OMP: Info #156: KMP_AFFINITY: 16 available OS procs
OMP: Info #157: KMP_AFFINITY: Uniform topology
OMP: Info #179: KMP_AFFINITY: 1 packages x 8 cores/pkg x 2 threads/core (8 total cores)
OMP: Info #206: KMP_AFFINITY: OS proc to physical thread map:
OMP: Info #171: KMP_AFFINITY: OS proc 0 maps to package 0 core 0 thread 0 
OMP: Info #171: KMP_AFFINITY: OS proc 8 maps to package 0 core 0 thread 1 
OMP: Info #171: KMP_AFFINITY: OS proc 1 maps to package 0 core 1 thread 0 
OMP: Info #171: KMP_AFFINITY: OS proc 9 maps to package 0 core 1 thread 1 
OMP: Info #171: KMP_AFFINITY: OS proc 2 maps to package 0 core 2 thread 0 
OMP: Info #171: KMP_AFFINITY: OS proc 10 maps to package 0 core 2 thread 1 
OMP: Info #171: KMP_AFFINITY: OS proc 3 maps to package 0 core 3 thread 0 
OMP: Info #171: KMP_AFFINITY: OS proc 11 maps to package 0 core 3 thread 1 
OMP: Info #171: KMP_AFFINITY: OS proc 4 maps to package 0 core 4 thread 0 
OMP: Info #171: KMP_AFFINITY: OS proc 12 maps to package 0 core 4 thread 1 
OMP: Info #171: KMP_AFFINITY: OS proc 5 maps to package 0 core 5 thread 0 
OMP: Info #171: KMP_AFFINITY: OS proc 13 maps to package 0 core 5 thread 1 
OMP: Info #171: KMP_AFFINITY: OS proc 6 maps to package 0 core 6 thread 0 
OMP: Info #171: KMP_AFFINITY: OS proc 14 maps to package 0 core 6 thread 1 
OMP: Info #171: KMP_AFFINITY: OS proc 7 maps to package 0 core 7 thread 0 
OMP: Info #171: KMP_AFFINITY: OS proc 15 maps to package 0 core 7 thread 1 
OMP: Info #242: KMP_AFFINITY: pid 4864 thread 0 bound to OS proc set {0}
Running on 8 thread(s)
--- Mesh Information ---
Points:  1001
Zones:  900
Sides:  3600
Edges:  1900
Side chunks:  8
Point chunks:  2
Zone chunks:  2
Chunk size:  512
------------------------
OMP: Info #242: KMP_AFFINITY: pid 4864 thread 1 bound to OS proc set {1}
OMP: Info #242: KMP_AFFINITY: pid 4864 thread 2 bound to OS proc set {2}
OMP: Info #242: KMP_AFFINITY: pid 4864 thread 3 bound to OS proc set {3}
OMP: Info #242: KMP_AFFINITY: pid 4864 thread 4 bound to OS proc set {4}
OMP: Info #242: KMP_AFFINITY: pid 4864 thread 5 bound to OS proc set {5}
OMP: Info #242: KMP_AFFINITY: pid 4864 thread 6 bound to OS proc set {6}
OMP: Info #242: KMP_AFFINITY: pid 4864 thread 7 bound to OS proc set {7}
Warning: One or more threads in the application accessed the stack of another thread. This may indicate one or more bugs in your application. Setting the Inspector to detect data races on stack accesses and running another analysis may help you locate these and other bugs.
Energy check:  total energy  =   9.424778e-01
(internal =   9.424778e-01, kinetic =   0.000000e+00)
End cycle      1, time = 1.00000e-02, dt = 1.00000e-02, wall = 6.03153e-01
dt limiter: Initial timestep
End cycle     10, time = 2.59587e-01, dt = 5.15978e-02, wall = 2.02562e-01
dt limiter: Recovery: Initial timestep

Run complete
cycle =     10,         cstop =     10
time  =   2.595868e-01, tstop =   6.000000e+00

************************************
hydro cycle run time=   8.086300e-01
************************************
Energy check:  total energy  =   9.424778e-01
(internal =   9.359219e-01, kinetic =   6.555903e-03)
Writing .xy file...
  
4 new problem(s) found 
    4 Data race problem(s) detected 
jrhammon@klondike:~/Work/DOE/APEX/PENNANT-github$ inspxe-cl -report problems -result-dir xxx
P1: Error: Data race: New
 P1.13: Error: Data race: New
  pennant!0x1c3ce: Error X32: Allocation site: Function Driver: Module /home/jrhammon/Work/DOE/APEX/PENNANT-github/build/pennant
  pennant!0x1f8cc: Error X33: Write: Function doCycle: Module /home/jrhammon/Work/DOE/APEX/PENNANT-github/build/pennant
  pennant!0x1f868: Error X34: Read: Function doCycle: Module /home/jrhammon/Work/DOE/APEX/PENNANT-github/build/pennant
  pennant!0x23f6e: Error X35: HINT: Synchronization allocation site: Function writeEnergyCheck: Module /home/jrhammon/Work/DOE/APEX/PENNANT-github/build/pennant

P2: Error: Data race: New
 P2.14: Error: Data race: New
  pennant!0x10c98: Error X36: Allocation site: Function init: Module /home/jrhammon/Work/DOE/APEX/PENNANT-github/build/pennant
  pennant!0xe3de: Error X37: Write: Function calcEdgeLen: Module /home/jrhammon/Work/DOE/APEX/PENNANT-github/build/pennant
  pennant!0xa0d3: Error X38: Read: Function setCornerDiv: Module /home/jrhammon/Work/DOE/APEX/PENNANT-github/build/pennant
 P2.15: Error: Data race: New
  pennant!0x10c98: Error X39: Allocation site: Function init: Module /home/jrhammon/Work/DOE/APEX/PENNANT-github/build/pennant
  pennant!0xe3de: Error X40: Write: Function calcEdgeLen: Module /home/jrhammon/Work/DOE/APEX/PENNANT-github/build/pennant
  pennant!0xa0f2: Error X41: Read: Function setCornerDiv: Module /home/jrhammon/Work/DOE/APEX/PENNANT-github/build/pennant
 P2.16: Error: Data race: New
  pennant!0x10c98: Error X42: Allocation site: Function init: Module /home/jrhammon/Work/DOE/APEX/PENNANT-github/build/pennant
  pennant!0xe3de: Error X43: Write: Function calcEdgeLen: Module /home/jrhammon/Work/DOE/APEX/PENNANT-github/build/pennant
  pennant!0xacf9: Error X44: Read: Function setQCnForce: Module /home/jrhammon/Work/DOE/APEX/PENNANT-github/build/pennant
 P2.17: Error: Data race: New
  pennant!0x10c98: Error X45: Allocation site: Function init: Module /home/jrhammon/Work/DOE/APEX/PENNANT-github/build/pennant
  pennant!0xe3de: Error X46: Write: Function calcEdgeLen: Module /home/jrhammon/Work/DOE/APEX/PENNANT-github/build/pennant
  pennant!0xb1f7: Error X47: Read: Function setForce: Module /home/jrhammon/Work/DOE/APEX/PENNANT-github/build/pennant
 P2.18: Error: Data race: New
  pennant!0x10c98: Error X48: Allocation site: Function init: Module /home/jrhammon/Work/DOE/APEX/PENNANT-github/build/pennant
  pennant!0xe3de: Error X49: Write: Function calcEdgeLen: Module /home/jrhammon/Work/DOE/APEX/PENNANT-github/build/pennant
  pennant!0x9250: Error X50: Read: Function calcForce: Module /home/jrhammon/Work/DOE/APEX/PENNANT-github/build/pennant
  pennant!0x10d2f: Error X51: HINT: Synchronization allocation site: Function init: Module /home/jrhammon/Work/DOE/APEX/PENNANT-github/build/pennant
 P2.20: Error: Data race: New
  pennant!0x10c98: Error X55: Allocation site: Function init: Module /home/jrhammon/Work/DOE/APEX/PENNANT-github/build/pennant
  pennant!0xe3de: Error X56: Write: Function calcEdgeLen: Module /home/jrhammon/Work/DOE/APEX/PENNANT-github/build/pennant
  pennant!0xe3de: Error X57: Write: Function calcEdgeLen: Module /home/jrhammon/Work/DOE/APEX/PENNANT-github/build/pennant
 P2.22: Error: Data race: New
  pennant!0x10c98: Error X62: Allocation site: Function init: Module /home/jrhammon/Work/DOE/APEX/PENNANT-github/build/pennant
  pennant!0xe3de: Error X63: Write: Function calcEdgeLen: Module /home/jrhammon/Work/DOE/APEX/PENNANT-github/build/pennant
  pennant!0xad5d: Error X64: Read: Function setQCnForce: Module /home/jrhammon/Work/DOE/APEX/PENNANT-github/build/pennant
 P2.23: Error: Data race: New
  pennant!0x10c98: Error X65: Allocation site: Function init: Module /home/jrhammon/Work/DOE/APEX/PENNANT-github/build/pennant
  pennant!0xe3de: Error X66: Write: Function calcEdgeLen: Module /home/jrhammon/Work/DOE/APEX/PENNANT-github/build/pennant
  pennant!0xa0d9: Error X67: Read: Function setCornerDiv: Module /home/jrhammon/Work/DOE/APEX/PENNANT-github/build/pennant
  pennant!0x10d2f: Error X68: HINT: Synchronization allocation site: Function init: Module /home/jrhammon/Work/DOE/APEX/PENNANT-github/build/pennant
 P2.24: Error: Data race: New
  pennant!0x10c98: Error X69: Allocation site: Function init: Module /home/jrhammon/Work/DOE/APEX/PENNANT-github/build/pennant
  pennant!0xe3de: Error X70: Write: Function calcEdgeLen: Module /home/jrhammon/Work/DOE/APEX/PENNANT-github/build/pennant
  pennant!0xa0ec: Error X71: Read: Function setCornerDiv: Module /home/jrhammon/Work/DOE/APEX/PENNANT-github/build/pennant
  pennant!0x10d2f: Error X72: HINT: Synchronization allocation site: Function init: Module /home/jrhammon/Work/DOE/APEX/PENNANT-github/build/pennant
 P2.25: Error: Data race: New
  pennant!0x10c98: Error X73: Allocation site: Function init: Module /home/jrhammon/Work/DOE/APEX/PENNANT-github/build/pennant
  pennant!0xe3de: Error X74: Write: Function calcEdgeLen: Module /home/jrhammon/Work/DOE/APEX/PENNANT-github/build/pennant
  pennant!0xb1ed: Error X75: Read: Function setForce: Module /home/jrhammon/Work/DOE/APEX/PENNANT-github/build/pennant

P3: Error: Data race: New
 P3.9: Error: Data race: New
  pennant!0x10b24: Error X16: Allocation site: Function init: Module /home/jrhammon/Work/DOE/APEX/PENNANT-github/build/pennant
  pennant!0xe7ab: Error X17: Write: Function calcCtrs: Module /home/jrhammon/Work/DOE/APEX/PENNANT-github/build/pennant
  pennant!0x9f4c: Error X18: Read: Function setCornerDiv: Module /home/jrhammon/Work/DOE/APEX/PENNANT-github/build/pennant
  pennant!0x1ea8b: Error X19: HINT: Synchronization allocation site: Function doCycle: Module /home/jrhammon/Work/DOE/APEX/PENNANT-github/build/pennant
 P3.11: Error: Data race: New
  pennant!0x10b24: Error X24: Allocation site: Function init: Module /home/jrhammon/Work/DOE/APEX/PENNANT-github/build/pennant
  pennant!0xe7ab: Error X25: Write: Function calcCtrs: Module /home/jrhammon/Work/DOE/APEX/PENNANT-github/build/pennant
  pennant!0xa05a: Error X26: Read: Function setCornerDiv: Module /home/jrhammon/Work/DOE/APEX/PENNANT-github/build/pennant
  pennant!0x1ea8b: Error X27: HINT: Synchronization allocation site: Function doCycle: Module /home/jrhammon/Work/DOE/APEX/PENNANT-github/build/pennant
 P3.19: Error: Data race: New
  pennant!0x10b24: Error X52: Allocation site: Function init: Module /home/jrhammon/Work/DOE/APEX/PENNANT-github/build/pennant
  pennant!0xe7ab: Error X53: Write: Function calcCtrs: Module /home/jrhammon/Work/DOE/APEX/PENNANT-github/build/pennant
  pennant!0xe7ab: Error X54: Write: Function calcCtrs: Module /home/jrhammon/Work/DOE/APEX/PENNANT-github/build/pennant

P4: Error: Data race: New
 P4.10: Error: Data race: New
  pennant!0x10b24: Error X20: Allocation site: Function init: Module /home/jrhammon/Work/DOE/APEX/PENNANT-github/build/pennant
  pennant!0xe7b1: Error X21: Write: Function calcCtrs: Module /home/jrhammon/Work/DOE/APEX/PENNANT-github/build/pennant
  pennant!0x9f6b: Error X22: Read: Function setCornerDiv: Module /home/jrhammon/Work/DOE/APEX/PENNANT-github/build/pennant
  pennant!0x1ea8b: Error X23: HINT: Synchronization allocation site: Function doCycle: Module /home/jrhammon/Work/DOE/APEX/PENNANT-github/build/pennant
 P4.12: Error: Data race: New
  pennant!0x10b24: Error X28: Allocation site: Function init: Module /home/jrhammon/Work/DOE/APEX/PENNANT-github/build/pennant
  pennant!0xe7b1: Error X29: Write: Function calcCtrs: Module /home/jrhammon/Work/DOE/APEX/PENNANT-github/build/pennant
  pennant!0xa077: Error X30: Read: Function setCornerDiv: Module /home/jrhammon/Work/DOE/APEX/PENNANT-github/build/pennant
  pennant!0x1ea8b: Error X31: HINT: Synchronization allocation site: Function doCycle: Module /home/jrhammon/Work/DOE/APEX/PENNANT-github/build/pennant
 P4.21: Error: Data race: New
  pennant!0x10b24: Error X58: Allocation site: Function init: Module /home/jrhammon/Work/DOE/APEX/PENNANT-github/build/pennant
  pennant!0xe7b1: Error X59: Write: Function calcCtrs: Module /home/jrhammon/Work/DOE/APEX/PENNANT-github/build/pennant
  pennant!0xe7b1: Error X60: Write: Function calcCtrs: Module /home/jrhammon/Work/DOE/APEX/PENNANT-github/build/pennant
  pennant!0x10d2f: Error X61: HINT: Synchronization allocation site: Function init: Module /home/jrhammon/Work/DOE/APEX/PENNANT-github/build/pennant
@brobey
Copy link
Member

brobey commented Feb 10, 2017 via email

@brobey
Copy link
Member

brobey commented Feb 10, 2017 via email

@jeffhammond
Copy link
Author

I compiled with -g3 and spent an hour reading online docs but could not figure out how to get line numbers. Obviously, I should be able to figure this help from the appropriate Intel colleague.

@brobey
Copy link
Member

brobey commented Feb 11, 2017 via email

@cferenba
Copy link
Member

@jeffhammond : Sorry for the delay in replying.
I believe the last three results you found are false positives. The code is set up so that each OpenMP thread writes/reads its own section of the large arrays for edge lengths, cell centers, etc.; but the way that's enforced by the code probably can't be figured out by the analyzer.
I can't quite tell what the first result is referring to. Were you ever able to get line numbers?

@brobey
Copy link
Member

brobey commented Feb 16, 2017 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@jeffhammond @brobey @cferenba