snap-stanford / snap Goto Github PK
View Code? Open in Web Editor NEWStanford Network Analysis Platform (SNAP) is a general purpose network analysis and graph mining library.
License: Other
Stanford Network Analysis Platform (SNAP) is a general purpose network analysis and graph mining library.
License: Other
Hello, would it be OK with snap team to convert the readme files into markdown format?
Hi,
I had a little trouble with installing the snap.py win x64 from snap.stanford.edu. Turns out the setup.py file had a few errors. I've corrected them for you though, please see attached file and replace in the zip file on the snap site :)
The changes are lines 94, 158-161, 163.
94 had tabs in, so I removed these with 12 spaces.
the remaining lines required parenthesis.
Thank you for the files though and data sets, I'm now going to have a play!
Hi, this is an enhancement request.
Please include a function to compute the all-to-all distance matrix and also the average pathlength of a graph.
I am testing packages because I need to compute the pathlength of graphs of size 10^4 and larger. My own package will struggle at that size. I came across SNAP and I found that the only manner to compute the average pathlength is to use the function GetShortPath() to compute the distance for one node to all others and iterate that for all nodes. While this is already a waste of computational effort, in my case it is even worse because I tried the Python interface and having to do that for-loop in Python spoils any enhancement the library has to offer.
Best.
Russells-MacBook-Pro-OLD:rolx rjurney$ make
g++ -fopenmp -o testrolx testrolx.cpp ../../snap-adv/rolx.cpp ../../snap-core/Snap.o -I../../snap-core -I../../snap-adv -I../../glib-core -I../../snap-exp
ld: library not found for -lgomp
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [testrolx] Error 1
TInt and TUInt64 both support the += operator but it is missing from TUInt.
Currently, because SNAP uses min and max as define directives, if another library or developer code makes use of min / max functions with other signatures, it generates a conflict.
The solution I came with is un-defining min/max after SNAP inclusion:
Otherwise, trying to use, for example, numeric_limits::min() will generate a compile error. This also implies that some standard library headers or custom headers need to be included afterwards.
When my program finishes executing I get the following error (with backtrace):
malloc: *** error for object 0x100000010: pointer being freed was not allocated
*** set a breakpoint in malloc_error_break to debug
Program received signal SIGABRT, Aborted.
0x00007fff8a66ad46 in __kill ()
(gdb) backtrace
#0 0x00007fff8a66ad46 in __kill ()
#1 0x00007fff8fd5fdf0 in abort ()
#2 0x00007fff8fd339b9 in free ()
#3 0x0000000100003763 in TVec<TInt, int>::~TVec (this=0x101dbffe8) at ds.h:438
#4 0x0000000100003ba2 in TNEANet::TNode::~TNode (this=0x101dbffe0) at network.h:1466
#5 0x0000000100003bc0 in THashKeyDat<TInt, TNEANet::TNode>::~THashKeyDat (this=0x101dbffd4) at hash.h:7
#6 0x0000000100003c3c in TVec<THashKeyDat<TInt, TNEANet::TNode>, int>::~TVec (this=0x100609830) at ds.h:438
#7 0x0000000100003190 in THash<TInt, TNEANet::TNode, TDefaultHashFunc<TInt> >::~THash (this=0x100609820) at hash.h:88
#8 0x000000010000518a in TNEANet::~TNEANet (this=0x100609810) at network.h:1461
#9 0x00000001000051ed in TPt<TNEANet>::UnRef (this=0x7fff5fbff8c0) at bd.h:472
#10 0x000000010000412a in TPt<TNEANet>::~TPt (this=0x7fff5fbff8c0) at bd.h:480
build (make all) fails:
make all
/Applications/Xcode.app/Contents/Developer/usr/bin/make -C snap-core
g++ -c -std=c++98 -Wall -O3 -DNDEBUG -DNOMP Snap.cpp -I../glib-core
Snap.cpp:29:10: fatal error: 'triad.cpp' file not found
^
1 error generated.
make[1]: *** [Snap.o] Error 1
make: *** [MakeAll] Error 2
Which function in the MAGFit classes gives me the Log Likelihood (measures the possibility that the probabilistic adjacency matrix P generates network A)? I can find this in KronFit to be LogLike() but cannot figure this out for MAGFit. Please help! Thanks!
Iterating through the non-root nodes and displaying tree->GetNI(node)->GetInDeg() sometimes shows values larger than 1 (meaning that a node would have more than one parent).
Iterating through the values contained in GetInNId() I see the same value multiple times (so it seems that for some reason there are multiple edges from the parent to the child node).
Being able to label edge weights with GraphViz would be a great feature.
Hi,
I am not able to find the kronecker.h file in this. Can you please provide me the full code for the krogen which will be helpful for me in my research.
Thanks,
Narendra.
I was also wondering what changes should I make in the code if i want to compute the log likelihood in Kronfit a way such that it considers the log likelihood of the graph considering the probability of the edges for nodes which are non existent in the initial graph as zero. I tried the KronEM version but it scales the initial parameters after 1 iteration and doesn't run (gives an error) for a single iteration for both E and M.
Thanks!
Python 2.7.10 (v2.7.10:15c95b7d81dc, May 23 2015, 09:33:12)
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import snap
Fatal Python error: PyThreadState_Get: no current thread
Abort trap: 6
Please help...............
error log see below:
Process: Python [11381]
Path: /Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python
Identifier: Python
Version: 2.7.10 (2.7.10)
Code Type: X86-64 (Native)
Parent Process: bash [11371]
Responsible: iTerm [7379]
User ID: 501
Date/Time: 2015-10-16 20:26:42.407 -0500
OS Version: Mac OS X 10.11 (15A284)
Report Version: 11
Anonymous UUID: D5054451-BC61-CA61-9BF6-89966A16F02C
Sleep/Wake UUID: 3BF6D8EC-E1CA-4FBD-9843-44EF0D90D3C8
Time Awake Since Boot: 11000 seconds
Time Since Wake: 1500 seconds
System Integrity Protection: enabled
Crashed Thread: 0 Dispatch queue: com.apple.main-thread
Exception Type: EXC_CRASH (SIGABRT)
Exception Codes: 0x0000000000000000, 0x0000000000000000
Application Specific Information:
abort() called
Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0 libsystem_kernel.dylib 0x00007fff8894b0ae pthread_kill + 10
1 libsystem_pthread.dylib 0x00007fff969c7500 pthread_kill + 90
2 libsystem_c.dylib 0x00007fff8cef937b abort + 129
3 org.python.python 0x0000000105a69ce2 Py_FatalError + 49
4 org.python.python 0x0000000105a691f0 PyThreadState_Get + 28
5 org.python.python 0x0000000105a6603a Py_InitModule4_64 + 62
6 _snap.so 0x000000010443697f init_snap + 559
7 org.python.python 0x00000001000deba1 _PyImport_LoadDynamicModule + 177
8 org.python.python 0x00000001000dcc58 imp_load_module + 184
9 org.python.python 0x00000001000c357d PyEval_EvalFrameEx + 24829
10 org.python.python 0x00000001000c467e PyEval_EvalFrameEx + 29182
11 org.python.python 0x00000001000c58e3 PyEval_EvalCodeEx + 2115
12 org.python.python 0x00000001000c5a06 PyEval_EvalCode + 54
13 org.python.python 0x00000001000da0a0 PyImport_ExecCodeModuleEx + 208
14 org.python.python 0x00000001000db2a2 load_source_module + 626
15 org.python.python 0x00000001000dd28b import_submodule + 315
16 org.python.python 0x00000001000dd73a load_next + 234
17 org.python.python 0x00000001000dda30 PyImport_ImportModuleLevel + 336
18 org.python.python 0x00000001000bafe3 builtin___import + 131
19 org.python.python 0x000000010000c612 PyObject_Call + 98
20 org.python.python 0x00000001000bc1c7 PyEval_CallObjectWithKeywords + 87
21 org.python.python 0x00000001000c0432 PyEval_EvalFrameEx + 12210
22 org.python.python 0x00000001000c58e3 PyEval_EvalCodeEx + 2115
23 org.python.python 0x00000001000c5a06 PyEval_EvalCode + 54
24 org.python.python 0x00000001000e9f4c PyRun_InteractiveOneFlags + 380
25 org.python.python 0x00000001000ea1ae PyRun_InteractiveLoopFlags + 78
26 org.python.python 0x00000001000ea9c1 PyRun_AnyFileExFlags + 161
27 org.python.python 0x000000010010187d Py_Main + 3101
28 org.python.python 0x0000000100000f14 0x100000000 + 3860
Thread 0 crashed with X86 Thread State (64-bit):
rax: 0x0000000000000000 rbx: 0x0000000000000006 rcx: 0x00007fff5fbfe638 rdx: 0x0000000000000000
rdi: 0x0000000000000a0b rsi: 0x0000000000000006 rbp: 0x00007fff5fbfe660 rsp: 0x00007fff5fbfe638
r8: 0x0000000000000040 r9: 0x00007fff758f91e0 r10: 0x0000000008000000 r11: 0x0000000000000206
r12: 0x0000000000000000 r13: 0x0000000105429efb r14: 0x00007fff76e9a000 r15: 0x00000001036355d4
rip: 0x00007fff8894b0ae rfl: 0x0000000000000206 cr2: 0x00007fff758f7038
Logical CPU: 0
Error Code: 0x02000148
Trap Number: 133
Binary Images:
0x100000000 - 0x100000fff +org.python.python (2.7.10 - 2.7.10) /Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python
0x100003000 - 0x100170fff +org.python.python (2.7.10, [c] 2001-2015 Python Software Foundation. - 2.7.10) <5E0C1150-83D5-6364-A820-E7AD67962D79> /Library/Frameworks/Python.framework/Versions/2.7/Python
0x1002f2000 - 0x1002f4ff7 +_locale.so (???) <3C1429AD-B0EF-96BF-9E7E-2F7B48975B36> /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload/_locale.so
0x1020f0000 - 0x1020f2ff7 +readline.so (???) /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload/readline.so
0x102240000 - 0x102294fe7 +libncursesw.5.dylib (5) <3F0079C0-01C1-3CB8-19CA-F9B49AA4F4A4> /Library/Frameworks/Python.framework/Versions/2.7/lib/libncursesw.5.dylib
0x103680000 - 0x1055f6fe7 +_snap.so (0) <3D2FCCCC-4937-3BC0-AF45-3FAAE655B204> /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/_snap.so
0x1059c8000 - 0x105ab9ff7 org.python.python (2.7.10 - 2.7.10) <5A7838D3-24D4-395B-BE96-ADD007C86E55> /System/Library/Frameworks/Python.framework/Versions/2.7/Python
0x7fff65a4e000 - 0x7fff65a84f5f dyld (360.14) /usr/lib/dyld
0x7fff8576f000 - 0x7fff8579cfff libdispatch.dylib (500.1.5) <6B38497E-9448-3433-9D6B-6223F2A99431> /usr/lib/system/libdispatch.dylib
0x7fff85a25000 - 0x7fff85a27ff7 libsystem_configuration.dylib (801.1.1) /usr/lib/system/libsystem_configuration.dylib
0x7fff85c2c000 - 0x7fff85c4affb libedit.3.dylib (43) <744915BA-9B98-3256-8DBB-5C760132623F> /usr/lib/libedit.3.dylib
0x7fff85c4b000 - 0x7fff85c52ff7 libcompiler_rt.dylib (62) <253B36E5-572D-377D-AE99-A02CE32590E5> /usr/lib/system/libcompiler_rt.dylib
0x7fff86670000 - 0x7fff866beff7 libstdc++.6.dylib (104.1) <77780A99-22DB-35AA-BD9E-ADB83417E4BD> /usr/lib/libstdc++.6.dylib
0x7fff866d2000 - 0x7fff866eeff7 libsystem_malloc.dylib (67) <1B57A614-3D60-3F87-876F-7DB4AF38120F> /usr/lib/system/libsystem_malloc.dylib
0x7fff86d02000 - 0x7fff86d03ffb libSystem.B.dylib (1225.1.1) /usr/lib/libSystem.B.dylib
0x7fff86f0d000 - 0x7fff8711afff libicucore.A.dylib (551.24) /usr/lib/libicucore.A.dylib
0x7fff873c0000 - 0x7fff87406ff7 libauto.dylib (186) <460B0167-C89B-37EC-823C-52F684B31C26> /usr/lib/libauto.dylib
0x7fff8745f000 - 0x7fff878d3ff7 com.apple.CoreFoundation (6.9 - 1253) /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation
0x7fff878d4000 - 0x7fff87c376d7 libobjc.A.dylib (680) <7C5FAD04-2C01-3ED6-AA40-78925C12A456> /usr/lib/libobjc.A.dylib
0x7fff87ca2000 - 0x7fff87caafef libsystem_platform.dylib (73.1.1) <3F4D2390-E3DE-3C24-A515-95DFAC8671C4> /usr/lib/system/libsystem_platform.dylib
0x7fff87cab000 - 0x7fff87d22fc7 libcorecrypto.dylib (334) <4E1B969F-8449-3B21-9880-51AD58E25AA6> /usr/lib/system/libcorecrypto.dylib
0x7fff87d23000 - 0x7fff87d3afff libsystem_asl.dylib (322) <97D794DA-8CE5-3676-AC5E-364F6D172BDA> /usr/lib/system/libsystem_asl.dylib
0x7fff881c4000 - 0x7fff88217ff7 libc++.1.dylib (120.1) <54190E1B-EE49-3D6D-AC29-2813D7380BA5> /usr/lib/libc++.1.dylib
0x7fff887e4000 - 0x7fff88813fc3 libsystem_m.dylib (3105) <07D50372-30ED-3B03-9FA0-0662BF8F0098> /usr/lib/system/libsystem_m.dylib
0x7fff88934000 - 0x7fff88952fff libsystem_kernel.dylib (3247.1.106) <7DD242A1-E2BF-39D1-8787-B174046E4F15> /usr/lib/system/libsystem_kernel.dylib
0x7fff88960000 - 0x7fff88965ff3 libunwind.dylib (35.3) /usr/lib/system/libunwind.dylib
0x7fff89504000 - 0x7fff89504ff7 liblaunch.dylib (755.1.19) <7EC0F297-43CC-3D11-B46B-7E72E372648A> /usr/lib/system/liblaunch.dylib
0x7fff8a6a5000 - 0x7fff8a6b6fff libz.1.dylib (60) <43317BEA-ACA2-34C2-AF37-902AA926C83A> /usr/lib/libz.1.dylib
0x7fff8a6f1000 - 0x7fff8a6faffb libsystem_notify.dylib (149) <56ABC155-CB99-30A8-A8B1-C204B5615092> /usr/lib/system/libsystem_notify.dylib
0x7fff8a98f000 - 0x7fff8a992fff libsystem_sandbox.dylib (459.1.8) <2F36D536-482C-39EC-BAFD-72297728F0A4> /usr/lib/system/libsystem_sandbox.dylib
0x7fff8afb6000 - 0x7fff8afb6ff7 libunc.dylib (29) /usr/lib/system/libunc.dylib
0x7fff8b280000 - 0x7fff8b281fff libDiagnosticMessagesClient.dylib (100) /usr/lib/libDiagnosticMessagesClient.dylib
0x7fff8ce9b000 - 0x7fff8cf28fe7 libsystem_c.dylib (1081.1.3) /usr/lib/system/libsystem_c.dylib
0x7fff8d9bd000 - 0x7fff8d9c2ff7 libmacho.dylib (875.1) /usr/lib/system/libmacho.dylib
0x7fff8d9c3000 - 0x7fff8d9cbfff libcopyfile.dylib (127) /usr/lib/system/libcopyfile.dylib
0x7fff8e038000 - 0x7fff8e096fff libsystem_network.dylib (582.1.4) <14ECA259-D471-3E47-A843-FF0990577893> /usr/lib/system/libsystem_network.dylib
0x7fff8e097000 - 0x7fff8e097ff7 libkeymgr.dylib (28) <47080280-8B57-3D75-8A20-9E100864DE27> /usr/lib/system/libkeymgr.dylib
0x7fff923a8000 - 0x7fff923acfff libcache.dylib (75) <4948E2C8-867F-3E9D-AAE7-2F30F0B345C6> /usr/lib/system/libcache.dylib
0x7fff925a5000 - 0x7fff925a6fff libsystem_secinit.dylib (20) <932ED582-E80F-39DA-B0FA-F1BC5F1AD2F8> /usr/lib/system/libsystem_secinit.dylib
0x7fff925b8000 - 0x7fff925b9ffb libremovefile.dylib (41) /usr/lib/system/libremovefile.dylib
0x7fff92851000 - 0x7fff9287afff libc++abi.dylib (125) /usr/lib/libc++abi.dylib
0x7fff92901000 - 0x7fff9292afff libxpc.dylib (755.1.19) <3E09C275-A33B-357A-B0AB-A2DDF88EC9D5> /usr/lib/system/libxpc.dylib
0x7fff93025000 - 0x7fff93056ff7 libncurses.5.4.dylib (46) <766F2188-F523-3FAA-AC1F-49447F09E133> /usr/lib/libncurses.5.4.dylib
0x7fff93137000 - 0x7fff93139fff libsystem_coreservices.dylib (19) <692631A0-1923-32CA-9BD5-044B1382FFDE> /usr/lib/system/libsystem_coreservices.dylib
0x7fff932c3000 - 0x7fff932ecff7 libsystem_info.dylib (476) <65D0643A-C8AE-3E8D-9F6E-E4AD823F16B2> /usr/lib/system/libsystem_info.dylib
0x7fff95a07000 - 0x7fff95a09ff7 libquarantine.dylib (80) <1693C5FE-EA0A-3122-85EB-7950ECC7435A> /usr/lib/system/libquarantine.dylib
0x7fff95d22000 - 0x7fff95d2afff libsystem_networkextension.dylib (384.1.2) <4736FCC5-9DBA-31F4-AAC8-CD0A177CF502> /usr/lib/system/libsystem_networkextension.dylib
0x7fff9694e000 - 0x7fff96962fff libsystem_coretls.dylib (82) <21EDACF1-D9B3-3086-9821-60EB75E7F965> /usr/lib/system/libsystem_coretls.dylib
0x7fff96966000 - 0x7fff96967fff libsystem_blocks.dylib (65) <1B4F1F10-823E-3781-8162-6884D14DF0D6> /usr/lib/system/libsystem_blocks.dylib
0x7fff969c1000 - 0x7fff969caff7 libsystem_pthread.dylib (137.1.1) <1373D0F1-C6CA-364E-A6BA-8BDBD0D34670> /usr/lib/system/libsystem_pthread.dylib
0x7fff96a45000 - 0x7fff96a56ff7 libsystem_trace.dylib (200) /usr/lib/system/libsystem_trace.dylib
0x7fff97794000 - 0x7fff97797ffb libdyld.dylib (360.14) /usr/lib/system/libdyld.dylib
0x7fff97798000 - 0x7fff977a3ff7 libcommonCrypto.dylib (60074) /usr/lib/system/libcommonCrypto.dylib
0x7fff98089000 - 0x7fff98091ffb libsystem_dnssd.dylib (624.1.2) /usr/lib/system/libsystem_dnssd.dylib
External Modification Summary:
Calls made by other processes targeting this process:
task_for_pid: 0
thread_create: 0
thread_set_state: 0
Calls made by this process:
task_for_pid: 0
thread_create: 0
thread_set_state: 0
Calls made by all processes on this machine:
task_for_pid: 18946
thread_create: 0
thread_set_state: 0
VM Region Summary:
ReadOnly portion of Libraries: Total=140.2M resident=0K(0%) swapped_out_or_unallocated=140.2M(100%)
Writable regions: Total=60.0M written=0K(0%) resident=0K(0%) swapped_out=0K(0%) unallocated=60.0M(100%)
VIRTUAL REGION
REGION TYPE SIZE COUNT (non-coalesced)
=========== ======= =======
Activity Tracing 2048K 2
Kernel Alloc Once 4K 2
MALLOC 49.7M 18
MALLOC guard page 16K 4
STACK GUARD 56.0M 2
Stack 8192K 2
VM_ALLOCATE 4K 2
__DATA 4184K 57
__LINKEDIT 91.4M 10
__TEXT 48.8M 55
__UNICODE 552K 2
shared memory 8K 3
=========== ======= =======
TOTAL 260.5M 147
The dataset is DBLP dataset.
Code -
import snap,sys
UGraph = snap.LoadEdgeList(snap.PUNGraph, "../dataset/new_graph.txt",0,1)
CmtyV = snap.TCnComV()
modularity = snap.CommunityCNM(UGraph, CmtyV)
Error message -
Traceback (most recent call last):
File "cnm.py", line 5, in <module>
modularity = snap.CommunityCNM(UGraph, CmtyV)
File "/amd/hamsa.cs.iiests.ac.in/users1/student/be2014/sarbajits/Graph/CNM/snap.py", line 35949, in CommunityCNM
return _snap.CommunityCNM(*args)
RuntimeError: Execution stopped: (0<=ValN)&&(ValN<Vals) [Reason:'Index:-2147403659 Vals:1073789418 MxVals:1298759680 Type:4TVecI7TTripleI4TFlt4TIntS2_EiE'], file /home/rok/include/glib/ds.h, line 469
Specifications of machine used -
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 24
On-line CPU(s) list: 0-23
Thread(s) per core: 2
Core(s) per socket: 6
Socket(s): 2
NUMA node(s): 2
Vendor ID: GenuineIntel
CPU family: 6
Model: 45
Stepping: 7
CPU MHz: 1200.000
BogoMIPS: 3791.94
Virtualization: VT-x
L1d cache: 32K
L1i cache: 32K
L2 cache: 256K
L3 cache: 15360K
NUMA node0 CPU(s): 0-5,12-17
NUMA node1 CPU(s): 6-11,18-23
If any more information is needed, please let me know. Thanks!
Dear all,
I'm trying to install snap for python (https://snap.stanford.edu/snappy/index.html) on my mac OSX but I get the following error:
"Fatal Python error: PyThreadState_Get: no current thread
Abort trap: 6"
Any idea?
Thanks in advance.
Matteo
I am using the C++ version of SNAP. I have an undirected graph with 174093228 nodes and 309401867 edges. The maximum node id in the graph is 174093228. The node iterator is behaving strangely when I input it nodes ids that do not exist in the graph. Please see the commands and the outputs below.
printf("%d\n", Graph->IsNode(500000000)); //output: 0
TUNGraph::TNodeI NI = Graph->GetNI(500000000);
printf("%d\n", NI.GetOutDeg()); //output: 1
printf("%d\n", NI.GetOutNId(0)); //output: 111788745
printf("%d\n", Graph->IsNode(NI.GetOutNId(0))); //output: 1
printf("%d\n", Graph->IsEdge(500000000, NI.GetOutNId(0))); //output: 0
When building with Visual Studio 2010, I get errors on "log(n)" statements where n is an integer.
The attached patch solves this issue.
patch.diff.txt
I noticed that TStr is thread unsafe. It would be useful to have some comments warning about this. An example that I ran into is the following:
Let C be a TStr and consider these two lines running in parallel:
TStr A = C;
TStr B = C;
C.RStr->MkRef() would be run concurrently which can cause the reference count to be incorrect.
I am working on training node2vec embeddings on Wikipedia using snap/examples/node2vec
. I am wanting to understand what is causing high memory usage and slow runtime to see if there is something I can do to improve performance.
System: AWS EC2 x1.32xlarge instance, 2TB RAM, 128 cores
Dataset:
s3://entilzha-us-west-2/wiki-network/titles-sorted.txt
s3://entilzha-us-west-2/wiki-network/links-simple-sorted.txt
preprocess_titles
and n2v_edge_list
node2vec -i:edge_list.txt -o:wiki.emb -v -dr
Final dataset info (I can post this input if its helpful):
When this is run it uses 750GB of RAM and it has gotten to learning the word2vec embeddings, but the process seems very slow at about 1% per hour for having ~100% utilization on all 128 cores.
My general questions are:
emb_dim*number_of_nodes~500,000,000
and assuming that each entry is a double should be 500,000,000*8 bytes = 4GB. For walks it should be something like n_walks*walk_size*number_of_nodes=10*80*4,000,000= 3,200,000,000
entries and again assuming each entry is a double would lead to 25GB. Memory usage seems to go up here https://github.com/snap-stanford/snap/blob/master/snap-adv/n2v.cpp#L14. Any thoughts on why it's using 30x that amount of memory?From what I can tell things that should affect these would be:
Thanks!
Hi,
I am trying to test the new multithreaded PNEANetMP. However, I cannot even add a node. Here is code that reproduces the bug:
#include "Snap.h"
using namespace std;
int main() {
PNEANetMP gg = TNEANetMP::New();
gg->AddNode(1);
}
the gg->AddNode(1)
line results in the following error and crash: "Floating point exception (core dumped)".
Let me know if I can help with debugging/testing, or if my code does something wrong. Thanks!
A colleague recently showed me the WWW 2014 paper “Finding Progression Stages in Time-evolving Event Sequences” recently, and it could be just the thing I need for analyzing some health claims data.
In the paper, it says the code is available in the snap package, but I couldn’t find it in the repo (https://github.com/snap-stanford/snap-dev ). I did find some implementation (https://github.com/m-ochi/progression_stage_model ), and it seems to work, but I’m hoping yours is faster.
Is it around here somewhere?
For TNEANet nodes, attribute-values can be added to nodes as key-values to a hash:
TNEANet::TNodeI NI = GU->BegNI();
GU->AddIntAttrDatN(NI, 10, "test");
GU->AddIntAttrDatN(NI, 20, "test1");
GU->AddIntAttrDatN(NI, 30, "test2");
Then when attributes need to be retrieved (all of them) for a node, they can only be retrieved as a vector, instead of a hash. So, the access at values for each of the attributes associated to a node is positional now, rather than key-based. Given that adding attribute values is hash-like, how can one know where to retrieve in the vector the value for a certain attribute (key)? For example:
NId = NI.GetId();
GU->AttrValueNI(NId, NIdAttrValue);
std::string curTestString = NIdAttrValue2;
// value at index 2 corresponds to what attribute (test, test1, or test2)?
int curTest = atoi(curTestString.c_str());
printf("%d\n", curTest);
Is there a way to retrieve all attributes for a node in a hash (i.e. based of attribute names) instead of in a vector? Would an iterator for each attribute (vertical retrieval in the dataset) be sufficient – meaning, multiple iterators, each for one attribute: if advancing iterators in parallel, but independently, will attribute values be retrieved for the same object?
I am trying to make a graph with GenSmallWorld function with different probabilities.
When the probability is more than 0, the graph has some missing edges.
For instance: in a graph with 12 nodes and 4 neighbours there should be 24 edges, but I get 22.
What I am doing wrong? It's not like it should be in the Watts-Strogatz model. Edges should be rewired, but not be deleted.
There is a bug in gcc compiler ( http://gcc.gnu.org/bugzilla/show_bug.cgi?id=51245 ) which is reported as fixed but I still have problem with it using gcc version 4.7.3 building snap.
I guess changing the code can solve the issue:
int n = 1;
// little endian if true
if(*(char *)&n == 1) {...}
Using BigClam with the default parameters, I received duplicate communities in the output. Is this the intended behavior? Perhaps, duplicate communities should be filtered?
Graph
0 5
0 9
0 16
0 17
0 23
0 26
1 18
1 11
1 21
2 25
2 18
2 14
3 4
3 7
3 16
3 18
3 19
3 29
4 8
4 10
5 18
5 19
5 7
6 8
6 24
6 26
6 27
7 18
7 27
7 23
8 9
8 13
8 22
8 29
9 27
10 20
10 27
10 29
11 18
11 21
12 16
12 17
13 18
14 21
14 22
16 20
16 29
17 20
20 21
21 28
Output
3 29 16
11 1 18
21 11 1
3 7 18
18 3 7
18 7 5
18 11 1
18 5 7
21 11 1
5 7 18
5 7 18
5 7 18
5 7 18
file cliquesmain.cpp in cliques folder of examples
The default value set for k in cliquesmain is k=2
which then is incremented by one here..
TCliqueOverlap::GetCPMCommunities(G, OverlapSz+1, CmtyV);
This is fine
but the problem is when user argument suppose k=5
then the algorithm finds cliques of k bcoz of increment
Thus, need to change default value of k to 3 (line 11)
and remove the increment (+1) in this line (line number 37). So change lines to these...
[line 11] const int OverlapSz = Env.GetIfArgPrefixInt("-k:", 3, "Min clique overlap");
[line 37] TCliqueOverlap::GetCPMCommunities(G, OverlapSz, CmtyV);
Location:
snap-core/ggen.h: 218
Assert checks that the desired number of edges to be created in the generated graph does not exceed the maximum number of edges possible, however since this arithmetic occurs on variables of type 'int', which are generally compiled to 32-bit integers even on x86_64 architectures, it is possible that even though there are enough nodes to fit the desires edges, the assertion will fail due to integer overflow. For instance, the error would be encountered when trying to generate a graph of size 1 million nodes and 1 million edges.
Replacing these with uint64_t will solve the problem, and some similar modifications within the function would be necessary to eliminate the resulting type casting warnings.
According to the documentation, Gnuplot is not a hard dependency of the SNAP library and is expected to be in the system path only when you try to plot structural properties of networks. But this doesn't seem to be the case, because even the act of linking against Snap.o
results in an error when starting the program if the system doesn't contain Gnuplot.
Example aa.cpp
:
#include <stdio.h>
int main() {
printf("foobar\n");
return(0);
}
Compile without Snap.o
:
$ g++ -std=c++98 -Wall -O3 -DNDEBUG -o aa aa.cpp -I../snap/snap-core -I../snap/snap-adv -I../snap/glib-core -I../snap/snap-exp -lrt
$ export PATH=""
$ ./aa
foobar
Compile with Snap.o
:
$ g++ -std=c++98 -Wall -O3 -DNDEBUG -o aa aa.cpp ../snap/snap-core/Snap.o -I../snap/snap-core -I../snap/snap-adv -I../snap/glib-core -I../snap/snap-exp -lrt
$ export PATH=""
$ ./aa
sh: 1: gnuplot: not found
foobar
In my opinion some initialization code of some objects is being executed before main()
and it tries to run something like system("gnuplot")
. Tested with g++ (Debian 4.7.2-5) 4.7.2.
Hi, When I tried the centrality
code example, I tried another datasets which has about 6 million nodes and 9 million edges, and the program just crashed, the error message was as follows
R6010
- abort() has been called
My OS is windows server 2008, with 192G RAM, VS10. Thx.
Hi Jure,
I recently downloaded the latest Version of Snap and tried to feed it with the latest Wikipedia (en Version) link network dump using netstat. It looks like the hop plots operation seems to fail for some reason if being applied (all other flags work fine). Any ideas why this happens?
Cheers,
Chris
Edit: I am running Mac OS X 10.8.2
Here my output:
./netstat -i:/Users/xt/wikipedia_links.txt -p:cdwsh
Input graph (one edge per line, tab/space separated) (-i:)=/Users/xt/wikipedia_links.txt
Directed graph (-d:)=Yes
Output file prefix (-o:)=graph
Title (description) (-t:)=
What statistics to plot string:
c: cummulative degree distribution
d: degree distribution
h: hop plot (diameter)
w: distribution of weakly connected components
s: distribution of strongly connected components
C: clustering coefficient
v: singular values
V: left and right singular vector
Loading...directed graph (TXT format)
/Users/xt/wikipedia_links.txt: Directed
Nodes: 27534914
Edges: 596307448
Zero Deg Nodes: 0
Zero InDeg Nodes: 15266343
Zero OutDeg Nodes: 353159
NonZero In-Out Deg Nodes: 11915412
Creating plots...
*** Error: Execution stopped: 0<=_Vals, file ../../glib-core/ds.h, line 443
*** Error: Execution stopped: ((this!=GetNullRStr())&&(Refs==0))||((this==GetNullRStr())&&(Refs==1)), file ../../glib-core/dt.h, line 351
When trying to use both SNAP and spdlog (https://github.com/gabime/spdlog), the compiler complains about ambiguous operators:
d:\develop\git\tspe\deploy\include\spdlog\details../sinks/file_sinks.h(169): error C2593: 'operator >=' is ambiguous
1> C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\include\thread(246): note: could be 'bool std::operator >=(std::thread::id,std::thread::id) noexcept' [found using argument-dependent lookup]
1> d:\develop\git\tspe\deploy\include\snap\bd.h(423): note: or 'bool operator >=std::chrono::system_clock::time_point(const TRec &,const TRec &)'
1> with
1> [
1> TRec=std::chrono::system_clock::time_point
1> ]
1> C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\include\chrono(683): note: or 'bool std::chrono::operator >=std::chrono::system_clock,std::chrono::system_clock::duration,std::chrono::system_clock::duration(const std::chrono::time_pointstd::chrono::system_clock,std::chrono::system_clock::duration &,const std::chrono::time_pointstd::chrono::system_clock,std::chrono::system_clock::duration &)'
1> d:\develop\git\tspe\deploy\include\spdlog\details../sinks/file_sinks.h(169): note: while trying to match the argument list '(std::chrono::system_clock::time_point, std::chrono::system_clock::time_point)'
1> d:\develop\git\tspe\deploy\include\spdlog\details../sinks/file_sinks.h(168): note: while compiling class template member function 'void spdlog::sinks::daily_file_sinkspdlog::details::null_mutex::_sink_it(const spdlog::details::log_msg &)'
1> C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\include\type_traits(391): note: see reference to class template instantiation 'spdlog::sinks::daily_file_sinkspdlog::details::null_mutex' being compiled
1> C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\include\memory(526): note: see reference to class template instantiation 'std::is_convertible<_Ty2 _,_Ty *>' being compiled
1> with
1> [
1> _Ty2=spdlog::sinks::daily_file_sink_st,
1> Ty=spdlog::sinks::sink
1> ]
1> d:\develop\git\tspe\deploy\include\spdlog\details/spdlog_impl.h(49): note: see reference to function template instantiation 'std::shared_ptrspdlog::logger spdlog::create<spdlog::sinks::daily_file_sink_st,std::string,const char,int,int,bool>(const std::string &,std::string,const char *,int,int,bool)' being compiled
1>
I tried to defrag a THashSet (TIntSet) and got an compile-error:
....
/snap/glib-core/shash.h:1364:7: error: no matching function for call to ‘THashSet::GetKey(int&, TInt&)’
I suppose in function: void THashSet<TKey, THashFunc>::Defrag()
GetKey(KeyId, Key); should be GetKey(KeyId);
File: shash.h in glib-core
Line: 1364
//Example:
int main(int argc, char* argv[]) {
//TIntSet is a typedef for THashSet
TIntSet* a_set = new TIntSet();
//do something with set that leaves it fragmented (IsKeyIdEqKeyN() will be false)
a_set->AddKey(4);
a_set->AddKey(2);
a_set->DelKey(2);
//compile error
//a_set->Defrag();
//this will not work with a fragmented set
a_set->DelKeyId(a_set->GetRndKeyId(TInt::Rnd));
delete a_set;
return 0;
}
I'm tryint to call TSnap::IsConnected(PGraph &) with a TPt<TNodeEDatNet<Tpair<int,int>,Tpair<int,int>>> as argument, and I thought it would implicitly convert it to a pointer to Graph, the base class. This is not the case and however I try I'm not able to assign a derived class to a snap base pointer. Why is that? How can I call IsConnected on a TNodeEDatNet graph?
I have an error when I use the method provided by snap to detect the community of a social network dataset. You can see the detail in the appendix, which includes the file named "cnm_test.py" that the source code about my test program and another file named "ca-CondMat.txt" which is a social network dataset and the last file named "log.txt" recording the output error information in my computer. I am looking forward to your reply. Thank you for your help!
#!/usr/bin/python
import sys
import snap
def create_graph_from_file(filename):
sf = open(filename)
lines = sf.readlines()
TG = snap.TUNGraph.New()
for item in lines:
if '#' in item:
continue
item = item[:-1]
item = item.split('\t')
if TG.IsNode(int(item[0])) == False:
TG.AddNode(int(item[0]))
if TG.IsNode(int(item[1])) == False:
TG.AddNode(int(item[1]))
TG.AddEdge(int(item[1]),int(item[0]))
print "load nodes : %d" % TG.GetNodes()
sf.close()
return TG
def main():
print "create graph start!!"
G = create_graph_from_file("./ca-CondMat.txt");
print "create graph finish!!"
CmtyV = snap.TCnComV()
modularity = snap.CommunityCNM(G,CmtyV)
ret_list = []
for Cmty in CmtyV:
temp = []
for NI in Cmty:
temp.append(NI)
ret_list.append(temp)
print "community size: %d " % len(ret_list)
return 0;
if name == "main":
main()
create graph start!!
load nodes : 23133
create graph finish!!
Traceback (most recent call last):
File "./cnm_test.py", line 40, in
main()
File "./cnm_test.py", line 29, in main
modularity = snap.CommunityCNM(G,CmtyV)
File "/usr/local/lib/python2.7/dist-packages/snap.py", line 35949, in CommunityCNM
return _snap.CommunityCNM(*args)
RuntimeError: Execution stopped: (0<=ValN)&&(ValN<Vals) [Reason:'Index:-1 Vals:23133 MxVals:23133 Type:4TVecI11THashKeyDatI4TIntN5TSnap11TSnapDetail11TCNMQMatrix8TCmtyDatEEiE'], file ../../snap/glib-core/ds.h, line 469
I am downloading snap-3.0.0 version, but cannot install it on Corn using pip install -user setup.py. It produces the following error:
"Collecting setup.py
/usr/local/lib/python2.7/dist-packages/pip/vendor/requests/packages/urllib3/util/ssl.py:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#snimissingwarning.
SNIMissingWarning
/usr/local/lib/python2.7/dist-packages/pip/vendor/requests/packages/urllib3/util/ssl.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Could not find a version that satisfies the requirement setup.py (from versions: )
No matching distribution found for setup.py
You are using pip version 8.1.2, however version 9.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
"
Am I not using the right command?
Hi there,
I'm running into a segmentation fault when using the magfit executable. I'm on Ubuntu 12.04LTS 64-bit. The error is as follows:
EStep iteration : 1
Program received signal SIGSEGV, Segmentation fault.
0x000000000040e8ac in TMAGFitBern::GetProbMu(int const&, int const&, int const&, int const&, int const&, bool, bool) const ()
A backtrace reveals:
(gdb) backtrace
#0 0x000000000040e8ac in TMAGFitBern::GetProbMu(int const&, int const&, int const&, int const&, int const&, bool, bool) const ()
#1 0x000000000040eb25 in TMAGFitBern::GetAvgThetaLL(int const&, int const&, int const&, bool, bool) const ()
#2 0x00000000004164da in TMAGFitBern::UpdateApxPhiMI(double const&, int const&, int const&, double&, TVVec&) ()
#3 0x00000000004189e9 in TMAGFitBern::DoEStepApxOneIter(TVec<TFlt, int> const&, TVVec&, double const&) ()
#4 0x00000000004209fd in TMAGFitBern::DoEMAlg(int const&, int const&, int const&, double const&, double const&, double const&, double const&, int const&) ()
#5 0x00000000004039b5 in main ()
Have been trying to use the 'LoadEdgeListStr' function so that I could map the actual node value to the assigned node ID. Didn't manage to do so however.
One probable suspect could be because the syntax "LoadEdgeList(PGraph, InFNm, SrcColId, DstColId, Separator)" is not the same as the code.
For example, if I run the command "snap.LoadEdgeListStr(snap.PUNGraph, "", 0, 1, H)", I keep getting an error message as below:
" TypeError: in method 'LoadEdgeListStr_PUNGraph', argument 4 of type 'TStrHash< TInt > &' "
But if I simply run "snap.LoadEdgeListStr( "", 0, 1, H)" (as referenced from the source code in http://snap.stanford.edu/snap/doc/snapdev-ref/db/d68/gio_8h_source.html) I get no error at all.
Is this actually a bug or am I missing something? Currently I'm using SNAP distribution "snap-1.1-2.3-Win-x64-py2.7"
Hi, guys
Did SNAP provides algorithms for calculating betweenness centrality for directed graphs ?
I have found 4 related functions, but all of them only accept undirected graphs as parameters, as followings:
void TSnap::GetBetweennessCentr (const PUNGraph &Graph, TIntFltH &NodeBtwH, const double &NodeFrac)'
void TSnap::GetBetweennessCentr (const PUNGraph &Graph, TIntPrFltH &EdgeBtwH, const double &NodeFrac)
void TSnap::GetBetweennessCentr (const PUNGraph &Graph, TIntFltH &NodeBtwH, TIntPrFltH &EdgeBtwH, const double &NodeFrac)
void TSnap::GetBetweennessCentr (const PUNGraph &Graph, const TIntV &BtwNIdV, TIntFltH &NodeBtwH, const bool &DoNodeCent, TIntPrFltH &EdgeBtwH, const bool &DoEdgeCent)
I use snap library without any problems for sometime now but lately I stumbled upon the following:
There are trwo macros in: glib-core/bd.h which sometimes cause problems
The problem is described here http://wordaligned.org/articles/macros-with-halos
I have the same exact issue. In some cases changing the include order fixes the problem but sometimes not. In order to work with snap, before I include my files I had to do:
undef min
undef max
The above solution works but it's still dirty.
When my students import a background that was larger than the stage, the background renders as "blurry." I think it's automatically resizing to the width? She imported an image that 960px wide. Here's the project.
Thanks for your help.
on snap.stanford.edu the file quicktest.py is broken. Line 12 needs to be altered to:
print ("SUCCESS, your version of Snap.py is ",version)
and line 14 needs parenthesis (Although I think this is only for python version 3+)
I have uploaded the fixed copy.
When running the examples the runtime reported increases as more threads are used. Example output:
Run with single thread:
$ ./bigclam
ragm. build: 14:58:44, Apr 8 2016. Time: 17:38:36 [Mar 24 2016]
================================================================
[...]
Graph: 6474 Nodes 13895 Edges
conductance computation completed [0.02s]
MLE completed with 550 iterations(4 secs)6, Diff: 16.180445 [4 secs]]]]]
run time: 11.27s (17:38:40)
Run with multiple threads:
$ ./bigclam -nt:24
ragm. build: 14:58:44, Apr 8 2016. Time: 17:38:49 [Mar 24 2016]
================================================================
[...]
Graph: 6474 Nodes 13895 Edges
conductance computation completed [0.02s]
MLE completed with 319 iterations(1 secs)Community vector generated. 2 communities are ommitted
run time: 26.06s (17:38:51)
Due to problem with gcc ( Issue #26 ), I tried to compile snap with clang.
I got error in bigclam target because clang didn't support openmp.
bigclam.cpp:6:10: fatal error: 'omp.h' file not found
$ clang++ -v
Ubuntu clang version 3.2-1~exp9ubuntu1 (tags/RELEASE_32/final) (based on LLVM 3.2)
Target: x86_64-pc-linux-gnu
Thread model: posix
It might be worth considering upgrading SNAP to take advantage of the Autoconf/Automake system to allow SNAP to more easily utilize third party dependencies like libgefx and libhdfs.
It would also allow SNAP to be more portable, and use the standard make command:
./configure && make && make install
For the example netstat program, with the following input file
1 2
1 3
2 3
We find that for both options -d:T or -d:F the average clustering coefficient to be 1.0, but for the directed version this should be 0.5.
I have a very simple code that the AddNode function creates the following error for me:
*** Error: Execution stopped: !IsNode(NId) [Reason:'NodeId 1 already exists'], file graph.cpp, line 12
The problematic line :
if (!Graph->IsNode(srcNId)) Graph->AddNode(srcNId);
This totally fine by compiling under Windows. But, I must compile my program on a cluster that works with linux (sharcnet.ca), unfortunately SNAP does not fit with their system, there are several problems I found out. For example, using just GenGrid function gave me segmentation fault.
This is the whole code:
PUNGraph Graph = TUNGraph::New();
TSsParser parser("grid.txt", ssfWhiteSep, true, true, true);
while (parser.Next()) {
const int srcNId = parser.GetInt(0);
const int dstNId = parser.GetInt(1);
if (!Graph->IsNode(srcNId)) Graph->AddNode(srcNId);
if (!Graph->IsNode(dstNId)) Graph->AddNode(dstNId);
if (!Graph->IsEdge(srcNId, dstNId))
Graph->AddEdge(srcNId, dstNId);
}
Thanks in advance.
Hi all,
I just upgraded my snap.py to the newest version, but has below problems when importing the package
/Users/danqing0703/anaconda/lib/python2.7/site-packages/snap.py in <module>()
4726 TDbStr.__eq__ = new_instancemethod(_snap.TDbStr___eq__,None,TDbStr)
4727 TDbStr.__lt__ = new_instancemethod(_snap.TDbStr___lt__,None,TDbStr)
-> 4728 TDbStr.GetStr = new_instancemethod(_snap.TDbStr_GetStr,None,TDbStr)
4729 TDbStr.GetPrimHashCd = new_instancemethod(_snap.TDbStr_GetPrimHashCd,None,TDbStr)
4730 TDbStr.GetSecHashCd = new_instancemethod(_snap.TDbStr_GetSecHashCd,None,TDbStr)
AttributeError: 'module' object has no attribute 'TDbStr_GetStr'
Would really appreciate it if someone could take some time to answer this , thanks!
The input for netevol is like graph*.txt, which is supposed to read the evolving graphs as in order of graph1.txt, graph2.txt, graph3.txt
but in linux the readdir gives the filename in random order, which produce incorrect results.
Temporary work around: get file in list and sort them
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.