Comments (8)
You can certainly use the framework for that albeit an EFGraph might be more appropriate. We never worked on that tho...
from webgraph-big.
We found that many papers cites the layered label propagation paper, among which includes graph processing system like GraphX/Gemini/PowerGraph and graph neural network system like P3. Specially, many of them use dataset like uk-2007-05. However, we found those who opened their source code like GraphX/Gemini system tried to transfer the .graph file into <source id, target id> txt format via ArcListASCIIGraph class. After this operation, the graph data loaded into memory is not compressed. So when the graph data is large, there may cause out of memory issue, but we thought what WebGraph/LLP really aims to do is to resolve this memory issue when facing huge large graph. That's really confused me, do you have any suggestion?
from webgraph-big.
We distribute data in the WebGraph format, and a lot of people use the data, but the same people does not want to use the WebGraph interface, so they convert to ASCII. Sad, but it works for them apparently.
from webgraph-big.
We also observed an experiment that we list the figure as follow and it's from the paper of graph processing system Gemini.
Hereby they set up an experiment for their called chunk-based partitioning method which partition graph with load balance. Although they use the <source id, target id> txt format rather than .graph format, the results show that locality property of uk-2007-05 graph data is very good which helps getting a better performance. So I think although graph data with <source id, target id> txt format is not compressed but the node ordering still helped to maintain the locality property, right? Sorry I have not totally understand the theory of WebGraph or LLP, so need your help to check the judgement.
from webgraph-big.
Totally. The ordering of the graph is a form of clustering. Chunks of consecutive nodes have dense connections.
from webgraph-big.
Great thx.
from webgraph-big.
Just one idea, I observed that BVGraph/EVGraph/ASCIIGraph class provide many load and store methods, I think maybe WebGraph could provide more methods for fast random access to a link , such as fast access to neighbors or subgraph for one node. If so, maybe people would more likely trends to use the WebGraph interface rather than convert to ASCII to get coo data format which is not compressed.
from webgraph-big.
I think you really misunderstood what WebGraph does. Please have a look at ImmutableGraph.
from webgraph-big.
Related Issues (1)
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from webgraph-big.