Comments (7)
Hi Misha,
I don't think we have ever systematically checked this, @kaukrise @sgalan , please correct me if I am wrong.
Of the factors that you mentioned, the matrix resolution has the strongest influence on the runtime, simply because of the quadratic growth of the amount of data with decreasing bin size. The runtime should just approximately linearly depend on the number of windows (controlled by step and genome size). I am unsure about the effect of the window size; I think this mostly influences the runtime of the comparison step (probably quadratically), which involves subsetting the data that is already in memory and computing the similarity score; I guess the time here grows quadratically with the window size, but this step is very fast, so the overall runtime might not change so much because of this.
@kaukrise , do you have anything more helpful on this?
from chess.
In the paper we have one example in Extended Data Figure 6. chess sim
took 100 seconds to process human chr19 on a single cpu on a Intel Xeon W, 3GHz with 128 gb ram, with a 1 mb window and 500 kb step size.
The data was from Bonev et al 2017, binned at 5 kb.
from chess.
I don't have too much to add, except that the O/E calculation is by far the most time-consuming step. If you want to run many CHESS calculations on the same dataset, it is strongly recommended to convert your data beforehand. Either to FAN-C (you could use fanc from-cooler
or fanc from-txt
) or Juicer format, which is very flexible, or using the chess oe
command if you are working with sparse TXT matrices.
from chess.
Ok, thanks.
I am currently trying this on 20 and 50 kb resolution for the whole human genome, so let's see. What I noticed so far is that the longest part is loading reference/query contact matrices.
@kaukrise, yes, I had to switch to .hic format, since .cool is indeed very time-consuming. What I can actually suggest about this, is to use pre-computed obs/exp matrix for .cool format, which can be easily generated by cooltools: https://github.com/open2c/cooltools. Maybe you can consider this in a future.
from chess.
If you already have O/E matrices, even in .cool
format, you should be able to use the --oe-input
flag to skip the O/E conversion, isn't that right @nickmachnik?
from chess.
@kaukrise, as far as I know, .cool format does not store obs/exp counts, only raw/corrected matrix itself. Therefore, it is rather hard to overcome this and create a new .cool with obs/exp counts only.
from chess.
Okay, then I misunderstood your comment. I thought cooltools allowed you to generate a .cool
file that only contains O/E values (as matrix entries).
from chess.
Related Issues (20)
- Get observed/expected from Juicer Hi-C HOT 2
- chess --version doesn't work?
- CNV bias in normalization HOT 2
- Conditions for conservation analysis of syntenic blocks HOT 5
- Nan Continued HOT 2
- No valid region pairs found? HOT 1
- Different resolution produce different result HOT 1
- Should the users be concerned about the problem raised in the new Contradictory Results bioRxiv preprint? HOT 2
- conservation analysis when only a few syntenic blocks are available HOT 3
- speed up the chess run HOT 1
- error of the chess extract HOT 3
- issue of normalized/chess extract HOT 1
- error on running chess sim HOT 2
- error when running extract on .hic files HOT 1
- something different from plotting HOT 9
- _pickle.PicklingError HOT 2
- chess extract error: operands could not be broadcast together with shapes HOT 1
- data_range parameter not specified - error HOT 7
- Chess sim output .tsv file explained HOT 1
- Normalization of .hic files HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from chess.