matrix interactions too large - cannot handle in memory

[liz-is]

matrix interactions too large - cannot handle in memory

[blajoie]

Hi - I will remove that hard limit in the code.

For this code (matrix2insulation) the memory usage will scale with your selected insulation square size * nrow/ncol.

So memory usage should always be low [MBs] given normal usage.
We only load into memory N diagonals up to your specified insulation square size.

e.g.
1,000,000bp square size for a 5KB binned matrix would load into memory ~400 diagonals.

This script is more intended for matrices < 10,000 x 10,000.
For the higher resolution data and larger matrices, a more efficient data format (sparse, hdf5, etc) would be more ideal, though this script may still work.

I tested the code on a ~50,000 x ~50,000 matrix, it takes quite a bit of time (15 minutes) to run. Mainly to simply read the gzipped txt file, but it does complete AOK.

Script will work for any species, as long as the matrix is binned into equal sized intervals, and the intervals are formatted in the my5C format.
https://github.com/blajoie/crane-nature-2015/wiki
e.g. bin1|hg19|chr1:40000-80000

The code is updated now, give it another shot!

[blajoie]

issue looks to be fixed now! closing this.