wtsi-hgi / 13amp Goto Github PK
View Code? Open in Web Editor NEWUserspace CRAM-to-BAM translation virtual filesystem
License: GNU General Public License v3.0
Userspace CRAM-to-BAM translation virtual filesystem
License: GNU General Public License v3.0
When trying to access members of the FUSE context structure, within threads, a segmentation fault is raised... This is most likely due to me doing something stupid, given my basic knowledge of pthreads!
If the source directory contains the mount directory, then bad things happen if you try to access the mounted mount. Presumably FUSE gets stuck in an infinite loop, but it bails out and crashes after a timeout.
Ideally, one would check to see if the source directories within the mount resolve to the mount directory. However, for this to work, you would need to know the mount point (i.e., to compare against). Unfortunately, this is no way to ascertain this from within FUSE -- as it's not necessarily static or unique (see [1]) -- and presumably no portable way to get it from within the host OS.
Unless I'm missing a trick, this is in wontfix
territory!
[1] http://fuse.996288.n3.nabble.com/Getting-the-mountpoint-from-within-a-fusemod-td11620.html
The pread
call on regular (i.e., non-virtual BAM files) is filling the buffer with zeros...albeit the correct number of them.
Running ./autogen.sh
in a fresh git checkout fails because bootstrap looks for the presence of README-hacking
as a sign that it is a git checkout rather than a tarball distribution.
Easiest solution is to check-in a README-hacking. Alternatively, you could find another file that won't be present in the tar dist and modify the bootstrap script to check for that instead.
$ 13amp /bam -S /cram
$ ls -l /bam
So far the 13amp process has consumed 82m of CPU time (at 100%+ utillisation) but it has not returned anything for the directory listing yet.
13amp is running inside a docker container where '/cram' is a bind-mounted directory containing a single ~24GiB cram:
$ ls -l /lustre/scratch115/teams/hgi/users/jr17/pomak_merged_cram_50/*.cram
-rw-r--r-- 1 mercury hgi 25027902374 Aug 24 17:47 /lustre/scratch115/teams/hgi/users/jr17/pomak_merged_cram_50/14106455.CCXX.paired310.1898143677.cram
It appears this is simply an issue of HTSLIB taking 3+ hours to fully stream through a 24GiB CRAM file. Suggest we have a "fast" directory listing mode that lies about the size of the virtual BAM files.
The read loop will consume the whole converted file, even if it has fetched the requested chunk. Break the loop (which will close the pipe, etc. and return) once the fetched size is equal to the requested size.
The file size of the injected virtual BAM file must be accurate, so the OS can send the EOF signal when reading. Other FUSE conversion filesystems do this by either:
We will have to implement option 2 in cramp_getattr
. Precalculating may be far too expensive to be viable; even in getattr
, it might just make everything too slow...
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.