Comments (16)
Yeah - I changed the break reads to 0.4
Will try with the fast basecaller and report back
from readfish.
Hi,
This doesn't look like a Guppy issue - could you confirm what version of MinKNOW you are running? Make sure you do this from the GridION - click on MinKNOW and "about MinKNOW".
You should get something like below:
GridION Release 19.12.6
MinKNOW Core 3.6.5
GUI 3.6.16
Guppy 3.2.10
Bream 4.3.16
Script Config 1.0.9
Note that we use Guppy 3.4 on the GridION now by running a separate Guppy Basecall Server using version 3.4.5 on a distinct port. It works - at least for us - very well.
Let us know your MinKNOW version and we will go from there!
Thanks,
Matt
from readfish.
Thanks! I think my problem might be my understanding of how this works - so in addition to the standard guppy server running as part of minknow you need an additional guppy basecall server that ru uses?
from readfish.
Hi Matt,
So this is confusing.
Because ONT released the pyGuppy client for a version of Guppy which isn't yet integrated into MinKNOW on the GridION you currently need to run two guppy basecall servers. Once ONT integrate Guppy3.4 into GridION MinKNOW all will be good.
So - I think we now understand what you are doing - presumably you are running read until itself on a remote computer - i.e not on the GridION itself. That is OK, but we suspect the version of MinKNOW that you have installed on your local machine differs to the version installed on the GridION. We use some code from MinKNOW and we pull it from ONTs installation rather than packaging it ourselves.
So - if you install our RU code on your GridION you can either set up a second basecall server on the GridION or you can point to a remote guppy basecall server (but network latency may be an issue).
You should be able to do as you are trying using host and port but we haven't explicitly tested this AND you must have matching MinKNOW versions on the two machines. We will flag this as something to test for in a future update of the code.
Check the MinKNOW version on your local machine and see if that differs to the GridION - if you update to the latest version you should be compatible and the error message you see above should go away. You may need to uninstall and reinstall RU and read_until_api_v2 to make sure this change is picked up.
Hope this helps.
I'll leave this open whilst you are testing - please let us know how you get on.
from readfish.
Yes currently I am running ru and now a basecall server from a remote machine that is not the GridION (for some reason I could not get the second part of the tutorial to work without starting a guppy server on a new port? Maybe because it's not 3.4?)
Anyway this is the version of MinKNOW on the remote machine
What I was hoping is that I could leave the GridION untouched and run ru and guppy on the remote machine and just specify the GridION as the host when starting ru.
I'll try your suggestion with ru and guppy 3.4 on a gridion (we have the old one laying around waiting for collection that I could play around with :))
from readfish.
@mattdmem could you post your command that you were using eg ru_generators ...
and your TOML file? You can clip out any paths/IPs that you don't want to post.
from readfish.
So if I run locally, with a MinION with config cell, MinION release 19.12.5 without a separate basecall server:
ru_generators --device MN33331 --experiment-name "RU_TEST4" --toml human_chr_selection.toml --log-level debug
[caller_settings]
config_name = "dna_r9.4.1_450bps_hac"
host = "127.0.0.1"
port = 5555
[conditions]
reference = "/data/reference/hg38.mmi"
[conditions.0]
name = "select_chr_21_22"
control = false
min_chunks = 0
max_chunks = inf
targets = ["chr21", "chr22"]
single_on = "stop_receiving"
multi_on = "stop_receiving"
single_off = "unblock"
multi_off = "unblock"
no_seq = "proceed"
no_map = "proceed"
MinKNOW reports that read until is controlling sequencing but nothing happens:
2020-03-03 15:54:24,655 read_until_api_v2.main Client type: many chunk
2020-03-03 15:54:24,655 read_until_api_v2.main Cache type: AccumulatingCache
2020-03-03 15:54:24,655 read_until_api_v2.main Filter for classes: strand and adapter
2020-03-03 15:54:24,655 read_until_api_v2.main Creating rpc connection for device MN33331.
2020-03-03 15:54:25,016 read_until_api_v2.main Loaded RPC
2020-03-03 15:54:25,026 read_until_api_v2.main Signal data-type: int16
2020-03-03 15:54:25,035 Manager Creating 1 workers
2020-03-03 15:54:25,035 Manager Creating 1 workers
2020-03-03 15:54:25,036 read_until_api_v2.main Processing started
2020-03-03 15:54:25,036 read_until_api_v2.main Sending init command, channels:1-512, min_chunk:0
It just hangs there and no other output as the sequencing can be seen to continue.
If I run a basecall server (3.4) locally on port 5556:
ru_generators --device MN33331 --experiment-name "RU_TEST4" --toml human_chr_selection.toml --log-level debug
[caller_settings]
config_name = "dna_r9.4.1_450bps_hac"
host = "127.0.0.1"
port = 5556
[conditions]
reference = "/data/reference/hg38.mmi"
[conditions.0]
name = "select_chr_21_22"
control = false
min_chunks = 0
max_chunks = inf
targets = ["chr21", "chr22"]
single_on = "stop_receiving"
multi_on = "stop_receiving"
single_off = "unblock"
multi_off = "unblock"
no_seq = "proceed"
no_map = "proceed"
Then it seems to work - I get mapping results
2020-03-03 16:23:47,341 read_until_api_v2.main Processing started
2020-03-03 16:23:47,341 read_until_api_v2.main Sending init command, channels:1-512, min_chunk:0
2020-03-03 16:24:36,313 DEC client_iteration read_in_loop read_id channel read_number seq_len counter mode decision condition min_threshold count_threshold start_analysis end_analysis timestamp
2020-03-03 16:24:36,674 PAF 1c4e57c2-1fc5-438d-a7be-6b256d8b09bf 311 0 311 - chr7 159345973 97673668 97673979 295 32460 tp:A:P ts:A:. cg:Z:13M1D8M1D11M2I2M1D44M2I17M1D10M2I45M2I28M1I59M1D14M2I7M2I2M3D8M1D15M4D15M
2020-03-03 16:24:36,674 DEC 1 1 1c4e57c2-1fc5-438d-a7be-6b256d8b09bf 348 194 311 1 single_off unblock select_chr_21_22 False False 9894.185407268 9894.185898751 1583252676.674857
Do you want to see the commands I was using for the GridION? It's pretty much the same but with the host and port of the GirdION specified.
from readfish.
So in the first instance, this is symptomatic of the base call server being the wrong version; and if using the one that is started by MinKNOW on the GridION this will be the wrong version.
There is the possibility that Read Until can be run remotely from the sequencing machine. This would be achieved by something like:
Running on Computer A:
# sequencing-machine-ip should point to computer B
ru_generators --toml exp.toml --exp test --device X1 --host <sequencing-machine-ip>
exp.toml
[caller_settings]
config_name = "..."
# Should point to the guppy server on computer A which should be local
host = "127.0.0.1"
port = 5555
This would allow you to run Read Until and a Guppy server on computer A and sequencing on computer B. Though network latency may reduce the efficiency.
from readfish.
Thanks!
So if I were to install ru and guppy 3.4 on a gridion I would just follow your instructions in the README? This shouldn't affect the current install of MinKNOW?
from readfish.
Yes, you should be fine. Just make sure that the TOML file points towards the guppy 3.4 server!
from readfish.
OK - I have this working on our gridion - I think. The read size looks a little bigger than your example - is this OK? The peak is a little higher:
command
ru_generators --device X5 --experiment-name RU_TEST2 --toml human_chr_selection.toml --log-file ru_test.log --log-level info
toml
[caller_settings]
config_name = "dna_r9.4.1_450bps_hac"
host = "127.0.0.1"
port = 5556 #this is the port I started the guppy 3.4 server on
[conditions]
reference = "/data/hg38.mmi"
[conditions.0]
name = "select_chr_21_22"
control = false
min_chunks = 0
max_chunks = inf
targets = ["chr21", "chr22"]
single_on = "stop_receiving"
multi_on = "stop_receiving"
single_off = "unblock"
multi_off = "unblock"
no_seq = "proceed"
no_map = "proceed"
ru output:
2020-03-04 11:27:04,411 ru.ru_gen 111R/1.82324s
2020-03-04 11:27:07,057 ru.ru_gen 42R/2.64603s
2020-03-04 11:27:09,505 ru.ru_gen 104R/2.44856s
2020-03-04 11:27:11,174 ru.ru_gen 52R/1.66827s
2020-03-04 11:27:13,363 ru.ru_gen 38R/2.18883s
2020-03-04 11:27:15,609 ru.ru_gen 105R/2.24586s
2020-03-04 11:27:18,171 ru.ru_gen 41R/2.56186s
2020-03-04 11:27:20,821 ru.ru_gen 104R/2.64991s
2020-03-04 11:27:22,802 ru.ru_gen 72R/1.98141s
2020-03-04 11:27:24,977 ru.ru_gen 75R/2.17375s
2020-03-04 11:27:26,813 ru.ru_gen 36R/1.83636s
2020-03-04 11:27:28,522 ru.ru_gen 89R/1.70878s
2020-03-04 11:27:30,897 ru.ru_gen 53R/2.37496s
from readfish.
The unblock peaks do look a bit higher and we would prefer them to be shorter. Have you changed the break_reads_after_seconds
parameter in the sequencing_*.toml
file to 0.4?
Also, you may find that the fast caller is better here, eg config_name = "dna_r9.4.1_450bps_fast"
from readfish.
Okay so it's much faster now but the peak is still higher - at least it's not bimodal now:
2020-03-04 11:40:40,848 ru.ru_gen 22R/0.09533s
2020-03-04 11:40:41,077 ru.ru_gen 6R/0.22446s
2020-03-04 11:40:41,231 ru.ru_gen 8R/0.15333s
2020-03-04 11:40:41,484 ru.ru_gen 14R/0.25298s
2020-03-04 11:40:41,606 ru.ru_gen 8R/0.12171s
2020-03-04 11:40:41,676 ru.ru_gen 6R/0.07043s
2020-03-04 11:40:41,838 ru.ru_gen 9R/0.13179s
2020-03-04 11:40:41,925 ru.ru_gen 3R/0.08752s
2020-03-04 11:40:41,985 ru.ru_gen 4R/0.04765s
2020-03-04 11:40:42,134 ru.ru_gen 3R/0.09587s
2020-03-04 11:40:42,305 ru.ru_gen 7R/0.16683s
2020-03-04 11:40:42,438 ru.ru_gen 9R/0.13263s
2020-03-04 11:40:42,552 ru.ru_gen 7R/0.11415s
from readfish.
Is this running on GridION or GridION MK1?
from readfish.
GridION (the old one)
Starting to see a smaller peak now 13mins in:
from readfish.
So after 1hr 40mins the peak around 500-600b is much bigger:
Is this acceptable? Is it purely due to the original GridION having an inferior graphics card to the Mk1?
from readfish.
Related Issues (20)
- readfish with just depletion? HOT 4
- Not being able to finish the tutorial/test HOT 6
- Provide additional TOML files which provide examples of host depletion and other use cases for readfish.
- location of readfish output HOT 4
- toml_live handling issue
- Depletion setting and unexpected behavior HOT 2
- Question about Readfish basecalling performance for messy samples HOT 6
- Playback issue HOT 14
- Problem communicating with basecaller after MinKnow update HOT 10
- Detect MinKNOW version and warn. HOT 1
- Circular Binary Segmentation HOT 3
- Having only one region in toml caused issue HOT 13
- chore: update github workflow actions HOT 1
- Handshake failed with fatal error SSL_ERROR_SSL: error:1000007d:SSL routines:OPENSSL_internal:CERTIFICATE_VERIFY_FAILED HOT 4
- comparison between MinKNOW and Readfish HOT 4
- Error with flongle flow cell splitting HOT 6
- chore: update FAQ
- next release candidate checklist
- High count of no-barcode that map to my targets HOT 15
- Readfish is not running for Flongle Playback Data HOT 9
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from readfish.