Installing on GridION

[looselab]

Installing on GridION

[mattdmem]

Yeah - I changed the break reads to 0.4

Will try with the fast basecaller and report back

[mattloose]

Hi,

This doesn't look like a Guppy issue - could you confirm what version of MinKNOW you are running? Make sure you do this from the GridION - click on MinKNOW and "about MinKNOW".

You should get something like below:

GridION Release 19.12.6

MinKNOW Core 3.6.5
GUI 3.6.16
Guppy 3.2.10
Bream 4.3.16
Script Config 1.0.9

Note that we use Guppy 3.4 on the GridION now by running a separate Guppy Basecall Server using version 3.4.5 on a distinct port. It works - at least for us - very well.

Let us know your MinKNOW version and we will go from there!

Thanks,

Matt

[mattdmem]

Thanks! I think my problem might be my understanding of how this works - so in addition to the standard guppy server running as part of minknow you need an additional guppy basecall server that ru uses?

[mattloose]

Hi Matt,

So this is confusing.

Because ONT released the pyGuppy client for a version of Guppy which isn't yet integrated into MinKNOW on the GridION you currently need to run two guppy basecall servers. Once ONT integrate Guppy3.4 into GridION MinKNOW all will be good.

So - I think we now understand what you are doing - presumably you are running read until itself on a remote computer - i.e not on the GridION itself. That is OK, but we suspect the version of MinKNOW that you have installed on your local machine differs to the version installed on the GridION. We use some code from MinKNOW and we pull it from ONTs installation rather than packaging it ourselves.

So - if you install our RU code on your GridION you can either set up a second basecall server on the GridION or you can point to a remote guppy basecall server (but network latency may be an issue).

You should be able to do as you are trying using host and port but we haven't explicitly tested this AND you must have matching MinKNOW versions on the two machines. We will flag this as something to test for in a future update of the code.

Check the MinKNOW version on your local machine and see if that differs to the GridION - if you update to the latest version you should be compatible and the error message you see above should go away. You may need to uninstall and reinstall RU and read_until_api_v2 to make sure this change is picked up.

Hope this helps.

I'll leave this open whilst you are testing - please let us know how you get on.

[mattdmem]

Yes currently I am running ru and now a basecall server from a remote machine that is not the GridION (for some reason I could not get the second part of the tutorial to work without starting a guppy server on a new port? Maybe because it's not 3.4?)

Anyway this is the version of MinKNOW on the remote machine

What I was hoping is that I could leave the GridION untouched and run ru and guppy on the remote machine and just specify the GridION as the host when starting ru.

I'll try your suggestion with ru and guppy 3.4 on a gridion (we have the old one laying around waiting for collection that I could play around with :))

[alexomics]

@mattdmem could you post your command that you were using eg ru_generators ... and your TOML file? You can clip out any paths/IPs that you don't want to post.

[mattdmem]

So if I run locally, with a MinION with config cell, MinION release 19.12.5 without a separate basecall server:

ru_generators --device MN33331 --experiment-name "RU_TEST4" --toml human_chr_selection.toml --log-level debug

[caller_settings]
config_name = "dna_r9.4.1_450bps_hac"
host = "127.0.0.1"
port = 5555

[conditions]
reference = "/data/reference/hg38.mmi"

[conditions.0]
name = "select_chr_21_22"
control = false
min_chunks = 0
max_chunks = inf
targets = ["chr21", "chr22"]
single_on = "stop_receiving"
multi_on = "stop_receiving"
single_off = "unblock"
multi_off = "unblock"
no_seq = "proceed"
no_map = "proceed"

MinKNOW reports that read until is controlling sequencing but nothing happens:

2020-03-03 15:54:24,655 read_until_api_v2.main Client type: many chunk
2020-03-03 15:54:24,655 read_until_api_v2.main Cache type: AccumulatingCache
2020-03-03 15:54:24,655 read_until_api_v2.main Filter for classes: strand and adapter
2020-03-03 15:54:24,655 read_until_api_v2.main Creating rpc connection for device MN33331.
2020-03-03 15:54:25,016 read_until_api_v2.main Loaded RPC
2020-03-03 15:54:25,026 read_until_api_v2.main Signal data-type: int16
2020-03-03 15:54:25,035 Manager Creating 1 workers
2020-03-03 15:54:25,035 Manager Creating 1 workers
2020-03-03 15:54:25,036 read_until_api_v2.main Processing started
2020-03-03 15:54:25,036 read_until_api_v2.main Sending init command, channels:1-512, min_chunk:0

It just hangs there and no other output as the sequencing can be seen to continue.

If I run a basecall server (3.4) locally on port 5556:

ru_generators --device MN33331 --experiment-name "RU_TEST4" --toml human_chr_selection.toml --log-level debug

[caller_settings]
config_name = "dna_r9.4.1_450bps_hac"
host = "127.0.0.1"
port = 5556

[conditions]
reference = "/data/reference/hg38.mmi"

[conditions.0]
name = "select_chr_21_22"
control = false
min_chunks = 0
max_chunks = inf
targets = ["chr21", "chr22"]
single_on = "stop_receiving"
multi_on = "stop_receiving"
single_off = "unblock"
multi_off = "unblock"
no_seq = "proceed"
no_map = "proceed"

Then it seems to work - I get mapping results

2020-03-03 16:23:47,341 read_until_api_v2.main Processing started
2020-03-03 16:23:47,341 read_until_api_v2.main Sending init command, channels:1-512, min_chunk:0
2020-03-03 16:24:36,313 DEC client_iteration	read_in_loop	read_id	channel	read_number	seq_len	counter	mode	decision	condition	min_threshold	count_threshold	start_analysis	end_analysis	timestamp
2020-03-03 16:24:36,674 PAF 1c4e57c2-1fc5-438d-a7be-6b256d8b09bf	311	0	311	-	chr7	159345973	97673668	97673979	295	32460	tp:A:P	ts:A:.	cg:Z:13M1D8M1D11M2I2M1D44M2I17M1D10M2I45M2I28M1I59M1D14M2I7M2I2M3D8M1D15M4D15M
2020-03-03 16:24:36,674 DEC 1	1	1c4e57c2-1fc5-438d-a7be-6b256d8b09bf	348	194	311	1	single_off	unblock	select_chr_21_22	False	False	9894.185407268	9894.185898751	1583252676.674857

Do you want to see the commands I was using for the GridION? It's pretty much the same but with the host and port of the GirdION specified.

[alexomics]

So in the first instance, this is symptomatic of the base call server being the wrong version; and if using the one that is started by MinKNOW on the GridION this will be the wrong version.

There is the possibility that Read Until can be run remotely from the sequencing machine. This would be achieved by something like:

Running on Computer A:

# sequencing-machine-ip should point to computer B
ru_generators --toml exp.toml --exp test --device X1 --host <sequencing-machine-ip> 

exp.toml

[caller_settings]
config_name = "..."
# Should point to the guppy server on computer A which should be local
host = "127.0.0.1"
port = 5555

This would allow you to run Read Until and a Guppy server on computer A and sequencing on computer B. Though network latency may reduce the efficiency.

[mattdmem]

Thanks!

So if I were to install ru and guppy 3.4 on a gridion I would just follow your instructions in the README? This shouldn't affect the current install of MinKNOW?

[alexomics]

Yes, you should be fine. Just make sure that the TOML file points towards the guppy 3.4 server!

[mattdmem]

OK - I have this working on our gridion - I think. The read size looks a little bigger than your example - is this OK? The peak is a little higher:

command

ru_generators --device X5 --experiment-name RU_TEST2 --toml human_chr_selection.toml --log-file ru_test.log --log-level info

toml

[caller_settings]
config_name = "dna_r9.4.1_450bps_hac"
host = "127.0.0.1"
port = 5556 #this is the port I started the guppy 3.4 server on

[conditions]
reference = "/data/hg38.mmi"

[conditions.0]
name = "select_chr_21_22"
control = false
min_chunks = 0
max_chunks = inf
targets = ["chr21", "chr22"]
single_on = "stop_receiving"
multi_on = "stop_receiving"
single_off = "unblock"
multi_off = "unblock"
no_seq = "proceed"
no_map = "proceed"

ru output:

2020-03-04 11:27:04,411 ru.ru_gen 111R/1.82324s
2020-03-04 11:27:07,057 ru.ru_gen 42R/2.64603s
2020-03-04 11:27:09,505 ru.ru_gen 104R/2.44856s
2020-03-04 11:27:11,174 ru.ru_gen 52R/1.66827s
2020-03-04 11:27:13,363 ru.ru_gen 38R/2.18883s
2020-03-04 11:27:15,609 ru.ru_gen 105R/2.24586s
2020-03-04 11:27:18,171 ru.ru_gen 41R/2.56186s
2020-03-04 11:27:20,821 ru.ru_gen 104R/2.64991s
2020-03-04 11:27:22,802 ru.ru_gen 72R/1.98141s
2020-03-04 11:27:24,977 ru.ru_gen 75R/2.17375s
2020-03-04 11:27:26,813 ru.ru_gen 36R/1.83636s
2020-03-04 11:27:28,522 ru.ru_gen 89R/1.70878s
2020-03-04 11:27:30,897 ru.ru_gen 53R/2.37496s

[alexomics]

The unblock peaks do look a bit higher and we would prefer them to be shorter. Have you changed the break_reads_after_seconds parameter in the sequencing_*.toml file to 0.4?
Also, you may find that the fast caller is better here, eg config_name = "dna_r9.4.1_450bps_fast"

[mattdmem]

Okay so it's much faster now but the peak is still higher - at least it's not bimodal now:

2020-03-04 11:40:40,848 ru.ru_gen 22R/0.09533s
2020-03-04 11:40:41,077 ru.ru_gen 6R/0.22446s
2020-03-04 11:40:41,231 ru.ru_gen 8R/0.15333s
2020-03-04 11:40:41,484 ru.ru_gen 14R/0.25298s
2020-03-04 11:40:41,606 ru.ru_gen 8R/0.12171s
2020-03-04 11:40:41,676 ru.ru_gen 6R/0.07043s
2020-03-04 11:40:41,838 ru.ru_gen 9R/0.13179s
2020-03-04 11:40:41,925 ru.ru_gen 3R/0.08752s
2020-03-04 11:40:41,985 ru.ru_gen 4R/0.04765s
2020-03-04 11:40:42,134 ru.ru_gen 3R/0.09587s
2020-03-04 11:40:42,305 ru.ru_gen 7R/0.16683s
2020-03-04 11:40:42,438 ru.ru_gen 9R/0.13263s
2020-03-04 11:40:42,552 ru.ru_gen 7R/0.11415s

[alexomics]

Is this running on GridION or GridION MK1?

[mattdmem]

GridION (the old one)

Starting to see a smaller peak now 13mins in:

[mattdmem]

So after 1hr 40mins the peak around 500-600b is much bigger:

Is this acceptable? Is it purely due to the original GridION having an inferior graphics card to the Mk1?