Comments (3)
Thank you @JoseCorCab. We will talk about creating a cached search that you can save/load/filter later. That's a good idea for large searches you can do more with the results without rerunning them. I think you might be able to accomplish this by running the searches repeatedly and joining the tables into a larger table that you can write to a file. You could do this ahead of time off of the cluster and write the table to a text file or .rData file that you can read when you start running on the cluster. This would effectively accomplish what you are requesting if I'm understanding correctly. If you have trouble with any of the queries timing out please provide an example and I will look into it. The data transfer can sometimes take extra time but any of the individual queries should be relatively quick. If they aren't quick then I need to also look at that problem.
For a custom query you should be able to submit one. Please look at the documentation for search_multimir(query). This should allow you to create your own queries and submit them. I understand the desire to create a local copy from those results that you can further query. I don't think it's technically challenging to implement this. I will bring this up as well. Thank you.
from multimir.
One other idea based on another request is maybe we can offer the option to store the result with a unique ID on the server so you could just retreive the results. However I think the transfer of a large query should usually be the slower part of the query.
from multimir.
Hello, thanks for the suggestion. My problem arises because a R loop is taken as a unique job in the cluster and the login node has a time limit for each job. I will try to make small individual queries (inside of a R script) inside a bash loop. In this way each query is a different job and it shouldn't break.
One other idea based on another request is maybe we can offer the option to store the result with a unique ID on the server so you could just retreive the results. However I think the transfer of a large query should usually be the slower part of the query.
I think that this in not a good option because it will overload your server and this can handicap other users.
Thanks for everything!!
from multimir.
Related Issues (20)
- Prediction scores from multiMiR output HOT 1
- database update status HOT 6
- Time out using get_multimir with 'predicted' parameter HOT 2
- include Diana micro-T CDS?
- are there local databases like 'org.Hs.eg.db' for search HOT 2
- Score integration
- connection refused in R HOT 3
- Error with “mmquery_bioc” HOT 1
- Can't load library(multiMiR) HOT 5
- Error with loading multiMiR HOT 2
- Failed to connect to multimir.org port 80: Connection refused HOT 2
- What the functional column does mean please HOT 2
- Issue with get_multimir: table = "predicted" HOT 5
- miRTarBase update request HOT 7
- Update Databases / Add New Databases HOT 11
- Failure when receiving data from the peer HOT 8
- TargetScan Prediction Score cutoff interpretation HOT 1
- deprecated 'as_data_frame()' HOT 2
- failure when library(multiMiR) HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from multimir.