Giter Site home page Giter Site logo

bel-commons / bel-commons Goto Github PK

View Code? Open in Web Editor NEW
4.0 11.0 0.0 9.11 MB

An environment for curating, validating, and exploring knowledge assemblies encoded in BEL

License: MIT License

Python 33.98% CSS 0.77% JavaScript 19.87% HTML 45.30% Dockerfile 0.06% Shell 0.02%
biological-expression-language systems-biology bioinformatics networks-biology

bel-commons's Introduction

BEL Commons Travis-CI Build Status Documentation Status

An environment for curating, validating, and exploring knowledge assemblies encoded in Biological Expression Language (BEL) to support elucidating disease-specific, mechanistic insight.

Installation

BEL Commons can be installed easily from PyPI with the following code in your favorite shell:

$ pip install bel_commons

Get the latest code on GitHub with:

$ python3 -m pip install git+https://github.com/bel-commons/bel-commons.git

It's also suggested to use a relational database management system like PostgreSQL and install their corresponding connectors:

$ python3 -m pip install psycopg2-binary

Run with Manual Setup

A test server can be easily run with:

$ bel-commons run

A more powerful server like gunicorn can also be used like:

$ gunicorn bel_commons.wsgi:flask_app

Running with the Parser

To run the parser, you'll need an instance of a message queue like RabbitMQ (or any other message queue supported by Celery), a results backend like Redis, and a worker. It's best to run in docker if you want to do this.

Run with docker-compose

Clone this repo from GitHub

$ git clone https://github.com/bel-commons/bel-commons.git
$ cd bel-commons

Create a file called .env and generate both SECRET_KEY and SECURITY_PASSWORD_SALT.

SECRET_KEY=mypassword
SECURITY_PASSWORD_SALT=mypassword
BUTLER_NAME="BEL Commons Butler"
[email protected]
BUTLER_PASSWORD=butlerpassword

Other options:

  • If you want to run BEL Commons so networks can be made private, add DISALLOW_PRIVATE=false.
  • If you want to disable self-serve user registration, add SECURITY_REGISTERABLE=false.
  • If you want to run BEL Commons so none of the pages/endpoints are accessible without login, add LOCKDOWN=true.
  • If you want to automatically add some example networks, add REGISTER_EXAMPLES=true.

Run docker-compose. If you don't have it, check the docker-compose installation guide:

$ docker-compose up

If you want it to run in the background, use docker-compose up --detach.

Ports exposed:

  • 5002: BEL Commons web application
  • 5432: PostgreSQL database

Figure our your container ID with:

docker ps | grep bel

Check the logs with:

docker exec -it <your container id> /usr/bin/tail -f web_log.txt

Same can be done for celery_log.txt or worker_log.txt.

Make an existing user an admin with:

docker exec -it <your container id> bel-commons manage users make-admin <user email>

Reset the Database

For the times when you just have to burn it down and start over:

  1. bel-commons manage drop will nuke the database and output a user list
  2. bel-commons manage load will automatically add the most recently exported user list
  3. bel-commons manage examples load will automatically load some example networks and data sets

Citation

If you find BEL Commons useful in your work, please consider citing [Hoyt2018] and [Hoyt2017]:

[Hoyt2018]Hoyt, C. T., Domingo-Fernández, D., & Hofmann-Apitius, M. (2018). BEL Commons: an environment for exploration and analysis of networks encoded in Biological Expression Language. Database, 2018(3), 1–11.
[Hoyt2017]Hoyt, C. T., Konotopez, A., & Ebeling, C., (2017). PyBEL: a computational framework for Biological Expression Language. Bioinformatics, 34(4), 703–704.

Acknowledgements

Supporters

This project has been supported by several organizations:

Logo

The BEL Commons logo was designed by Scott Colby.

bel-commons's People

Contributors

alexmasny avatar cthoyt avatar ddomingof avatar jmarinllao avatar

Stargazers

 avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

bel-commons's Issues

Checksum calculations of BEL Script/Network

I just realized that if we solely check that two documents are the same based on their name and version, some users could get hacky and steal the rights to other documents. Therefore, it would be necessary to store the checksum of a BEL script and make sure that those are the same too before giving the rights away.

Do this with the same hashlib.sha256(to_bytes(graph)).hexdigest()

What happens when two users upload the same network?

This will definitely be a problem when the second person goes to upload the Selventa Small Corpus. We need to change the way the report system works to allow two users to report the same network - and not try and upload a network a second time if it's already in the database.

First step - Hack up the current parser to do some checks after the document metadata section is done parsing

Query to grandpa

Create a recursive function in Query Model that asks if it has a parent and returns it until it finds the root

Change opacity by function (include in tools)

Ideas:

  • Add "highlight" tab next to annotations, nodes, and edges with filters for each of the node types, for nodes with certain transformations (molecular activity, translocation, or reaction)

Update footer text

To be more specific, say that PyBEL web is developed and maintained by Charlie + daniel on top of the open source packages pybel and pybel tools

Exporting options using BytesIO not working [ 'graphml', 'sif', 'csv' , 'gsea']

[2017-07-03 12:01:20,605] ERROR in app: Exception on /api/network/query/9/export/csv [GET]
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1982, in wsgi_app
response = self.full_dispatch_request()
File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1614, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1517, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/usr/local/lib/python3.6/site-packages/flask/_compat.py", line 33, in reraise
raise value
File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1612, in full_dispatch_request
rv = self.dispatch_request()
File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1598, in dispatch_request
return self.view_functionsrule.endpoint
File "/home/ddomingofernandez/Projects/pybel-web/src/pybel_web/database_service.py", line 260, in download_network
return serve_network(network, serve_format=serve_format)
File "/home/ddomingofernandez/Projects/pybel-web/src/pybel_web/send_utils.py", line 94, in serve_network
to_csv(graph, bio)
File "/home/ddomingofernandez/Projects/pybel/src/pybel/io/extras.py", line 66, in to_csv
file=file
TypeError: a bytes-like object is required, not 'str'

Nightly Builds on Development Server

Need to create a dockerfile that has a self contained sql database. Might have to switch the repo over to gitlab, then have it automatically build and deploy on http://bart.scai.fraunhofer.de:5001 on a nightly basis (http://dev.pybel.fraunhofer.scai.de points here)

Perhaps Kono made some progress in this a month or two ago and can give advice

  • Set up a cron job on jenkins that pulls the repositories and can do this all automatically, at around 6PM at night.

Using Travis-CI

Follow the instructions from Travis (https://docs.travis-ci.com/user/docker/#Pushing-a-Docker-Image-to-a-Registry)

  1. Add private build variables
  2. Update .travis.yml for pushing to Artifactory

Create Swagger file

Option 1: flask-swagger

This allows you to write the relevant swagger spec inline with the functions in flask. This is really nice because I like documentation being part of the code. It automatically sucks it all up and makes the right file at the end

Option 2: flasgger

This one seems even more fully featured, and also builds a Swagger UI page

Fix NetworkXNoPath

https://sentry.io/pybel/pybel-web/issues/308608071/

NetworkXNoPath: 
(10 additional frame(s) were not displayed)
...
  File "pybel_tools/selection/induce_subgraph.py", line 320, in get_subgraph
    result = get_subgraph_by_all_shortest_paths(graph, seed_data)
  File "pybel_tools/selection/induce_subgraph.py", line 174, in get_subgraph_by_all_shortest_paths
    return graph.subgraph(get_nodes_in_all_shortest_paths(graph, nodes, weight=weight))
  File "pybel_tools/selection/paths.py", line 79, in get_nodes_in_all_shortest_paths
    for u, v in product(nodes, repeat=2)
  File "pybel_tools/selection/paths.py", line 80, in <setcomp>
    for path in all_shortest_paths(graph, u, v, weight=weight)
  File "networkx/algorithms/shortest_paths/generic.py", line 359, in all_shortest_paths
    raise nx.NetworkXNoPath()

NetworkXNoPath

Error union multiple graphs

If you select multiple networks and create the query the union seems to fail

[2017-06-16 16:50:05,906] ERROR in app: Exception on /api/get_query/8 [GET]
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1982, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1614, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1517, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/local/lib/python3.6/site-packages/flask/_compat.py", line 33, in reraise
    raise value
  File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/home/ddomingofernandez/Projects/pybel-web/src/pybel_web/database_service.py", line 822, in query_to_url
    network = query(api)  # use duck typing, and the api ensures relabeling
  File "/home/ddomingofernandez/Projects/pybel-tools/src/pybel_tools/query.py", line 44, in __call__
    networks = manager.get_networks_by_ids(self.network_ids)
  File "/home/ddomingofernandez/Projects/pybel-tools/src/pybel_tools/api.py", line 260, in get_networks_by_ids
    for network in self.manager.get_networks_by_ids(network_ids)
  File "/home/ddomingofernandez/Projects/pybel/src/pybel/struct/operations.py", line 191, in union
    target = next(networks_iter).copy()
StopIteration

Add automatic user creation

Make an account for Charlie and daniel
That have admin rights automatically with app.before_request filter in application.py

Suggested modification in pybel_network

SAWarning: The IN-predicate on "pybel_network.id" was invoked with an empty sequence. This results in a contradiction, which nonetheless can be expensive to evaluate. Consider alternative strategies for improved performance.

Highlight last change

Once the query system is implemented and replaces the current tracking of exploration, it would be cool to highlight the changes to the network from the final step

Expand Subgraphs Pipeline

Here's an example query for showing the state of the Tau subgraph. Ultimately it should be as simple as emailing Martin a link to the page (something like http://pybel.fraunhofer.scai.de/explore?query=1) which we're almost at!

  1. Assemble Universe as:
    • parkinson knowledge assembly
    • parkinson neurommsig
    • alzheimer knowledge assembly
    • alzheimer neurommsig
  2. Seed by annotation = Subgraph and value = Tau protein subgraph
  3. Filters and Mutations:
    1. Infer central dogma
    2. Enrich unqualified
    3. Expand internal edges
    4. Remove pathologies
    5. Prune central dogma

Belief Integration Blueprint

  • Add endpoint that accepts multiple files in POST and does a Merge on them
  • add endpoint that accepts JSON with multiple files in it as lists of strings representing their lines and does a merge on it
  • for BELIEF, the format is the same so the stupid merge is possible. also implement a semantic merge. Needs testing as well...

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.