dfci/matchminer-api

A platform for matching patient-specific genomic profiles to precision cancer medicine clinical trials

matchminer-api's Issues

matchminer-ui hitting api and elasticsearch: no mapping found for field

MacOS 10.14.1
conda env with python 2.7
Locally installed matchminer-ui and matchminer-api, started as described in the docs.
mongo in a container as described in the matchminer-api docs.

When the UI frontpage loads (8001) it gives correct number of patients (91) so it communicates with mongo OK.

When I try to type anything in "Disease Center" search box (or any search box for that matter) in developer tools/network I see these errors:

Performing suggest!
angular.js:11756 POST http://localhost:8001/elasticsearch/matchminer/trial/_search 400 (Bad Request)
(anonymous) @ angular.js:11756
sendReq @ angular.js:11517
serverRequest @ angular.js:11227
processQueue @ angular.js:15961
(anonymous) @ angular.js:15977
$eval @ angular.js:17229
$digest @ angular.js:17045
processQueue @ angular-material.js:1734
(anonymous) @ angular.js:19157
completeOutstandingRequest @ angular.js:5869
(anonymous) @ angular.js:6145
setTimeout (async)
Browser.self.defer @ angular.js:6143
timeout @ angular.js:19155
nextTick @ angular-material.js:1711
handleAsyncResults @ angular-material.js:26145
fetchResults @ angular-material.js:26136
handleQuery @ angular-material.js:26253
focus @ angular-material.js:25824
fn @ VM541:4
expensiveCheckFn @ angular.js:15485
callback @ angular.js:25018
$eval @ angular.js:17229
$apply @ angular.js:17329
(anonymous) @ angular.js:25023
dispatch @ jquery.js:5183
elemData.handle @ jquery.js:4991
angular.js:13550 Error occurred while searching for suggestions Error: [illegal_argument_exception] no mapping found for field [_suggest.disease_center_suggest]
at respond (http://localhost:8001/bower_components/elasticsearch/elasticsearch.angular.js:15491:16)
at checkRespForFailure (http://localhost:8001/bower_components/elasticsearch/elasticsearch.angular.js:15450:8)
at http://localhost:8001/bower_components/elasticsearch/elasticsearch.angular.js:319:8
at processQueue (http://localhost:8001/bower_components/angular/angular.js:15961:28)
at http://localhost:8001/bower_components/angular/angular.js:15977:27
at Scope.$eval (http://localhost:8001/bower_components/angular/angular.js:17229:28)
at Scope.$digest (http://localhost:8001/bower_components/angular/angular.js:17045:31)
at Scope.$apply (http://localhost:8001/bower_components/angular/angular.js:17337:24)
at http://localhost:8001/bower_components/angular/angular.js:17633:22
at completeOutstandingRequest (http://localhost:8001/bower_components/angular/angular.js:5869:10)

I suspect there should be performed some "elasticsearch configuration"-related step, some mapping / indexing for the elasticsearch.

Setup failed during elasticsearch initializaiton: No route to host

I followed the setup instructions, but it didn't work out of the box:

$ git clone [email protected]:dfci/matchminer-api.git
$ cd matchminer-api
$ ./setup.sh

I ran into and resolved two errors:

1. An error due to the mongo executable not being shipped with the mm-mongo container (mongo image). So, I replaced it with mongosh.
2. An error due to numpy 1.23.3 not being available when the mm-api container tried to run pip install -r requirements.txt.

Resolved these with

--- a/requirements.txt
+++ b/requirements.txt
@@ -91,7 +91,8 @@ nose==1.3.7
     # via
     #   -r requirements.in
     #   oncotreenx
-numpy==1.23.3
+numpy==1.21.6
    # via pandas
 oauth2client==2.0.2
     # via -r requirements.in


--- a/setup.sh
+++ b/setup.sh
@@ -24,7 +24,7 @@ echo "*****************"
 sleep 5

 echo "Add dev user to database to bypass authentication"
-docker-compose exec mm-mongo mongo matchminer --eval 'db.user.replaceOne({
+docker-compose exec mm-mongo mongosh matchminer --eval 'db.user.replaceOne({

But now,

$ ./setup
...
Using Mongosh:          1.6.1
...
 ⠿ Container mm-elastic  Started                                                                [382/530]
[WARNING] settings:
[WARNING] settings: mongodb://mm-mongo:27017/matchminer
/matchminerAPI/cerberus1/cerberus.py:1272: UserWarning: No validation schema is defined for the arguments
 of rule 'consented'
  "'%s'" % '_'.join(method_name.split('_')[2:]))
/matchminerAPI/cerberus1/cerberus.py:1272: UserWarning: No validation schema is defined for the arguments
 of rule 'match'
  "'%s'" % '_'.join(method_name.split('_')[2:]))
/matchminerAPI/cerberus1/cerberus.py:1272: UserWarning: No validation schema is defined for the arguments
 of rule 'normalized'
  "'%s'" % '_'.join(method_name.split('_')[2:]))
/matchminerAPI/cerberus1/cerberus.py:1272: UserWarning: No validation schema is defined for the arguments
 of rule 'unique'
  "'%s'" % '_'.join(method_name.split('_')[2:]))
/usr/local/lib/python3.7/site-packages/cerberus/validator.py:1402: UserWarning: No validation schema is d
efined for the arguments of rule 'consented'
  "'%s'" % method_name.split('_', 2)[-1])
/usr/local/lib/python3.7/site-packages/cerberus/validator.py:1402: UserWarning: No validation schema is d
efined for the arguments of rule 'match'
  "'%s'" % method_name.split('_', 2)[-1])
/usr/local/lib/python3.7/site-packages/cerberus/validator.py:1402: UserWarning: No validation schema is d
efined for the arguments of rule 'normalized'
  "'%s'" % method_name.split('_', 2)[-1])
[WARNING] NO AUTHENTICATION IS ENABLED - SKIPPING HIPAA LOGGING
[WARNING] settings:
[WARNING] settings: mongodb://mm-mongo:27017/matchminer
/usr/local/lib/python3.7/site-packages/eve/flaskapp.py:310: UserWarning: XML setting is deprecated and wi
ll be removed in future release. Please use RENDERERS instead.
  warnings.warn(msg.format("XML"))
[INFO] reset_elasticsearch
[INFO] http://mm-elastic:9200/matchminer
[INFO]
[INFO]
[INFO] delete_elasticsearch_index
[WARNING] DELETE http://mm-elastic:9200/matchminer [status:N/A request:3.052s]
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/urllib3/connection.py", line 159, in _new_conn
    (self._dns_host, self.port), self.timeout, **extra_kw)
  File "/usr/local/lib/python3.7/site-packages/urllib3/util/connection.py", line 80, in create_connection
    raise err
  File "/usr/local/lib/python3.7/site-packages/urllib3/util/connection.py", line 70, in create_connection
    sock.connect(sa)
OSError: [Errno 113] No route to host

Any suggestions?

Additionally, what version of mongo was docker-compose supposed to retrieve?
docker-compose.yml did not specify a version number, and the current matchminer-api setup is apparently not compatible with the latest version of mongo, because it relies on the deprecated mongo executable rather than mongosh.

$ uname -a
Linux durus 6.1.4-arch1-1 #1 SMP PREEMPT_DYNAMIC Sat, 07 Jan 2023 15:10:07 +0000 x86_64 GNU/Linux

$ docker version
Client:
 Version:           20.10.22
 API version:       1.41
 Go version:        go1.19.4
 Git commit:        3a2c30b63a
 Built:             Tue Dec 20 20:43:40 2022
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true

Server:
 Engine:
  Version:          20.10.22
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.19.4
  Git commit:       42c8b31499
  Built:            Tue Dec 20 20:42:46 2022
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          v1.6.14
  GitCommit:        9ba4b250366a5ddde94bb7c9d1def331423aa323.m
 runc:
  Version:          1.1.4
  GitCommit:
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

$ docker-compose version
Docker Compose version 2.14.2

python pymm_run.py restore step: Use of SCRAM-SHA-256 requires undigested passwords

My pymongo is 2.9.1 (as in the requirements.txt)

I ran these two successfully as far as I can tell:

$ docker run --name match-mongo -d -p 27017:27017 mongo mongod --smallfiles --replSet=rs0
$ docker exec -it match-mongo mongo matchminer --eval "rs.initiate();"

Now:
$ python2.7 pymm_run.py restore

[INFO] settings: DEV
[INFO] settings: mongodb://localhost:27017/matchminer?replicaSet=rs0
/Users/ipozdnya/matchminer-api/cerberus1/cerberus.py:1272: UserWarning: No validation schema is defined for the arguments of rule 'consented'
  "'%s'" % '_'.join(method_name.split('_')[2:]))
/Users/ipozdnya/matchminer-api/cerberus1/cerberus.py:1272: UserWarning: No validation schema is defined for the arguments of rule 'match'
  "'%s'" % '_'.join(method_name.split('_')[2:]))
/Users/ipozdnya/matchminer-api/cerberus1/cerberus.py:1272: UserWarning: No validation schema is defined for the arguments of rule 'normalized'
  "'%s'" % '_'.join(method_name.split('_')[2:]))
/Users/ipozdnya/matchminer-api/cerberus1/cerberus.py:1272: UserWarning: No validation schema is defined for the arguments of rule 'unique'
  "'%s'" % '_'.join(method_name.split('_')[2:]))
[INFO] settings: DEV
[INFO] settings: mongodb://localhost:27017/matchminer?replicaSet=rs0
[INFO] mongorestore --host rs0/localhost:27017 --db matchminer --drop /Users/ipozdnya/matchminer-api/data/matchminer
2019-02-21T11:46:41.458+0100	building a list of collections to restore from /Users/ipozdnya/matchminer-api/data/matchminer dir
2019-02-21T11:46:41.465+0100	reading metadata file from /Users/ipozdnya/matchminer-api/data/matchminer/genomic.metadata.json
2019-02-21T11:46:41.465+0100	restoring matchminer.genomic from file /Users/ipozdnya/matchminer-api/data/matchminer/genomic.bson
2019-02-21T11:46:41.468+0100	reading metadata file from /Users/ipozdnya/matchminer-api/data/matchminer/clinical.metadata.json
2019-02-21T11:46:41.468+0100	restoring matchminer.clinical from file /Users/ipozdnya/matchminer-api/data/matchminer/clinical.bson
2019-02-21T11:46:41.532+0100	restoring indexes for collection matchminer.clinical from metadata
2019-02-21T11:46:41.534+0100	finished restoring matchminer.clinical (92 documents)
2019-02-21T11:46:44.459+0100	[#####################...]  matchminer.genomic  77.1 MB/84.8 MB  (91.0%)
2019-02-21T11:46:45.128+0100	restoring indexes for collection matchminer.genomic from metadata
2019-02-21T11:46:45.129+0100	finished restoring matchminer.genomic (87980 documents)
2019-02-21T11:46:45.129+0100	done
Traceback (most recent call last):
  File "pymm_run.py", line 467, in <module>
    args.func(args)
  File "pymm_run.py", line 186, in bootstrap_restore
    db = get_db()
  File "/Users/ipozdnya/matchminer-api/matchminer/database.py", line 50, in get_db
    connection[MONGO_DBNAME].add_user(MONGO_USERNAME, MONGO_PASSWORD)
  File "/Users/ipozdnya/miniconda3/envs/matchminer/lib/python2.7/site-packages/pymongo/database.py", line 966, in add_user
    (not uinfo["users"]), name, password, read_only, **kwargs)
  File "/Users/ipozdnya/miniconda3/envs/matchminer/lib/python2.7/site-packages/pymongo/database.py", line 892, in _create_or_update_user
    read_preference=ReadPreference.PRIMARY, **opts)
  File "/Users/ipozdnya/miniconda3/envs/matchminer/lib/python2.7/site-packages/pymongo/database.py", line 535, in command
    codec_options, **kwargs)[0]
  File "/Users/ipozdnya/miniconda3/envs/matchminer/lib/python2.7/site-packages/pymongo/database.py", line 440, in _command
    msg, allowable_errors)
  File "/Users/ipozdnya/miniconda3/envs/matchminer/lib/python2.7/site-packages/pymongo/helpers.py", line 213, in _check_command_response
    raise OperationFailure(msg % errmsg, code, response)
pymongo.errors.OperationFailure: command SON([('createUser', 'user'), ('pwd', u'<some encoded password here>'), ('digestPassword', False), ('roles', ['dbOwner']), ('writeConcern', {})]) on namespace matchminer.$cmd failed: Use of SCRAM-SHA-256 requires undigested passwords

That happened to me before when on another machine and then I fixed it by getting into mongo and creating an admin user:

> use admin
> db.createUser({     user:"admin",     pwd:"abc123",     roles:[{role:"userAdminAnyDatabase",db:"admin"}],     passwordDigestor:"server" })

This time it did not help. Dropping and re-creating that user did not help either.

Running $ nosetests tests results in the same error.

Please suggest a workaround. Thanks!

elasticsearch.exceptions.RequestError: RequestError(400, 'illegal_argument_exception', 'Action/metadata line [1] contains an unknown parameter [_type]') error when running python pymm_run.py reset-elasticsearch

Trying to install all the matchminer bits on a new server (without docker) and I found that the elastic search indexes do not seem to be set up. So I tried running python pymm_run.py reset-elasticsearch in the matchminer-api directory.
It seemed to be going along fine until it came to this error:

Any suggestions on what to do next?

Thanks!
Wes

[WARNING] POST http://localhost:9200/_bulk [status:400 request:0.005s]
Traceback (most recent call last):
  File "pymm_run.py", line 87, in <module>
    args.func(args)
  File "pymm_run.py", line 81, in <lambda>
    subp_p.set_defaults(func=lambda x: reset_elasticsearch())
  File "/opt/matchminer-api/matchminer/elasticsearch.py", line 189, in reset_elasticsearch
    add_all_trials_to_elasticsearch()]
  File "/opt/matchminer-api/matchminer/elasticsearch.py", line 44, in add_all_trials_to_elasticsearch
    add_trial_id_list_to_elasticsearch(chunked_trial_ids)
  File "/opt/matchminer-api/matchminer/elasticsearch.py", line 50, in add_trial_id_list_to_elasticsearch
    add_prepared_trials_to_elasticsearch(prepared_trials)
  File "/opt/matchminer-api/matchminer/elasticsearch.py", line 67, in add_prepared_trials_to_elasticsearch
    helpers.bulk(get_es_client(), prepared_trials)
  File "/opt/matchminer-api/env/lib/python3.7/site-packages/elasticsearch/helpers/actions.py", line 314, in bulk
    for ok, item in streaming_bulk(client, actions, *args, **kwargs):
  File "/opt/matchminer-api/env/lib/python3.7/site-packages/elasticsearch/helpers/actions.py", line 244, in streaming_bulk
    **kwargs
  File "/opt/matchminer-api/env/lib/python3.7/site-packages/elasticsearch/helpers/actions.py", line 130, in _process_bulk_chunk
    raise e
  File "/opt/matchminer-api/env/lib/python3.7/site-packages/elasticsearch/helpers/actions.py", line 126, in _process_bulk_chunk
    resp = client.bulk("\n".join(bulk_actions) + "\n", *args, **kwargs)
  File "/opt/matchminer-api/env/lib/python3.7/site-packages/elasticsearch/client/utils.py", line 101, in _wrapped
    return func(*args, params=params, **kwargs)
  File "/opt/matchminer-api/env/lib/python3.7/site-packages/elasticsearch/client/__init__.py", line 1581, in bulk
    headers={"content-type": "application/x-ndjson"},
  File "/opt/matchminer-api/env/lib/python3.7/site-packages/elasticsearch/transport.py", line 409, in perform_request
    timeout=timeout,
  File "/opt/matchminer-api/env/lib/python3.7/site-packages/elasticsearch/connection/http_urllib3.py", line 252, in perform_request
    self._raise_error(response.status, raw_data)
  File "/opt/matchminer-api/env/lib/python3.7/site-packages/elasticsearch/connection/base.py", line 254, in _raise_error
    status_code, error_message, additional_info
elasticsearch.exceptions.RequestError: RequestError(400, 'illegal_argument_exception', 'Action/metadata line [1] contains an unknown parameter [_type]')

logo

matchminer-ui hitting matchminer-api: unauthorized access and 404

MacOS 10.14.1
conda env with python 2.7
Locally installed matchminer-ui and matchminer-api, started as described in the docs.
mongo in a container as described in the matchminer-api docs.

When the UI frontpage loads (8001) it gives correct number of patients (91) so it communicates with mongo OK.

When I inspect network traffic in the browser's development tools, there are several errors trying to hit /api like this:

http://localhost:8001/api/user/577cf6ef2b9920002cef0337?cacheBuster=1553031094438

When I hit that in browser, I get

"unauthorized access".

Same message if I change the port to 5000 (as it's set in config.json).
Here is the relevant part of config.json:

  "dev": {
    "ENV": {
      "name": "development",
      "elasticsearch": {
        "host": "http://localhost:9200",
        "proxy": "http://localhost:8001/elasticsearch",
        "index": "matchminer"
      },
      "api": {
        "host": "http://localhost:5000",
        "endpoint": "/api"
      },
      "samlAuthentication": false,
      "devUser": {
        "token" : "some token here,
        "user_id" : "some user id here"
      },
      "certificate" : {
        "key": "../certificates/matchminer.key",
        "cert": "../certificates/matchminer.crt"
      },

.....

Both certificates files (matchminer.key and matchminer.crt) are in place

The messages might be coming from pymm_run.py from here:

@app.errorhandler(401)
def my_own_error_msg(err):
    return make_response("unauthorized access", 497)

What can this "unauthorized access" can by caused by?

There is an option to start matchminer-api with --no-auth switch. When I do that, on the front page I get red stripe across with "You are currently not authorized to access this page." message, the patient count is still correct. However, the api calls from the page produce different error:

e.g.
http://localhost:8001/api/user/577cf6ef2b9920002cef0337?cacheBuster=1553032900445

{
    _status: "ERR",
    _error: {
        message: "The requested URL was not found on the server. If you entered the URL manually please check your spelling and try again.", 
        code: 404
    }
}

This happens with both 5000 and 8001 ports

Hitting an API endpoint with no parameters (
http://localhost:8001/api/user/) gives this:

{
    _items: [ ],
    _links: {
    self: {
         href: "user",
         title: "user"
    },
    parent: {
        href: "/",
        title: "home"
    }
},
    _meta: {
        max_results: 100000,
        total: 0,
        page: 1
    }
}

UPDATE:
This might be a data issue, user data in particular. For example, http://localhost:8001/api/clinical/ being pasted in the browser returns the clinical data ok.
However http://localhost:8001/api/user/577cf6ef2b9920002cef033 call points at non-existing user id "577cf6ef2b9920002cef033" in a non-existing collection "user". Where are those two entities coming from in the angular frontend? Should "user" collection with that id exist before the ui is started? mongorestore creates only "clinical" and "genomic" collections, no "user". Or maybe that api call should not get executed on the frontpage load? The problem is this error disables everything in the interface, makes search fields not clickable.

UPDATE 2
The troublesome user API call from the frontpage goes away if:

1. I run matchminer-engine data loading script from matchminer-engine:
   python2.7 matchengine.py load -t examples/trial.example.yml -c examples/clinical.example.csv -g examples/genomic.example.csv --mongo-uri mongodb://localhost:27017 . It does not create any new collections in mongo, but it does bump up the number of patients from 91 to 110
2. restart python pymm_run.py serve WITHOUT --no-auth

The search still does not work (typing any char in the search fields gives another error) but that's a subject of a different issue. I guess since there is a workaround now this issue can be closed, leaving open just in case if the maintainer would like to comment on my install procedure.

demo: pip install -r requirements.txt dies with SyntaxError

Step pip install -r requirements.txt dies with this (probably python-package version mismatch):

Collecting dm.xmlsec.binding==1.3.2 (from python-saml==2.1.5->-r requirements.txt (line 46))
Downloading https://files.pythonhosted.org/packages/0c/47/c8e17a5fec7bc86b092ba6841c2db7ce5e828327eb72d6a8c4c93a19f62e/dm.xmlsec.binding-1.3.2.tar.gz (119kB)
100% |████████████████████████████████| 122kB 10.7MB/s
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "", line 1, in
File "/private/var/folders/4z/6tmqh9hx68v3081lpb6j33k00000gs/T/pip-install-pmczeksh/dm.xmlsec.binding/setup.py", line 72
print "Warning : cflag %s skipped" % flag
^
SyntaxError: Missing parentheses in call to 'print'

----------------------------------------

Command "python setup.py egg_info" failed with error code 1 in /private/var/folders/4z/6tmqh9hx68v3081lpb6j33k00000gs/T/pip-install-pmczeksh/dm.xmlsec.binding/

aliasing python to python2.7 inside the VE does not help.
commenting out saml package in requirements/.txt lets it finish with no other issues

demo: step "nosetests tests" gives errors: syntax, non-existing attribute and non-existent module 'rfc822' and 'engine'

matchminer-api$ nosetests tests

EEEEEE....E

ERROR: Failure: AttributeError (module 'sys' has no attribute 'maxint')

Traceback (most recent call last):
File "/Users/ipozdnya/ve_matchminer/lib/python3.5/site-packages/nose/failure.py", line 39, in runTest
raise self.exc_val.with_traceback(self.tb)
File "/Users/ipozdnya/ve_matchminer/lib/python3.5/site-packages/nose/loader.py", line 418, in loadTestsFromName
addr.filename, addr.module)
File "/Users/ipozdnya/ve_matchminer/lib/python3.5/site-packages/nose/importer.py", line 47, in importFromPath
return self.importFromDir(dir_path, fqname)
File "/Users/ipozdnya/ve_matchminer/lib/python3.5/site-packages/nose/importer.py", line 94, in importFromDir
mod = load_module(part_fqname, fh, filename, desc)
File "/Users/ipozdnya/ve_matchminer/lib/python3.5/imp.py", line 235, in load_module
return load_source(name, filename, file)
File "/Users/ipozdnya/ve_matchminer/lib/python3.5/imp.py", line 172, in load_source
module = _load(spec)
File "", line 693, in _load
File "", line 673, in _load_unlocked
File "", line 697, in exec_module
File "", line 222, in _call_with_frames_removed
File "/Users/ipozdnya/matchminer-api/tests/unit/test_api/test_neg_genomic.py", line 5, in
from matchminer.database import get_db
File "/Users/ipozdnya/matchminer-api/matchminer/database.py", line 4, in
from matchminer.settings import *
File "/Users/ipozdnya/matchminer-api/matchminer/settings.py", line 56, in
from matchminer.settings_dev import *
File "/Users/ipozdnya/matchminer-api/matchminer/settings_dev.py", line 20, in
TOKEN_TIMEOUT = sys.maxint
AttributeError: module 'sys' has no attribute 'maxint'

======================================================================
ERROR: Failure: ImportError (No module named 'rfc822')

Traceback (most recent call last):
File "/Users/ipozdnya/ve_matchminer/lib/python3.5/site-packages/nose/failure.py", line 39, in runTest
raise self.exc_val.with_traceback(self.tb)
File "/Users/ipozdnya/ve_matchminer/lib/python3.5/site-packages/nose/loader.py", line 418, in loadTestsFromName
addr.filename, addr.module)
File "/Users/ipozdnya/ve_matchminer/lib/python3.5/site-packages/nose/importer.py", line 47, in importFromPath
return self.importFromDir(dir_path, fqname)
File "/Users/ipozdnya/ve_matchminer/lib/python3.5/site-packages/nose/importer.py", line 94, in importFromDir
mod = load_module(part_fqname, fh, filename, desc)
File "/Users/ipozdnya/ve_matchminer/lib/python3.5/imp.py", line 235, in load_module
return load_source(name, filename, file)
File "/Users/ipozdnya/ve_matchminer/lib/python3.5/imp.py", line 172, in load_source
module = _load(spec)
File "", line 693, in _load
File "", line 673, in _load_unlocked
File "", line 697, in exec_module
File "", line 222, in _call_with_frames_removed
File "/Users/ipozdnya/matchminer-api/tests/unit/test_api/test_response.py", line 7, in
from rfc822 import formatdate
ImportError: No module named 'rfc822'

======================================================================
ERROR: Failure: AttributeError (module 'sys' has no attribute 'maxint')

Traceback (most recent call last):
File "/Users/ipozdnya/ve_matchminer/lib/python3.5/site-packages/nose/failure.py", line 39, in runTest
raise self.exc_val.with_traceback(self.tb)
File "/Users/ipozdnya/ve_matchminer/lib/python3.5/site-packages/nose/loader.py", line 418, in loadTestsFromName
addr.filename, addr.module)
File "/Users/ipozdnya/ve_matchminer/lib/python3.5/site-packages/nose/importer.py", line 47, in importFromPath
return self.importFromDir(dir_path, fqname)
File "/Users/ipozdnya/ve_matchminer/lib/python3.5/site-packages/nose/importer.py", line 94, in importFromDir
mod = load_module(part_fqname, fh, filename, desc)
File "/Users/ipozdnya/ve_matchminer/lib/python3.5/imp.py", line 235, in load_module
return load_source(name, filename, file)
File "/Users/ipozdnya/ve_matchminer/lib/python3.5/imp.py", line 172, in load_source
module = _load(spec)
File "", line 693, in _load
File "", line 673, in _load_unlocked
File "", line 697, in exec_module
File "", line 222, in _call_with_frames_removed
File "/Users/ipozdnya/matchminer-api/tests/unit/test_api/test_status.py", line 6, in
from matchminer.database import get_db
File "/Users/ipozdnya/matchminer-api/matchminer/database.py", line 4, in
from matchminer.settings import *
File "/Users/ipozdnya/matchminer-api/matchminer/settings.py", line 56, in
from matchminer.settings_dev import *
File "/Users/ipozdnya/matchminer-api/matchminer/settings_dev.py", line 20, in
TOKEN_TIMEOUT = sys.maxint
AttributeError: module 'sys' has no attribute 'maxint'

======================================================================
ERROR: Failure: SyntaxError (invalid token (test_trial.py, line 621))

Traceback (most recent call last):
File "/Users/ipozdnya/ve_matchminer/lib/python3.5/site-packages/nose/failure.py", line 39, in runTest
raise self.exc_val.with_traceback(self.tb)
File "/Users/ipozdnya/ve_matchminer/lib/python3.5/site-packages/nose/loader.py", line 418, in loadTestsFromName
addr.filename, addr.module)
File "/Users/ipozdnya/ve_matchminer/lib/python3.5/site-packages/nose/importer.py", line 47, in importFromPath
return self.importFromDir(dir_path, fqname)
File "/Users/ipozdnya/ve_matchminer/lib/python3.5/site-packages/nose/importer.py", line 94, in importFromDir
mod = load_module(part_fqname, fh, filename, desc)
File "/Users/ipozdnya/ve_matchminer/lib/python3.5/imp.py", line 235, in load_module
return load_source(name, filename, file)
File "/Users/ipozdnya/ve_matchminer/lib/python3.5/imp.py", line 172, in load_source
module = _load(spec)
File "", line 693, in _load
File "", line 673, in _load_unlocked
File "", line 693, in exec_module
File "", line 799, in get_code
File "", line 759, in source_to_code
File "", line 222, in _call_with_frames_removed
File "/Users/ipozdnya/matchminer-api/tests/unit/test_api/test_trial.py", line 621
"site": [{"id": [01], "value": "Dana-Farber Cancer Institute"}],
^
SyntaxError: invalid token

======================================================================
ERROR: Failure: AttributeError (module 'sys' has no attribute 'maxint')

Traceback (most recent call last):
File "/Users/ipozdnya/ve_matchminer/lib/python3.5/site-packages/nose/failure.py", line 39, in runTest
raise self.exc_val.with_traceback(self.tb)
File "/Users/ipozdnya/ve_matchminer/lib/python3.5/site-packages/nose/loader.py", line 418, in loadTestsFromName
addr.filename, addr.module)
File "/Users/ipozdnya/ve_matchminer/lib/python3.5/site-packages/nose/importer.py", line 47, in importFromPath
return self.importFromDir(dir_path, fqname)
File "/Users/ipozdnya/ve_matchminer/lib/python3.5/site-packages/nose/importer.py", line 94, in importFromDir
mod = load_module(part_fqname, fh, filename, desc)
File "/Users/ipozdnya/ve_matchminer/lib/python3.5/imp.py", line 235, in load_module
return load_source(name, filename, file)
File "/Users/ipozdnya/ve_matchminer/lib/python3.5/imp.py", line 172, in load_source
module = _load(spec)
File "", line 693, in _load
File "", line 673, in _load_unlocked
File "", line 697, in exec_module
File "", line 222, in _call_with_frames_removed
File "/Users/ipozdnya/matchminer-api/tests/unit/test_api/test_utilities.py", line 4, in
from matchminer.database import get_db
File "/Users/ipozdnya/matchminer-api/matchminer/database.py", line 4, in
from matchminer.settings import *
File "/Users/ipozdnya/matchminer-api/matchminer/settings.py", line 56, in
from matchminer.settings_dev import *
File "/Users/ipozdnya/matchminer-api/matchminer/settings_dev.py", line 20, in
TOKEN_TIMEOUT = sys.maxint
AttributeError: module 'sys' has no attribute 'maxint'

======================================================================
ERROR: Failure: ImportError (No module named 'engine')

Traceback (most recent call last):
File "/Users/ipozdnya/ve_matchminer/lib/python3.5/site-packages/nose/failure.py", line 39, in runTest
raise self.exc_val.with_traceback(self.tb)
File "/Users/ipozdnya/ve_matchminer/lib/python3.5/site-packages/nose/loader.py", line 418, in loadTestsFromName
addr.filename, addr.module)
File "/Users/ipozdnya/ve_matchminer/lib/python3.5/site-packages/nose/importer.py", line 47, in importFromPath
return self.importFromDir(dir_path, fqname)
File "/Users/ipozdnya/ve_matchminer/lib/python3.5/site-packages/nose/importer.py", line 94, in importFromDir
mod = load_module(part_fqname, fh, filename, desc)
File "/Users/ipozdnya/ve_matchminer/lib/python3.5/imp.py", line 235, in load_module
return load_source(name, filename, file)
File "/Users/ipozdnya/ve_matchminer/lib/python3.5/imp.py", line 172, in load_source
module = _load(spec)
File "", line 693, in _load
File "", line 673, in _load_unlocked
File "", line 697, in exec_module
File "", line 222, in _call_with_frames_removed
File "/Users/ipozdnya/matchminer-api/tests/test_cbioengine/test_engine.py", line 4, in
from tcm import CBioEngine
File "/Users/ipozdnya/matchminer-api/tcm/init.py", line 1, in
from engine import CBioEngine
ImportError: No module named 'engine'

======================================================================
ERROR: Failure: ImportError (No module named 'rfc822')

Traceback (most recent call last):
File "/Users/ipozdnya/ve_matchminer/lib/python3.5/site-packages/nose/failure.py", line 39, in runTest
raise self.exc_val.with_traceback(self.tb)
File "/Users/ipozdnya/ve_matchminer/lib/python3.5/site-packages/nose/loader.py", line 418, in loadTestsFromName
addr.filename, addr.module)
File "/Users/ipozdnya/ve_matchminer/lib/python3.5/site-packages/nose/importer.py", line 47, in importFromPath
return self.importFromDir(dir_path, fqname)
File "/Users/ipozdnya/ve_matchminer/lib/python3.5/site-packages/nose/importer.py", line 94, in importFromDir
mod = load_module(part_fqname, fh, filename, desc)
File "/Users/ipozdnya/ve_matchminer/lib/python3.5/imp.py", line 245, in load_module
return load_package(name, filename)
File "/Users/ipozdnya/ve_matchminer/lib/python3.5/imp.py", line 217, in load_package
return _load(spec)
File "", line 693, in _load
File "", line 673, in _load_unlocked
File "", line 697, in exec_module
File "", line 222, in _call_with_frames_removed
File "/Users/ipozdnya/matchminer-api/tests/test_matchminer/init.py", line 10, in
from rfc822 import formatdate
ImportError: No module named 'rfc822'

Looks like it's trying to import python3 packages for some reason even though my python is aliased to 2.7.
When I force it to run with python2.7, ( python2.7 -m nose tests ) another error happens:

ERROR: Failure: ImportError (No module named pytest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/Cellar/numpy/1.15.4_2/libexec/nose/lib/python2.7/site-packages/nose/loader.py", line 418, in loadTestsFromName
    addr.filename, addr.module)
  File "/usr/local/Cellar/numpy/1.15.4_2/libexec/nose/lib/python2.7/site-packages/nose/importer.py", line 47, in importFromPath
    return self.importFromDir(dir_path, fqname)
  File "/usr/local/Cellar/numpy/1.15.4_2/libexec/nose/lib/python2.7/site-packages/nose/importer.py", line 94, in importFromDir
    mod = load_module(part_fqname, fh, filename, desc)
  File "/Users/ipozdnya/Library/Python/2.7/lib/python/site-packages/tests/test_transport_http.py", line 35, in <module>
    import pytest
ImportError: No module named pytest

----------------------------------------------------------------------
Ran 28 tests in 0.963s

I checked for pytest:

$ python2.7 -m pip install pytest
Requirement already satisfied: pytest in /usr/local/lib/python2.7/site-packages (4.1.0)

What am I missing still?

Missing docker-compose.yml files for elasticsearch etc

I'm running into issues when trying to run matchminer UI locally. I can't find in the documentation where the docker-compose yaml files are for the elasticsearch docker containers. Can you advise where I should find that information.

Thanks

Miss matchminer-api/saml/settings_dev.json

Hi,

When I try to access https://localhost:5555, it returns This site cannot be reached. I checked log, it displayed the error below. I'm wondering if this file is missed. If yes, could you upload this file? Thanks!

endpoints

hey there. where can we find information about the api endpoints?

demo: "python pymm_run.py restore" step fails with ImportError: No module named ext.pymongo

After installing requirements (with saml2 package commented out, see issue 6) I tried "restore" step:

matchminer-api$ python --version
Python 2.7.15
matchminer-api$ python pymm_run.py restore
Traceback (most recent call last):
  File "pymm_run.py", line 7, in <module>
    from matchminer import managment
  File "/Users/ipozdnya/matchminer-api/matchminer/managment.py", line 10, in <module>
    from matchminer import events
  File "/Users/ipozdnya/matchminer-api/matchminer/events.py", line 12, in <module>
    from matchminer.hooks import UtilHooks
  File "/Users/ipozdnya/matchminer-api/matchminer/hooks/__init__.py", line 1, in <module>
    from matchminer.database import get_db
  File "/Users/ipozdnya/matchminer-api/matchminer/database.py", line 2, in <module>
    from flask.ext.pymongo import MongoClient
ImportError: No module named ext.pymongo

I double-checked flask_pymongo, seems to be in place:

matchminer-api$ pip2.7 install flask_pymongo
Collecting flask_pymongo
  Downloading https://files.pythonhosted.org/packages/d5/c8/22ddacfe05893884dceef5b9ecfa683f947ba155bd63cd9d841aea29b7b7/Flask_PyMongo-2.2.0-py2.py3-none-any.whl
Requirement already satisfied: Flask>=0.11 in /usr/local/lib/python2.7/site-packages (from flask_pymongo) (1.0.2)
Requirement already satisfied: PyMongo>=3.0 in /usr/local/lib/python2.7/site-packages (from flask_pymongo) (3.7.2)

Please advise.. Thanks

Link to demo data download gives 404

Deco docs:
Link https://drive.google.com/open?id=0B334b-HHVrwGc3djZUdvMktNekU gives 404

Dockerfile for development

Would be nice to have a docker-compose file spin up a dev environment with one command

demo: link to data samples is broken, 404

This link in demo docs gives 404:

https://drive.google.com/open?id=0B334b-HHVrwGc3djZUdvMktNekU