Giter Site home page Giter Site logo

higlass-manage's Introduction

Build Status Live Docs DOI Twitter Slack

Introduction

HiGlass is a web-based viewer for datasets too large to view at once. It features synchronized navigation of multiple views as well as continuous zooming and panning for navigation across genomic loci and resolutions. It supports visual comparison of genomic (e.g., Hi-C, ChIP-seq, or bed annotations) and other data (e.g., geographic maps, gigapixel images, or abstract 1D and 2D sequential data) from different experimental conditions and can be used to efficiently identify salient outcomes of experimental perturbations, generate new hypotheses, and share the results with the community.

A live instance can be found at https://higlass.io. A Docker container is available for running an instance locally, although we recommend using the higlass-manage package to start, stop and configure local instances.

For documentation about how to use and install HiGlass, please visit https://docs.higlass.io.

Citation

Kerpedjiev, P., Abdennur, N., Lekschas, F., McCallum, C., Dinkla, K., Strobelt, H., ... & Gehlenborg, N. HiGlass: Web-based Visual Exploration and Analysis of Genome Interaction Maps. Genome Biology (2018): 19:125. https://doi.org/10.1186/s13059-018-1486-1

Example

Development

To run higlass from its source code simply run the following:

npm clean-install // use --legacy-peer-deps if you get peer dependency errors
npm run start

This starts a server in development mode at http://localhost:5173/.

Warning The following examples need to be migrated to the latest build. Once started, a list of the examples can be found at http://localhost:8080/examples.html. Template viewconfs located at /docs/examples/viewconfs can viewed directly at urls such as http://localhost:8080/apis/svg.html?/viewconfs/overlay-tracks.json.

Tests

The tests for the React components and API functions are located in the test directory. Tests are run with web-test-runner, which you can learn more about the CLI in the documentation.

Useful commands:

  • Run all tests: npm test
  • Run all tests in interactive "watch" mode: npm test -- --watch
  • Run a specific test or "glob" of tests: npm test -- test/APITests.js [--watch]
  • Manually run individual tests in an open browser window: npm test -- --manual

Troubleshooting:

  • If the installation fails due to sharp > node-gyp try installing the node packages using python2:

    npm ci --python=/usr/bin/python2 && rm -rf node_modules/node-sass && npm ci
    

API

HiGlass provides an API for controlling the component from with JavaScript. Below is a minimal working example to get started and the complete documentation is availabe at docs.higlass.io.

<!DOCTYPE html>
<head>
  <meta charset="utf-8">
  <title>Minimal Working Example &middot; HiGlass</title>
  <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap.min.css">
  <link rel="stylesheet" href="https://unpkg.com/[email protected]/dist/hglib.css">

  <style type="text/css">
    html, body {
      width: 100vw;
      height: 100vh;
      overflow: hidden;
    }
  </style>

  <script crossorigin src="https://unpkg.com/react@16/umd/react.production.min.js"></script>
  <script crossorigin src="https://unpkg.com/react-dom@16/umd/react-dom.production.min.js"></script>
  <script crossorigin src="https://unpkg.com/pixi.js@5/dist/pixi.min.js"></script>
  <script crossorigin src="https://unpkg.com/[email protected]/dist/react-bootstrap.min.js"></script>
  <script crossorigin src="https://unpkg.com/[email protected]/dist/hglib.min.js"></script>
</head>
<body></body>
<script>
const hgApi = window.hglib.viewer(
  document.body,
  'https://higlass.io/api/v1/viewconfs/?d=default',
  { bounded: true },
);
</script>
</html>

Related

diagram of related tools

  • HiGlass Clodius - Package that provides implementations for aggregation and tile generation for many common 1D and 2D data types
  • HiGlass Python - Python bindings to the HiGlass for tile serving, view config generation, and Jupyter Notebook + Lab integration.
  • HiGlass Manage - Easy to use interface for deploying a local HiGlass instance
  • HiGlass Docker - Build an image containing all the components necessary to deploy HiGlass
  • HiGlass Server - Server component for serving multi-resolution data
  • HiGlass App - The code for the web application hosted at https://higlass.io
  • Cooler - Package for efficient storage of and access to sparse 2D data

License

HiGlass is provided under the MIT License.

higlass-manage's People

Contributors

alexpreynolds avatar flekschas avatar mccalluc avatar nboughton avatar nvictus avatar pkerpedjiev avatar pkerpedjiev-zymergen avatar sergpolly avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

higlass-manage's Issues

upload data into higlass-manage

Hi,
I try to follow this tutorial (https://docs.higlass.io/higlass_docker.html#running-locally) to upload data into HiGlass,
docker run --detach
--publish 8989:80 -------Do I need to change this argument?
--volume ~/hg-data:/data --------How to set this agrument?
--volume ~/tmp:/tmp ----------How to set this agrument?
--name higlass-container
-e SITE_URL=my.higlass.org
higlass/higlass-docker:v0.6.1

the format of input file I want to upload is .txt, do I need to convert it into the .cool format before uploading?
Best,

Timeout Issues with Starting higlass-manage

Hi there,

I am trying to using higlass-manage to view contact maps of my cooler files. I keep getting the following Timeout errors and I'm not entirely sure what to do to fix it. Any help would be really appreciated, thank you.

ariel@aphrodite:/$ higlass-manage start
Stopping previously running container
Pulling latest image... 
done
Data directory: /home/ariel/hg-data
Temp directory: ()
Starting... default 8989
Docker started: higlass-manage-container-default
sending request 1
Traceback (most recent call last):
  File "/home/ariel/.local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 426, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/home/ariel/.local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 421, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/lib/python3.6/http/client.py", line 1356, in getresponse
    response.begin()
  File "/usr/lib/python3.6/http/client.py", line 307, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python3.6/http/client.py", line 268, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/lib/python3.6/socket.py", line 586, in readinto
    return self._sock.recv_into(b)
socket.timeout: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ariel/.local/lib/python3.6/site-packages/requests/adapters.py", line 449, in send
    timeout=timeout
  File "/home/ariel/.local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 727, in urlopen
    method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
  File "/home/ariel/.local/lib/python3.6/site-packages/urllib3/util/retry.py", line 403, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/home/ariel/.local/lib/python3.6/site-packages/urllib3/packages/six.py", line 735, in reraise
    raise value
  File "/home/ariel/.local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 677, in urlopen
    chunked=chunked,
  File "/home/ariel/.local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 428, in _make_request
    self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
  File "/home/ariel/.local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 336, in _raise_timeout
    self, url, "Read timed out. (read timeout=%s)" % timeout_value
urllib3.exceptions.ReadTimeoutError: HTTPConnectionPool(host='gw-srv-01.rc.zi.columbia.edu', port=3128): Read timed out. (read timeout=5)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ariel/.local/lib/python3.6/site-packages/higlass_manage/start.py", line 368, in _start
    timeout=5,
  File "/home/ariel/.local/lib/python3.6/site-packages/requests/api.py", line 76, in get
    return request('get', url, params=params, **kwargs)
  File "/home/ariel/.local/lib/python3.6/site-packages/requests/api.py", line 61, in request
    return session.request(method=method, url=url, **kwargs)
  File "/home/ariel/.local/lib/python3.6/site-packages/requests/sessions.py", line 530, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/ariel/.local/lib/python3.6/site-packages/requests/sessions.py", line 643, in send
    r = adapter.send(request, **kwargs)
  File "/home/ariel/.local/lib/python3.6/site-packages/requests/adapters.py", line 529, in send
    raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: HTTPConnectionPool(host='gw-srv-01.rc.zi.columbia.edu', port=3128): Read timed out. (read timeout=5)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ariel/.local/bin/higlass-manage", line 11, in <module>
    sys.exit(cli())
  File "/home/ariel/.local/lib/python3.6/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/home/ariel/.local/lib/python3.6/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/home/ariel/.local/lib/python3.6/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/ariel/.local/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/ariel/.local/lib/python3.6/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/home/ariel/.local/lib/python3.6/site-packages/higlass_manage/start.py", line 150, in start
    redis_port,
  File "/home/ariel/.local/lib/python3.6/site-packages/higlass_manage/start.py", line 384, in _start
    except requests.exceptions.ReadTimout:
AttributeError: module 'requests.exceptions' has no attribute 'ReadTimout'

500 Server Error: Internal Server Error ("b'Mounts denied: EOF'")

Hi,
Any idea what could be causing this error? It started happening out of the blue today:

state False
HiGlass not running. Starting...
Stopping previously running container
Pulling latest image... doneData directory: /Users/xinyangbing/hg-data
Temp directory: /tmp/higlass-docker
Starting... default 8989
Traceback (most recent call last):
File "/Users/xinyangbing/miniconda3/lib/python3.8/site-packages/higlass_manage/ingest.py", line 66, in _ingest
get_temp_dir(hg_name)
File "/Users/xinyangbing/miniconda3/lib/python3.8/site-packages/higlass_manage/common.py", line 198, in get_temp_dir
raise HiGlassNotRunningException()
higlass_manage.common.HiGlassNotRunningException

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/Users/xinyangbing/miniconda3/lib/python3.8/site-packages/docker/api/client.py", line 259, in _raise_for_status
response.raise_for_status()
File "/Users/xinyangbing/miniconda3/lib/python3.8/site-packages/requests/models.py", line 939, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 500 Server Error: Internal Server Error for url: http+docker://localhost/v1.41/containers/c75641576a15e33ed3b42f8ff195a7b92346334ede58fede22b38418a6358792/start

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/Users/xinyangbing/miniconda3/bin/higlass-manage", line 8, in
sys.exit(cli())
File "/Users/xinyangbing/miniconda3/lib/python3.8/site-packages/click/core.py", line 829, in call
return self.main(*args, **kwargs)
File "/Users/xinyangbing/miniconda3/lib/python3.8/site-packages/click/core.py", line 782, in main
rv = self.invoke(ctx)
File "/Users/xinyangbing/miniconda3/lib/python3.8/site-packages/click/core.py", line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/Users/xinyangbing/miniconda3/lib/python3.8/site-packages/click/core.py", line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/Users/xinyangbing/miniconda3/lib/python3.8/site-packages/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "/Users/xinyangbing/miniconda3/lib/python3.8/site-packages/higlass_manage/ingest.py", line 40, in ingest
_ingest(filename,
File "/Users/xinyangbing/miniconda3/lib/python3.8/site-packages/higlass_manage/ingest.py", line 69, in _ingest
_start(hg_name=hg_name)
File "/Users/xinyangbing/miniconda3/lib/python3.8/site-packages/higlass_manage/start.py", line 135, in _start
container = client.containers.run(image,
File "/Users/xinyangbing/miniconda3/lib/python3.8/site-packages/docker/models/containers.py", line 816, in run
container.start()
File "/Users/xinyangbing/miniconda3/lib/python3.8/site-packages/docker/models/containers.py", line 404, in start
return self.client.api.start(self.id, **kwargs)
File "/Users/xinyangbing/miniconda3/lib/python3.8/site-packages/docker/utils/decorators.py", line 19, in wrapped
return f(self, resource_id, *args, **kwargs)
File "/Users/xinyangbing/miniconda3/lib/python3.8/site-packages/docker/api/container.py", line 1108, in start
self._raise_for_status(res)
File "/Users/xinyangbing/miniconda3/lib/python3.8/site-packages/docker/api/client.py", line 261, in _raise_for_status
raise create_api_error_from_http_exception(e)
File "/Users/xinyangbing/miniconda3/lib/python3.8/site-packages/docker/errors.py", line 31, in create_api_error_from_http_exception
raise cls(e, response=response, explanation=explanation)
docker.errors.APIError: 500 Server Error: Internal Server Error ("b'Mounts denied: EOF'")

requests / urllib3 error

Hello,

I'm having an error that was also mentioned in #65, and so is my colleague on a different OS version.

Installing higlass-manage in a fresh python virtual env with Python 3.9 on MacOS Mojave gives the following error:

Error traceback
(higlass-env) mpims-lingsim:~ lingsim$ higlass-manage start
Traceback (most recent call last):
  File "/Users/lingsim/higlass-env/lib/python3.9/site-packages/urllib3/response.py", line 572, in _update_chunk_length
    self.chunk_left = int(line, 16)
ValueError: invalid literal for int() with base 16: b''

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/lingsim/higlass-env/lib/python3.9/site-packages/urllib3/response.py", line 331, in _error_catcher
    yield
  File "/Users/lingsim/higlass-env/lib/python3.9/site-packages/urllib3/response.py", line 637, in read_chunked
    self._update_chunk_length()
  File "/Users/lingsim/higlass-env/lib/python3.9/site-packages/urllib3/response.py", line 576, in _update_chunk_length
    raise httplib.IncompleteRead(line)
http.client.IncompleteRead: IncompleteRead(0 bytes read)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/lingsim/higlass-env/lib/python3.9/site-packages/requests/models.py", line 749, in generate
    for chunk in self.raw.stream(chunk_size, decode_content=True):
  File "/Users/lingsim/higlass-env/lib/python3.9/site-packages/urllib3/response.py", line 461, in stream
    for line in self.read_chunked(amt, decode_content=decode_content):
  File "/Users/lingsim/higlass-env/lib/python3.9/site-packages/urllib3/response.py", line 665, in read_chunked
    self._original_response.close()
  File "/usr/local/Cellar/[email protected]/3.9.0_2/Frameworks/Python.framework/Versions/3.9/lib/python3.9/contextlib.py", line 135, in __exit__
    self.gen.throw(type, value, traceback)
  File "/Users/lingsim/higlass-env/lib/python3.9/site-packages/urllib3/response.py", line 349, in _error_catcher
    raise ProtocolError('Connection broken: %r' % e, e)
urllib3.exceptions.ProtocolError: ('Connection broken: IncompleteRead(0 bytes read)', IncompleteRead(0 bytes read))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/lingsim/higlass-env/bin/higlass-manage", line 8, in <module>
    sys.exit(cli())
  File "/Users/lingsim/higlass-env/lib/python3.9/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/Users/lingsim/higlass-env/lib/python3.9/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/Users/lingsim/higlass-env/lib/python3.9/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/lingsim/higlass-env/lib/python3.9/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/lingsim/higlass-env/lib/python3.9/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/Users/lingsim/higlass-env/lib/python3.9/site-packages/higlass_manage/start.py", line 134, in start
    _start(
  File "/Users/lingsim/higlass-env/lib/python3.9/site-packages/higlass_manage/start.py", line 180, in _start
    hg_container = client.containers.get(hg_container_name)
  File "/Users/lingsim/higlass-env/lib/python3.9/site-packages/docker/models/containers.py", line 889, in get
    resp = self.client.api.inspect_container(container_id)
  File "/Users/lingsim/higlass-env/lib/python3.9/site-packages/docker/utils/decorators.py", line 19, in wrapped
    return f(self, resource_id, *args, **kwargs)
  File "/Users/lingsim/higlass-env/lib/python3.9/site-packages/docker/api/container.py", line 774, in inspect_container
    self._get(self._url("/containers/{0}/json", container)), True
  File "/Users/lingsim/higlass-env/lib/python3.9/site-packages/docker/utils/decorators.py", line 46, in inner
    return f(self, *args, **kwargs)
  File "/Users/lingsim/higlass-env/lib/python3.9/site-packages/docker/api/client.py", line 237, in _get
    return self.get(url, **self._set_request_timeout(kwargs))
  File "/Users/lingsim/higlass-env/lib/python3.9/site-packages/requests/sessions.py", line 525, in get
    return self.request('GET', url, **kwargs)
  File "/Users/lingsim/higlass-env/lib/python3.9/site-packages/requests/sessions.py", line 512, in request
    resp = self.send(prep, **send_kwargs)
  File "/Users/lingsim/higlass-env/lib/python3.9/site-packages/requests/sessions.py", line 662, in send
    r.content
  File "/Users/lingsim/higlass-env/lib/python3.9/site-packages/requests/models.py", line 827, in content
    self._content = b''.join(self.iter_content(CONTENT_CHUNK_SIZE)) or b''
  File "/Users/lingsim/higlass-env/lib/python3.9/site-packages/requests/models.py", line 752, in generate
    raise ChunkedEncodingError(e)
requests.exceptions.ChunkedEncodingError: ('Connection broken: IncompleteRead(0 bytes read)', IncompleteRead(0 bytes read))

I was able to fix this by editing response.py as suggested here #65 (comment) (well, I get a timeout error at the moment but as I mentioned in slack I suspect that a server needs restarting?)

My colleague had the same issue on MacOS Catalina and the same fix worked. But it's obviously not ideal to have to manually edit files inside installed packages! I'm not sure why this is happening now since this issue seems to have been around in urllib3 for a couple of years. Could you see again if you can reproduce this and if there's a more robust fix? Let me know if I can provide any more information to help reproduce or debug this. Thank you!

HiGlass stucked at loading

I've used higlass-manage and could uploaded and visualize a mcool and bed files locally. But when I tried to visualize bw file in the browser, it kept "loading" and never showed up. The bw files can be visualized by igv.

Possible issues with options in v0.2.0?

I am running into some problems with running v0.2.0 of higlass-manage.

The installation seemed to go okay:

# pip uninstall higlass-manage
...
# pip install higlass-manage
...
Successfully built higlass-manage
Installing collected packages: higlass-manage
Successfully installed higlass-manage-0.2.0

Once I try to start a server, though, I get an unexpected error:

# higlass-manage start --port 80 --site-url hg-dev-rev2.altius.org
Usage: higlass-manage [OPTIONS] COMMAND [ARGS]...

Error: No such command "start".

The start option seems to have changed to _start:

# higlass-manage
Usage: higlass-manage [OPTIONS] COMMAND [ARGS]...

Options:
  --help  Show this message and exit.

Commands:
  _start
  browse           Launch a web browser for a running instance
  createsuperuser  Create a superuser in the container...
  deletesuperuser  Delete a superuser in the container...
  ingest           Ingest a dataset
  list             List running instances
  list_data        List the datasets in an instance
  log              Return the error log for this container
  shell            Start a shell in a higlass container
  stop             Stop a running instance
  view             View a file in higlass.

Also, if I use this option, the other options do not seem to be processed. For instance, if I specify a site-url parameter:

# higlass-manage _start --port 80 --site-url example.com
Traceback (most recent call last):
  File "/root/miniconda3/bin/higlass-manage", line 11, in <module>
    sys.exit(cli())
  File "/root/miniconda3/lib/python3.6/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/root/miniconda3/lib/python3.6/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/root/miniconda3/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/root/miniconda3/lib/python3.6/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/root/miniconda3/lib/python3.6/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
TypeError: _start() got an unexpected keyword argument 'site_url'

I was able to downgrade to v0.1.13, but the v0.1.14 tag was not available through pip:

# pip install higlass-manage==0.1.14
Collecting higlass-manage==0.1.14
  Could not find a version that satisfies the requirement higlass-manage==0.1.14 (from versions: 0.1.1, 0.1.2, 0.1.4, 0.1.5, 0.1.6, 0.1.7, 0.1.9, 0.1.10, 0.1.11, 0.1.12, 0.1.13, 0.2.0)
No matching distribution found for higlass-manage==0.1.14

`higlass-manage view file.mcool` -> ModuleNotFoundError

higlass-manage$ higlass-manage view ~/Downloads/4DNFI2A4OBS9.mcool 
Pulling latest image... 
done
Data directory: /Users/chuck/hg-data
Temp directory: ()
Starting... default 8989
Docker started: higlass-manage-container-default
sending request 1
...
Waiting to start (tilesets)...
...
Non 200 status code returned (502), waiting...
sending request 15
public_data: True
Replaced js file
Started
Inferred filetype: cooler
Inferred datatype: matrix
state True
Inferred datatype: matrix
state True
state True
name_text: 
hg_name: default
command: python higlass-server/manage.py ingest_tileset --filename /tmp/4DNFI2A4OBS9.mcool --filetype cooler --datatype matrix    --uid YFkVEOCtR_6w8HCkpdyAMg
Traceback (most recent call last):
  File "/Users/chuck/anaconda3/envs/higlass-tutorial/bin/higlass-manage", line 10, in <module>
    sys.exit(cli())
  File "/Users/chuck/anaconda3/envs/higlass-tutorial/lib/python3.7/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/Users/chuck/anaconda3/envs/higlass-tutorial/lib/python3.7/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/Users/chuck/anaconda3/envs/higlass-tutorial/lib/python3.7/site-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/chuck/anaconda3/envs/higlass-tutorial/lib/python3.7/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/chuck/anaconda3/envs/higlass-tutorial/lib/python3.7/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/Users/chuck/anaconda3/envs/higlass-tutorial/lib/python3.7/site-packages/higlass_manage/view.py", line 114, in view
    import higlass.client as hgc
ModuleNotFoundError: No module named 'higlass'

pip freeze:

$ pip freeze
asciitree==0.3.3
biopython==1.74
certifi==2019.6.16
chardet==3.0.4
Click==7.0
clodius==0.10.13
cooler==0.8.5
cytoolz==0.10.0
dask==2.2.0
dill==0.3.0
docker==4.0.2
h5py==2.9.0
higlass-manage==0.7.2
idna==2.8
multiprocess==0.70.8
negspy==0.2.22
nose==1.3.7
numpy==1.17.0
pandas==0.25.0
pybbi==0.2.0
pyfaidx==0.5.5.2
pypairix==0.3.7
pysam==0.15.3
python-dateutil==2.8.0
pytz==2019.2
PyYAML==5.1.2
requests==2.22.0
scipy==1.3.0
six==1.12.0
slugid==2.0.0
sortedcontainers==2.1.0
toolz==0.10.0
urllib3==1.25.3
websocket-client==0.56.0

In test.sh, we ingest before view, so maybe something is not being exercised with this path?

Install error - pybbi fails to build

Running: OSX 10.12.6; Xcode 9.2; Xcode CLT 2347; Python 3.7.1; Conda 4.6.11; Docker Engine 18.09.2.

Hi, thanks for making this. Excited to use it but I'm having an install problem. After calling pip install higlass-manage everything goes fine until....

Running setup.py bdist_wheel for pysam ... done Stored in directory: /Users/demmerle/Library/Caches/pip/wheels/34/3b/bc/f3faaba907f333632357aae7bf09a453209a8c9c3cec4cddae Running setup.py bdist_wheel for pybbi ... error Complete output from command /anaconda3/bin/python -u -c "import setuptools, tokenize;__file__='/private/var/folders/p_/xr0cxrx12msf6pb1xf0r07r40000gp/T/pip-install-74z1ht_7/pybbi/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /private/var/folders/p_/xr0cxrx12msf6pb1xf0r07r40000gp/T/pip-wheel-wum70wlg --python-tag cp37: running bdist_wheel running build running build_py creating build creating build/lib.macosx-10.7-x86_64-3.7 creating build/lib.macosx-10.7-x86_64-3.7/bbi copying bbi/__init__.py -> build/lib.macosx-10.7-x86_64-3.7/bbi running build_ext Compiling libkent... LIBRARY_PATH: /usr/local/opt/openssl/lib: C_INCLUDE_PATH: /anaconda3/include/libpng16:/anaconda3/include/openssl:/usr/local/opt/openssl/include:/usr/local/include/libpng16: cd src && /Applications/Xcode.app/Contents/Developer/usr/bin/make gcc -g -pthread -fPIC -static -Wall -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_GNU_SOURCE -DMACHTYPE_x86_64 -L/private/var/folders/p_/xr0cxrx12msf6pb1xf0r07r40000gp/T/pip-install-74z1ht_7/pybbi/src/x86_64 -L/usr/lib -lz -lc -lpthread -I/private/var/folders/p_/xr0cxrx12msf6pb1xf0r07r40000gp/T/pip-install-74z1ht_7/pybbi/include -I/private/var/folders/p_/xr0cxrx12msf6pb1xf0r07r40000gp/T/pip-install-74z1ht_7/pybbi/src -I/usr/include -o aliType.o -c aliType.c clang: warning: -lz: 'linker' input unused [-Wunused-command-line-argument] clang: warning: -lc: 'linker' input unused [-Wunused-command-line-argument] clang: warning: -lpthread: 'linker' input unused [-Wunused-command-line-argument] clang: warning: argument unused during compilation: '-L/private/var/folders/p_/xr0cxrx12msf6pb1xf0r07r40000gp/T/pip-install-74z1ht_7/pybbi/src/x86_64' [-Wunused-command-line-argument] clang: warning: argument unused during compilation: '-L/usr/lib' [-Wunused-command-line-argument]

...at which point the gcc command runs repeatedly for maybe 100 iterations before installing the rest of the wheels, and then printing it's last error message
udc.c:39:10: fatal error: 'openssl/sha.h' file not found #include <openssl/sha.h> ^~~~~~~~~~~~~~~ 1 error generated. make[1]: *** [udc.o] Error 1 make: *** [src/x86_64/libkent.a] Error 2 Traceback (most recent call last): File "<string>", line 1, in <module> File "/private/var/folders/p_/xr0cxrx12msf6pb1xf0r07r40000gp/T/pip-install-74z1ht_7/pybbi/setup.py", line 152, in <module> 'build_ext': build_ext File "/anaconda3/lib/python3.7/site-packages/setuptools/__init__.py", line 143, in setup return distutils.core.setup(**attrs) File "/anaconda3/lib/python3.7/distutils/core.py", line 148, in setup dist.run_commands() File "/anaconda3/lib/python3.7/distutils/dist.py", line 966, in run_commands self.run_command(cmd) File "/anaconda3/lib/python3.7/distutils/dist.py", line 985, in run_command cmd_obj.run() File "/anaconda3/lib/python3.7/site-packages/setuptools/command/install.py", line 61, in run return orig.install.run(self) File "/anaconda3/lib/python3.7/distutils/command/install.py", line 545, in run self.run_command('build') File "/anaconda3/lib/python3.7/distutils/cmd.py", line 313, in run_command self.distribution.run_command(command) File "/anaconda3/lib/python3.7/distutils/dist.py", line 985, in run_command cmd_obj.run() File "/anaconda3/lib/python3.7/distutils/command/build.py", line 135, in run self.run_command(cmd_name) File "/anaconda3/lib/python3.7/distutils/cmd.py", line 313, in run_command self.distribution.run_command(command) File "/anaconda3/lib/python3.7/distutils/dist.py", line 985, in run_command cmd_obj.run() File "/private/var/folders/p_/xr0cxrx12msf6pb1xf0r07r40000gp/T/pip-install-74z1ht_7/pybbi/setup.py", line 86, in run check_call(['make', 'build-c']) File "/anaconda3/lib/python3.7/subprocess.py", line 341, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command '['make', 'build-c']' returned non-zero exit status 2.

...and ends with this...
Command "/anaconda3/bin/python -u -c "import setuptools, tokenize;__file__='/private/var/folders/p_/xr0cxrx12msf6pb1xf0r07r40000gp/T/pip-install-74z1ht_7/pybbi/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /private/var/folders/p_/xr0cxrx12msf6pb1xf0r07r40000gp/T/pip-record-mgajcosf/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /private/var/folders/p_/xr0cxrx12msf6pb1xf0r07r40000gp/T/pip-install-74z1ht_7/pybbi/

I tried pre-installing setuptools 0.40.9, but that didn't make a difference. I assume it's something silly in my mac environment, but I can't figure it out and was hoping you might have some thoughts. Thanks!

Installation issue

Not sure if this is related issue #4 , but after installing higlass-manage on a python3 environment, I tried higlass-manage start, and it gave the following error message.

/anaconda2/envs/python36/lib/python3.6/site-packages/h5py/init.py:36: FutureWarning: Conversion of the second argument of issubdtype from float to np.floating is deprecated. In future, it will be treated as np.float64 == np.dtype(float).type.
from ._conv import register_converters as _register_converters
Traceback (most recent call last):
File "/anaconda2/envs/python36/lib/python3.6/site-packages/urllib3/connectionpool.py", line 601, in urlopen
chunked=chunked)
File "/anaconda2/envs/python36/lib/python3.6/site-packages/urllib3/connectionpool.py", line 357, in _make_request
conn.request(method, url, **httplib_request_kw)
File "/anaconda2/envs/python36/lib/python3.6/http/client.py", line 1239, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/anaconda2/envs/python36/lib/python3.6/http/client.py", line 1285, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/anaconda2/envs/python36/lib/python3.6/http/client.py", line 1234, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/anaconda2/envs/python36/lib/python3.6/http/client.py", line 1026, in _send_output
self.send(msg)
File "/anaconda2/envs/python36/lib/python3.6/http/client.py", line 964, in send
self.connect()
File "/anaconda2/envs/python36/lib/python3.6/site-packages/docker/transport/unixconn.py", line 42, in connect
sock.connect(self.unix_socket)
FileNotFoundError: [Errno 2] No such file or directory

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/anaconda2/envs/python36/lib/python3.6/site-packages/requests/adapters.py", line 440, in send
timeout=timeout
File "/anaconda2/envs/python36/lib/python3.6/site-packages/urllib3/connectionpool.py", line 639, in urlopen
_stacktrace=sys.exc_info()[2])
File "/anaconda2/envs/python36/lib/python3.6/site-packages/urllib3/util/retry.py", line 357, in increment
raise six.reraise(type(error), error, _stacktrace)
File "/anaconda2/envs/python36/lib/python3.6/site-packages/urllib3/packages/six.py", line 685, in reraise
raise value.with_traceback(tb)
File "/anaconda2/envs/python36/lib/python3.6/site-packages/urllib3/connectionpool.py", line 601, in urlopen
chunked=chunked)
File "/anaconda2/envs/python36/lib/python3.6/site-packages/urllib3/connectionpool.py", line 357, in _make_request
conn.request(method, url, **httplib_request_kw)
File "/anaconda2/envs/python36/lib/python3.6/http/client.py", line 1239, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/anaconda2/envs/python36/lib/python3.6/http/client.py", line 1285, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/anaconda2/envs/python36/lib/python3.6/http/client.py", line 1234, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/anaconda2/envs/python36/lib/python3.6/http/client.py", line 1026, in _send_output
self.send(msg)
File "/anaconda2/envs/python36/lib/python3.6/http/client.py", line 964, in send
self.connect()
File "/anaconda2/envs/python36/lib/python3.6/site-packages/docker/transport/unixconn.py", line 42, in connect
sock.connect(self.unix_socket)
urllib3.exceptions.ProtocolError: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/anaconda2/envs/python36/bin/higlass-manage", line 11, in
sys.exit(cli())
File "/anaconda2/envs/python36/lib/python3.6/site-packages/click/core.py", line 722, in call
return self.main(*args, **kwargs)
File "/anaconda2/envs/python36/lib/python3.6/site-packages/click/core.py", line 697, in main
rv = self.invoke(ctx)
File "/anaconda2/envs/python36/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/anaconda2/envs/python36/lib/python3.6/site-packages/click/core.py", line 895, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/anaconda2/envs/python36/lib/python3.6/site-packages/click/core.py", line 535, in invoke
return callback(*args, **kwargs)
File "/anaconda2/envs/python36/lib/python3.6/site-packages/higlass_manage.py", line 296, in start
container = client.containers.get(container_name)
File "/anaconda2/envs/python36/lib/python3.6/site-packages/docker/models/containers.py", line 843, in get
resp = self.client.api.inspect_container(container_id)
File "/anaconda2/envs/python36/lib/python3.6/site-packages/docker/utils/decorators.py", line 19, in wrapped
return f(self, resource_id, *args, **kwargs)
File "/anaconda2/envs/python36/lib/python3.6/site-packages/docker/api/container.py", line 730, in inspect_container
self._get(self._url("/containers/{0}/json", container)), True
File "/anaconda2/envs/python36/lib/python3.6/site-packages/docker/utils/decorators.py", line 46, in inner
return f(self, *args, **kwargs)
File "/anaconda2/envs/python36/lib/python3.6/site-packages/docker/api/client.py", line 198, in _get
return self.get(url, **self._set_request_timeout(kwargs))
File "/anaconda2/envs/python36/lib/python3.6/site-packages/requests/sessions.py", line 521, in get
return self.request('GET', url, **kwargs)
File "/anaconda2/envs/python36/lib/python3.6/site-packages/requests/sessions.py", line 508, in request
resp = self.send(prep, **send_kwargs)
File "/anaconda2/envs/python36/lib/python3.6/site-packages/requests/sessions.py", line 618, in send
r = adapter.send(request, **kwargs)
File "/anaconda2/envs/python36/lib/python3.6/site-packages/requests/adapters.py", line 490, in send
raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))

Not sure if this will be fixed in v0.1.14. I installed v0.1.13 through pip. I also tried some earlier versions, and they all gave the same error message. What might be the problem here? Is it related to that I am installing in a python3 environment? Thanks.

Delete tileset

How do you remove a tileset? I accidentally added a chrom sizes file which is now preventing me from adding bigwig files:
ERROR: CommandError: More than one available set of chromSizesfor this coordSystem

how to visualize pre-normalized data from HiCNorm in higlass

I've created a mcool file that can be ingested into the docker version higlass using higlass-mange ingest.

The mcool file has one hicnorm column (which is pre-calculated using HiCNorm) in addition to count column

$ cooler dump ${mcool}::resolutions/$res -H |awk '($4!=0)' |head

bin1_id bin2_id count   hicnorm
72648   72648   1351    73.4043
72648   72649   588     27.775
72648   72650   265     13.4901

My goal is to visualize this hicnorm values in higlass. But currently, I can only see raw counts in higlass.

Can you help me with this?

thanks!

No public data

Hi,

I'm trying to view Higlass public data locally. After running higlass-manage, I find no tileset.

`
higlass-manage start --port 5406 --public-data

Stopping previously running container
Pulling latest image...
done
Data directory: /home/yjliu/hg-data
Temp directory: ()
Starting... default 5406
Docker started: higlass-manage-container-default
sending request 1
Waiting to start (tilesets)...
sending request 2
Waiting to start (tilesets)...
sending request 3
Waiting to start (tilesets)...
sending request 4
Non 200 status code returned (502), waiting...
sending request 5
Non 200 status code returned (502), waiting...
sending request 6
Non 200 status code returned (502), waiting...
sending request 7
Non 200 status code returned (502), waiting...
sending request 8
Non 200 status code returned (502), waiting...
sending request 9
Non 200 status code returned (502), waiting...
sending request 10
Non 200 status code returned (502), waiting...
sending request 11
Non 200 status code returned (502), waiting...
sending request 12
Non 200 status code returned (502), waiting...
sending request 13
Non 200 status code returned (502), waiting...
sending request 14
Non 200 status code returned (502), waiting...
sending request 15
Non 200 status code returned (502), waiting...
sending request 16
Non 200 status code returned (502), waiting...
sending request 17
Non 200 status code returned (502), waiting...
sending request 18
Non 200 status code returned (502), waiting...
sending request 19
Non 200 status code returned (502), waiting...
sending request 20
Non 200 status code returned (502), waiting...
sending request 21
Non 200 status code returned (502), waiting...
sending request 22
Non 200 status code returned (502), waiting...
sending request 23
public_data: True
Replaced js file
Started
`

image

Duplicate entries for uploaded data

I get 2 entries for any data I upload through the admin UI or ingest with higlass-manage.

Example command:

higlass-manage ingest --project-name my-data --name WTSI-OESO_103 raw-data/WTSI-OESO_103.mcool

Everything installed fresh today, have stopped and cleared the hg-data/hg-tmp directories.

mount of /tmp different in start than in ingest on MacOS (workaround provided)

I'm not experienced with Docker, so perhaps I'm doing something dumb, but when trying to higlass-manage view a file, the file is linked to the local directory that is supposed to be mounted to /tmp in the docker image, but when trying to tell the image to ingest the file, the file isn't found. The problem appears to be that when getting the docker configuration with client.api.inspect_container(container_name), it returns the following (just showing the /tmp mount):

{'Type': 'bind', 'Source': '/host_mnt/private/tmp/higlass-docker', 'Destination': '/tmp', 'Mode': 'rw', 'RW': True, 'Propagation': 'rprivate'}

However, this is not the mapping that was set up by higlass-manage start. There, the default mapping is temp_dir="/tmp/higlass-docker".

Work-around is to symlink: cd /tmp; ln -s /host_mnt/private/tmp/higlass-docker .

Note also that because one can't create directories in / on MacOS Catalina, I needed to use synthetic.conf to make /host_mnt a symlink to a directory in my user directory.

NameError: name 'cca' is not defined

On Ubuntu 18.04, installed higlass-manage via pip3 and had to also install the following packages as build dependencies:

zlib1g-dev
libpng-dev
libssl-dev

When attempting to ingest a .bed file I get the following error:

higlass-manage ingest all_fDanKya3_1.sort.filteredPrimary.COVERAGE_ZERODEPTH_4gEVAL.bed --filetype bedfile --datatype bedlike --name fDanKya3_ZERODEPTH --project-name fDanKya3 --chromsizes-filename my.genome
Aggregating bedfile
Traceback (most recent call last):
  File "/usr/local/bin/higlass-manage", line 11, in <module>
    sys.exit(cli())
  File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/higlass_manage/ingest.py", line 45, in ingest
    project_name
  File "/usr/local/lib/python3.6/dist-packages/higlass_manage/ingest.py", line 67, in _ingest
    (to_import, filetype) = aggregate_file(filename, filetype, assembly, chromsizes_filename, has_header, no_upload, td)
  File "/usr/local/lib/python3.6/dist-packages/higlass_manage/ingest.py", line 83, in aggregate_file
    cca._bedfile(filename,
NameError: name 'cca' is not defined

I note that cca is not defined or imported in ingest.py.

old viewconf-s not working after migrating higlass instance

  • context:
    we had a local higlass on 1 machine (with a lot of things ingested) - now we need to move it to a new computer (different network). Ideally such that everything would remain functional, including saved viewconf-s.

  • what we do:
    grab an existing db.sqlite along with hg-data folder - copy those to a new computer, relaunch higlass-docker container with --hg-data pointing to the copied hg-data folder. Done! - everything works, ingested data is there. but ...

  • problem:
    of course viewconf-s saved in the database point to previous location for most of the tilesets. So, if we shut down the old machine viewconfs would stop working ...

  • question:
    is there a script/pulg-in/tutorial for migrating a higlass ?
    if not, would it be cool to have one ? as part of higlass-manage ?
    would a solution based on manual sqlite's database UPDATE ... table ... replace be ok e.g. https://stackoverflow.com/questions/10532656/sqlite-replace-part-of-a-string ?

Empty view fresh install

With higlass-manage I have a blank view when I try and run a fresh container. I ran higlass-manage start, and I was able to ingest a tileset. The server is running where it should, but my empty higlass page looks more empty than I expect.

Screen Shot 2020-05-06 at 3 42 55 PM

I am able to right click and add rulers but there is not header for the container with the cog for editing a viewconf.
Screen Shot 2020-05-06 at 3 44 12 PM

Also, higlass-manage doesn't have a "log" subcommand like the documentation says..

I am running higlass-manage 0.8.0

no vertical scrollbar in v0.6.34

image

^ I am using Chrome to view the tracks. The tracks below the view window are not accessible for view (i.e. there is no vertical scrollbar)

Ingesting issue

Hello, I'm trying to ingest a cool file for viewing in a browser. I'm able to run the ingest command as sudo higlass-manage ingest Caco2Dif_1000000.1000.iced.cool which runs without apparent error. When I go to the browser to try and view it however, it just shows a grey box with 'Loading...' indefinitely in the corner. When I check the logs, it does show an error traceback:

27/Sep/2018 18:17:02] ERROR [django.request:118] Internal Server Error: /api/v1/tileset_info/
Traceback (most recent call last):
  File "/opt/conda/lib/python3.6/site-packages/django/core/handlers/exception.py", line 35, in inner
    response = get_response(request)
  File "/opt/conda/lib/python3.6/site-packages/django/core/handlers/base.py", line 128, in _get_response
    response = self.process_exception_by_middleware(e, request)
  File "/opt/conda/lib/python3.6/site-packages/django/core/handlers/base.py", line 126, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/opt/conda/lib/python3.6/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view
    return view_func(*args, **kwargs)
  File "/opt/conda/lib/python3.6/site-packages/django/views/generic/base.py", line 69, in view
    return self.dispatch(request, *args, **kwargs)
  File "/opt/conda/lib/python3.6/site-packages/rest_framework/views.py", line 489, in dispatch
    response = self.handle_exception(exc)
  File "/opt/conda/lib/python3.6/site-packages/rest_framework/views.py", line 449, in handle_exception
    self.raise_uncaught_exception(exc)
  File "/opt/conda/lib/python3.6/site-packages/rest_framework/views.py", line 486, in dispatch
    response = handler(request, *args, **kwargs)
  File "/opt/conda/lib/python3.6/site-packages/rest_framework/decorators.py", line 53, in handler
    return func(*args, **kwargs)
  File "./tilesets/views.py", line 597, in tileset_info
    tut.get_datapath(tileset_object.datafile.url)
  File "/opt/conda/lib/python3.6/site-packages/hgtiles/cooler.py", line 511, in tileset_info
    (f, info) = make_mats(filepath)
  File "/opt/conda/lib/python3.6/site-packages/hgtiles/cooler.py", line 481, in make_mats
    info = get_info(filepath)
  File "/opt/conda/lib/python3.6/site-packages/hgtiles/cooler.py", line 169, in get_info
    logger.info('no zoom found')
NameError: name 'logger' is not defined

What really confuses me is that the folder "/opt/conda" doesn't even exist on my machine (conda stuff is installed elsewhere).

Using plugins with higlass-manage

Hi there,

I was wondering if it's possible to use some of the plugins, in particular, higlass-arcs with this tool? Could you advise on a way to import this package? I'm assuming this done through the shell functionality, but any help would be greatly appreciated.

Thanks,

Neil

ingest mcool file error

cmd:

higlass-manage ingest /tmp/higlass-docker/glucose_\ hi_30.mcool --project-name xxx

errmsg

state True
Inferred filetype: cooler
Inferred datatype: matrix
state True
state True
name_text:
hg_name: default
command: python higlass-server/manage.py ingest_tileset --filename /tmp/glucose_hi_30.mcool --filetype cooler --datatype matrix  --project-name "islet"  --uid SPC9uwo6RC6pkHmZKLs0qA
Traceback (most recent call last):
  File "/usr/local/bin/higlass-manage", line 11, in <module>
    load_entry_point('higlass-manage==0.7.3', 'console_scripts', 'higlass-manage')()
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/higlass_manage/ingest.py", line 49, in ingest
    project_name
  File "/usr/local/lib/python3.6/site-packages/higlass_manage/ingest.py", line 80, in _ingest
    return import_file(hg_name, to_import, filetype, datatype, assembly, name, uid, no_upload, project_name)
  File "/usr/local/lib/python3.6/site-packages/higlass_manage/common.py", line 189, in import_file
    (exit_code, output) = container.exec_run(command)
ValueError: not enough values to unpack (expected 2, got 0)

higlass-manage start logs "Non 200 status code returned (502)"

I get the following log when I start higlass (higlass-manage start). I am using v0.8.7 for your information. I don't remember seeing this log in v0.8.6.

Pulling latest image... 
done
Data directory: /Users/<user>/hg-data
Temp directory: ()
Starting... default 8989
Docker started: higlass-manage-container-default
sending request 1
Waiting to start (tilesets)...
sending request 2
Waiting to start (tilesets)...
sending request 3
Waiting to start (tilesets)...
sending request 4
Non 200 status code returned (502), waiting...
sending request 5
Non 200 status code returned (502), waiting...
sending request 6
Non 200 status code returned (502), waiting...
sending request 7
Non 200 status code returned (502), waiting...
sending request 8
Non 200 status code returned (502), waiting...
sending request 9
Non 200 status code returned (502), waiting...
sending request 10
Non 200 status code returned (502), waiting...
sending request 11
Non 200 status code returned (502), waiting...
sending request 12
Non 200 status code returned (502), waiting...
sending request 13
Non 200 status code returned (502), waiting...
sending request 14
Non 200 status code returned (502), waiting...
sending request 15
Non 200 status code returned (502), waiting...
sending request 16
Non 200 status code returned (502), waiting...
sending request 17
public_data: True
Replaced js file
Started

I do not see any of the public data when I load up Higlass on a browser.

higlass-manage view /tmp/sample.mcool returns tcgetpgrp failed: Not a tty (error)

Hi. I'm trying a test dataset from higlass through higlass-manage. Installation seems to work fine. I installed Docker for Windows and integrated it with WSL2 Ubuntu20.04 distro. Invoking higlass-manage start doesn't give any error, but invoking higlass-manage view /tmp/sample.mcool raises

tcgetpgrp failed: Not a tty

returning only a blank higlass browser window.

Any help will be much appreciated. Thanks!

Installation Error

Type Error

higlass-manage view hic-resolutions.cool
state True
temp_dir: /tmp/higlass-docker
Inferred filetype: cooler
Inferred datatype: matrix
Traceback (most recent call last):
  File "/anaconda3/envs/test/bin/higlass-manage", line 10, in <module>
    sys.exit(cli())
  File "/anaconda3/envs/test/lib/python3.6/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/anaconda3/envs/test/lib/python3.6/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/anaconda3/envs/test/lib/python3.6/site-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/anaconda3/envs/test/lib/python3.6/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/anaconda3/envs/test/lib/python3.6/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/anaconda3/envs/test/lib/python3.6/site-packages/higlass_manage/view.py", line 127, in view
    view = conf.add_view()
TypeError: add_view() missing 1 required positional argument: 'view'

Redis config missing

The docs state that redis should work out of the box:

higlass-manage ingest --project-name Dovetail-data --name WTSI-OESO_103 raw-data/WTSI-OESO_103.mcool

I've had to run the following pulling the config from the docker repo:

pip3 install higlass-manage
mkdir /home/ubuntu/.local/lib/python3.6/site-packages/higlass_manage/redis
curl -L https://raw.githubusercontent.com/higlass/higlass-docker/master/redis-context/redis.conf > /home/ubuntu/.local/lib/python3.6/site-packages/higlass_manage/redis/redis.conf

Error, No such image: gehlenborglab/higlass:latest

I am trying to start higlass-manage I use says that it is Pulling latest image..., however ends up erroring out with the message docker.errors.ImageNotFound: 404 Client Error: Not Found ("no such image: gehlenborglab/higlass:latest: No such image: gehlenborglab/higlass:latest"). I've tried v0.2.1, 0.2.0 and 0.1.13, and all have the same message.

`higlass-manage start` should pull down docker image if needed

Chuck [11:06 AM]

Should higlass-manage start pull down the docker image if it’s not already installed? (One of our summer interns will continue into the fall, and will be looking at HiGlass, and I think he got stuck at this point in the tutorial.)

pete [11:08 AM]

Yes

The error I got:

$ higlass-manage start
Pulling latest image... 
Traceback (most recent call last):
  File "/Users/chuck/anaconda3/envs/higlass-tutorial/lib/python3.7/site-packages/docker/api/client.py", line 261, in _raise_for_status
    response.raise_for_status()
  File "/Users/chuck/anaconda3/envs/higlass-tutorial/lib/python3.7/site-packages/requests/models.py", line 940, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: http+docker://localhost/v1.35/images/higlass/higlass-docker:latest/json

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/chuck/anaconda3/envs/higlass-tutorial/bin/higlass-manage", line 10, in <module>
    sys.exit(cli())
  File "/Users/chuck/anaconda3/envs/higlass-tutorial/lib/python3.7/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/Users/chuck/anaconda3/envs/higlass-tutorial/lib/python3.7/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/Users/chuck/anaconda3/envs/higlass-tutorial/lib/python3.7/site-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/chuck/anaconda3/envs/higlass-tutorial/lib/python3.7/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/chuck/anaconda3/envs/higlass-tutorial/lib/python3.7/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/Users/chuck/anaconda3/envs/higlass-tutorial/lib/python3.7/site-packages/higlass_manage/start.py", line 105, in start
    redis_port)
  File "/Users/chuck/anaconda3/envs/higlass-tutorial/lib/python3.7/site-packages/higlass_manage/start.py", line 224, in _start
    hg_image = client.images.pull(hg_repository, version)
  File "/Users/chuck/anaconda3/envs/higlass-tutorial/lib/python3.7/site-packages/docker/models/images.py", line 453, in pull
    repository, tag, '@' if tag.startswith('sha256:') else ':'
  File "/Users/chuck/anaconda3/envs/higlass-tutorial/lib/python3.7/site-packages/docker/models/images.py", line 316, in get
    return self.prepare_model(self.client.api.inspect_image(name))
  File "/Users/chuck/anaconda3/envs/higlass-tutorial/lib/python3.7/site-packages/docker/utils/decorators.py", line 19, in wrapped
    return f(self, resource_id, *args, **kwargs)
  File "/Users/chuck/anaconda3/envs/higlass-tutorial/lib/python3.7/site-packages/docker/api/image.py", line 245, in inspect_image
    self._get(self._url("/images/{0}/json", image)), True
  File "/Users/chuck/anaconda3/envs/higlass-tutorial/lib/python3.7/site-packages/docker/api/client.py", line 267, in _result
    self._raise_for_status(response)
  File "/Users/chuck/anaconda3/envs/higlass-tutorial/lib/python3.7/site-packages/docker/api/client.py", line 263, in _raise_for_status
    raise create_api_error_from_http_exception(e)
  File "/Users/chuck/anaconda3/envs/higlass-tutorial/lib/python3.7/site-packages/docker/errors.py", line 31, in create_api_error_from_http_exception
    raise cls(e, response=response, explanation=explanation)
docker.errors.ImageNotFound: 404 Client Error: Not Found ("no such image: higlass/higlass-docker:latest: No such image: higlass/higlass-docker:latest")

problem higlass-ingest

Hi,
I have a problem setting up a new instance of Higlass. I installed higlass-manage in the following way:

$ conda create -p condaEnv/py3.6 python=3.6
$ source activate condaEnv/python.3.6
$ pip install higlass-manage

I can start Higlass:
$ higlass-manage start -t ~/HiGlass/tmp -d ~/HiGlass/data -m ~/HiGlass/media -p 8181 --no-public-data -n test
but then i do not manage to ingest any data:

$ higlass-manage ingest --hg-name test --filetype cooler --datatype matrix --assembly mm10  ~/HiGlass/data/SRP063320_NPC_WT_HiC_SRX1182728_G1.pairix.10kb.mcool
state True
name_text: 
hg_name: test
Traceback (most recent call last):
  File "/home/scollomb/programs/condaEnv/python.3.6/bin/higlass-manage", line 11, in <module>
    load_entry_point('higlass-manage==0.5.0', 'console_scripts', 'higlass-manage')()
  File "/home/scollomb/programs/condaEnv/python.3.6/lib/python3.6/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/home/scollomb/programs/condaEnv/python.3.6/lib/python3.6/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/home/scollomb/programs/condaEnv/python.3.6/lib/python3.6/site-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/scollomb/programs/condaEnv/python.3.6/lib/python3.6/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/scollomb/programs/condaEnv/python.3.6/lib/python3.6/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/home/scollomb/programs/condaEnv/python.3.6/lib/python3.6/site-packages/higlass_manage-0.5.0-py3.6.egg/higlass_manage.py", line 835, in ingest
  File "/home/scollomb/programs/condaEnv/python.3.6/lib/python3.6/site-packages/higlass_manage-0.5.0-py3.6.egg/higlass_manage.py", line 859, in _ingest
  File "/home/scollomb/programs/condaEnv/python.3.6/lib/python3.6/site-packages/higlass_manage-0.5.0-py3.6.egg/higlass_manage.py", line 170, in import_file
AttributeError: 'str' object has no attribute 'decode'

It works with the higlass docker so I guess I did something wrong somewhere...
Many thanks,
Samuel

Avoid creating a dupicate copy of data in media directory

Thank you for this amazing tool!

I have searched the docs, and perhaps I am missing this, but it seems that a dataset must be copied to the media directory for it to be ingested into higlass with the --no-upload flag. Alternatively, if the flag is not provided, the data will be uploaded to the internal higlass databae and copied to the media folder? I tried to symlink my datasets to the media folder, but it looks like the links are not preserved within the higlass instance. I also tried creating sshfs mounts to datasets on a different server in the media folder, but that didn't work either. Is there a way to avoid copyng the data to visualize it in higlass? I have about 1000 bigwig tracks to load, so would like to avoid creating a redundant copy of the files if possible. Thank you!

avoid listing public datasets

I am trying to set up a server but do not want to include all of the cools (etc) from the higlass public server. To start the local server I ran higlass-manage start --no-public-data but the (long) list of public data are still listed.

cat /usr/local/bin/higlass-manage tells me I have 0.2.2 installed.

higlass-manage start > SyntaxError: invalid syntax

Hi,
I installed Docker version 19.03.14, build 5eb3275d40 and used pip version 20.3.1 from python version 3.5 to install higlass-manage.
But nothing is working:

higlass-manage start

Traceback (most recent call last):
  File "/home/nmary/.local/bin/higlass-manage", line 5, in <module>
    from higlass_manage.cli import cli
  File "/home/nmary/.local/lib/python3.5/site-packages/higlass_manage/cli.py", line 3, in <module>
    from higlass_manage.ingest import ingest
  File "/home/nmary/.local/lib/python3.5/site-packages/higlass_manage/ingest.py", line 2, in <module>
    import clodius.cli.aggregate as cca
  File "/home/nmary/.local/lib/python3.5/site-packages/clodius/cli/__init__.py", line 18, in <module>
    from . import (  # noqa: F401
  File "/home/nmary/.local/lib/python3.5/site-packages/clodius/cli/aggregate.py", line 204
    print(f"BEDPEDB Version {BED2DDB_VERSION}")
                                             ^
SyntaxError: invalid syntax

higlass-manage view hic-resolutions.cool

Traceback (most recent call last):
  File "/home/nmary/.local/bin/higlass-manage", line 5, in <module>
    from higlass_manage.cli import cli
  File "/home/nmary/.local/lib/python3.5/site-packages/higlass_manage/cli.py", line 3, in <module>
    from higlass_manage.ingest import ingest
  File "/home/nmary/.local/lib/python3.5/site-packages/higlass_manage/ingest.py", line 2, in <module>
    import clodius.cli.aggregate as cca
  File "/home/nmary/.local/lib/python3.5/site-packages/clodius/cli/__init__.py", line 18, in <module>
    from . import (  # noqa: F401
  File "/home/nmary/.local/lib/python3.5/site-packages/clodius/cli/aggregate.py", line 204
    print(f"BEDPEDB Version {BED2DDB_VERSION}")
                                             ^
SyntaxError: invalid syntax

Any idea?
Tkanks

Some issues installing the new installation

Sorry for bundling issues.
Have a new machine and just started installing Higlass from scratch following this:
after installing doing:

higlass-manage start

Gave me an error relating to bed files ingestion.

print('An assembly or set of chromosome sizes is required when importing bed files. Please specify one or the other using the --assembly or --chromsizes-filename parameters', file=sys.stderr)

I think this is pointing to something wrong.
What I realized was causing the issue was that I had python 2 instead of 3.
Once I updated, higlass-manage start, created the hg-data folder and starts
This
higlass-manage browse
leads to

Error: No such command "browse".

Higlass still opens on http://localhost:8989 but not on http://localhost:8989/app as you describe.

Hope this helps.

Error retrieving tilesetInfo from: /api/v1

I am trying to view a data made by Juicer. I converted the hic file to cool using hic2cool which appeared to work.

################
### hic2cool ###
################
hic file header info:
Chromosomes:  ['ALL', '1', '2', '3', '4', '5', '6', '7', '8', '9', '10', '11', '12', '13', '14', '15', '16', '17', '18', '19', '20', '21', '22', 'X', 'Y', 'MT']
Resolutions:  [2500000, 1000000, 500000, 250000, 100000, 50000, 25000, 10000, 5000]
Genome:  hg19
Resolution 10000 took: 1111.847009897232 seconds.
hic2cool is finished! Output written to: test.cool

But when I run
higlass-manage view test.cool
or
higlass-manage start
higlass-manage ingest test.cool
higlass-manage browse

and try to load my file it says 'loading' for a long time and then eventually 'Error retrieving tilesetInfo from: /api/v1'

Loading other public files works fine.

higlass-manage problem

Hi,
I'm a new user for higlass and the docker. I followed the tutorial https://docs.higlass.io/tutorial.html. I met the following problems in running the sample data. Could you help?

First, $higlass-manage start showed below. However no window is started in Firefox.
Pulling latest image...
done
Data directory: /home/he/hg-data
Temp directory: ()
Starting... default 8989
Docker started: higlass-manage-container-default
sending request 1
Waiting to start (tilesets)...
sending request 2
Waiting to start (tilesets)...
sending request 3
Waiting to start (tilesets)...
sending request 4
Non 200 status code returned (502), waiting...
sending request 5
Non 200 status code returned (502), waiting...
sending request 6
Non 200 status code returned (502), waiting...
sending request 7
Non 200 status code returned (502), waiting...
sending request 8
Non 200 status code returned (502), waiting...
sending request 9
Non 200 status code returned (502), waiting...
sending request 10
public_data: True
Replaced js file
Started

Second, I tried to view the sample data by $higlass-manage view sample.mcool and it gave the error message as below.

state True
temp_dir: /tmp/higlass-docker
Inferred filetype: cooler
Inferred datatype: matrix
state True
Inferred datatype: matrix
state True
state True
Traceback (most recent call last):
File "/home/condaEnv/py3.6/bin/higlass-manage", line 8, in
sys.exit(cli())
File "/home/condaEnv/py3.6/lib/python3.6/site-packages/click/core.py", line 1128, in call
return self.main(*args, **kwargs)
File "/home/condaEnv/py3.6/lib/python3.6/site-packages/click/core.py", line 1053, in main
rv = self.invoke(ctx)
File "/home/condaEnv/py3.6/lib/python3.6/site-packages/click/core.py", line 1659, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/condaEnv/py3.6/lib/python3.6/site-packages/click/core.py", line 1395, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/condaEnv/py3.6/lib/python3.6/site-packages/click/core.py", line 754, in invoke
return __callback(*args, **kwargs)
File "/home/condaEnv/py3.6/lib/python3.6/site-packages/higlass_manage/view.py", line 150, in view
chromsizes_filename=chromsizes_filename,
File "/home/condaEnv/py3.6/lib/python3.6/site-packages/higlass_manage/ingest.py", line 131, in _ingest
project_name,
File "/home/condaEnv/py3.6/lib/python3.6/site-packages/higlass_manage/common.py", line 216, in import_file
os.link(filepath, to_import_path)
OSError: [Errno 18] Invalid cross-device link: '/home/HiGlass/data/sample.mcool' -> '/tmp/higlass-docker/sample.mcool'

optional IP subnet for docker network create

providing custom IP subnet might be important in some cases: higlass/higlass-docker#148

network = client.networks.create(network_name, driver="bridge")

from docker.networks.create help page:

   ipam (IPAMConfig): Optional custom IP scheme for the network.
ipam_pool = docker.types.IPAMPool(subnet='172.34.0.1/16')
ipam_config = docker.types.IPAMConfig(pool_configs=[ipam_pool])
client.networks.create(
            "network1",
            driver="bridge",
            ipam=ipam_config
        )

Symbol not found: ___darwin_check_fd_set_overflow

Hi,

I am encountering the below error when trying to run higlass-manage view using the test dataset.
I am running it through command line on a Mac (Mojave V10.14.6).

$ higlass-manage view sample.mcool
Traceback (most recent call last):
  File "/Users/timothystephens/opt/miniconda2/envs/highglass-manager/bin/higlass-manage", line 5, in <module>
    from higlass_manage.cli import cli
  File "/Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/site-packages/higlass_manage/cli.py", line 3, in <module>
    from higlass_manage.ingest import ingest
  File "/Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/site-packages/higlass_manage/ingest.py", line 2, in <module>
    import clodius.cli.aggregate as cca
  File "/Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/site-packages/clodius/cli/__init__.py", line 18, in <module>
    from . import (  # noqa: F401
  File "/Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/site-packages/clodius/cli/convert.py", line 11, in <module>
    import bbi
  File "/Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/site-packages/bbi/__init__.py", line 1, in <module>
    from .cbbi import (
ImportError: dlopen(/Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/site-packages/bbi/cbbi.cpython-37m-darwin.so, 2): Symbol not found: ___darwin_check_fd_set_overflow
  Referenced from: /Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/site-packages/bbi/.dylibs/libcrypto.3.dylib (which was built for Mac OS X 11.0)
  Expected in: /usr/lib/libSystem.B.dylib
 in /Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/site-packages/bbi/.dylibs/libcrypto.3.dylib

Here is my conda environment

$ conda list
# packages in environment at /Users/timothystephens/opt/miniconda2/envs/highglass-manager:
#
# Name                    Version                   Build  Channel
asciitree                 0.3.3                    pypi_0    pypi
blas                      1.0                         mkl  
bzip2                     1.0.8                h1de35cc_0  
c-ares                    1.18.1               hca72f7f_0  
ca-certificates           2021.10.26           hecd8cb5_2  
cached-property           1.5.2                      py_0  
certifi                   2021.10.8        py37hecd8cb5_2  
charset-normalizer        2.0.12                   pypi_0    pypi
click                     8.0.3                    pypi_0    pypi
clodius                   0.18.0                   pypi_0    pypi
cloudpickle               2.0.0                    pypi_0    pypi
cooler                    0.8.11                   pypi_0    pypi
curl                      7.80.0               hca72f7f_0  
cytoolz                   0.10.1                   pypi_0    pypi
dask                      2021.12.0                pypi_0    pypi
dill                      0.3.4                    pypi_0    pypi
docker                    5.0.3                    pypi_0    pypi
fsspec                    2022.1.0                 pypi_0    pypi
h5py                      3.6.0            py37h4a1dd59_0  
hdf5                      1.10.6               hdbbcd12_0  
higlass-manage            0.8.0                    pypi_0    pypi
idna                      3.3                      pypi_0    pypi
importlib-metadata        4.11.1                   pypi_0    pypi
intel-openmp              2021.4.0          hecd8cb5_3538  
krb5                      1.19.2               hcd88c3b_0  
libcurl                   7.80.0               h6dfd666_0  
libcxx                    12.0.0               h2f01273_0  
libdeflate                1.0                  h1de35cc_1    bioconda
libedit                   3.1.20210910         hca72f7f_0  
libev                     4.33                 h9ed2024_1  
libffi                    3.3                  hb1e8313_2  
libgfortran               3.0.1                h93005f0_2  
libnghttp2                1.46.0               ha29bfda_0  
libssh2                   1.9.0                ha12b0ac_1  
locket                    0.2.1                    pypi_0    pypi
mkl                       2021.4.0           hecd8cb5_637  
mkl-service               2.4.0            py37h9ed2024_0  
mkl_fft                   1.3.1            py37h4ab4a9b_0  
mkl_random                1.2.2            py37hb2f4e1b_0  
multiprocess              0.70.12.2                pypi_0    pypi
ncurses                   6.3                  hca72f7f_2  
negspy                    0.2.24                   pypi_0    pypi
nose                      1.3.7                    pypi_0    pypi
numpy                     1.21.5                   pypi_0    pypi
numpy-base                1.21.2           py37he0bd621_0  
openssl                   1.1.1m               hca72f7f_0  
packaging                 21.3                     pypi_0    pypi
pandas                    1.0.4                    pypi_0    pypi
partd                     1.2.0                    pypi_0    pypi
pip                       21.2.2           py37hecd8cb5_0  
pybbi                     0.3.2                    pypi_0    pypi
pydantic                  1.9.0                    pypi_0    pypi
pyfaidx                   0.6.4                    pypi_0    pypi
pypairix                  0.3.7                    pypi_0    pypi
pyparsing                 3.0.7                    pypi_0    pypi
pysam                     0.18.0                   pypi_0    pypi
python                    3.7.11               h88f2d9e_0  
python-dateutil           2.8.2                    pypi_0    pypi
pytz                      2021.3                   pypi_0    pypi
pyyaml                    6.0                      pypi_0    pypi
readline                  8.1.2                hca72f7f_1  
requests                  2.27.1                   pypi_0    pypi
scipy                     1.7.3                    pypi_0    pypi
setuptools                58.0.4           py37hecd8cb5_0  
simplejson                3.17.6                   pypi_0    pypi
six                       1.16.0             pyhd3eb1b0_1  
slugid                    2.0.0                    pypi_0    pypi
sortedcontainers          2.4.0                    pypi_0    pypi
sqlite                    3.37.2               h707629a_0  
tk                        8.6.11               h7bc2e8c_0  
toolz                     0.11.2                   pypi_0    pypi
tqdm                      4.62.3                   pypi_0    pypi
typing-extensions         4.1.1                    pypi_0    pypi
urllib3                   1.26.8                   pypi_0    pypi
websocket-client          1.2.3                    pypi_0    pypi
wheel                     0.37.1             pyhd3eb1b0_0  
xz                        5.2.5                h1de35cc_0  
zipp                      3.7.0                    pypi_0    pypi
zlib                      1.2.11               h4dc903c_4  

Thanks for your help.
Tim.

when I use local python to install higlass-manage ,I use the next command "pip install higlass-manage" error.

Collecting pysam (from clodius>=0.10.3->higlass-manage)
Using cached https://files.pythonhosted.org/packages/15/f6/ce0611aaa1865a616f7dc164fbf046eaf38f2b17c6d404403c56250beb93/pysam-0.15.2.tar.gz
Complete output from command python setup.py egg_info:
'.' 不是内部或外部命令,也不是可运行的程序
或批处理文件。
'.' 不是内部或外部命令,也不是可运行的程序
或批处理文件。
# pysam: no cython available - using pre-compiled C
# pysam: htslib mode is shared
# pysam: HTSLIB_CONFIGURE_OPTIONS=None
# pysam: htslib configure options: None
Traceback (most recent call last):
File "", line 1, in
File "C:\Users\msq\AppData\Local\Temp\pip-install-jc60ds7l\pysam\setup.py", line 223, in
htslib_make_options = run_make_print_config()
File "C:\Users\msq\AppData\Local\Temp\pip-install-jc60ds7l\pysam\setup.py", line 69, in run_make_print_config
stdout = subprocess.check_output(["make", "-s", "print-config"])
File "d:\py3\lib\subprocess.py", line 395, in check_output
**kwargs).stdout
File "d:\py3\lib\subprocess.py", line 472, in run
with Popen(*popenargs, **kwargs) as process:
File "d:\py3\lib\subprocess.py", line 775, in init
restore_signals, start_new_session)
File "d:\py3\lib\subprocess.py", line 1178, in _execute_child
startupinfo)
FileNotFoundError: [WinError 2] 系统找不到指定的文件。

----------------------------------------

Command "python setup.py egg_info" failed with error code 1 in C:\Users\msq\AppData\Local\Temp\pip-install-jc60ds7l\pysa

higlass-manage browser

I'm trying to follow HiGlass documentation to locally setup HiGlass on a workstation with a static IP address, then use my desktop to view the data. "higlass-manage ingest" moves the test file sample.mcool to the default directory /tmp/higlass-docker/sample.mcool on the workstation. From the desktop, I can load the HiGlass browser (Workstation_IP:8888/app), and view the list of public data sets, but not the local data sample.mcool.

Also following the higlass-manage usage fails as shown here:

wget https://s3.amazonaws.com/pkerp/public/hic-resolutions.cool
sudo higlass-manage view hic-resolutions.cool
state True
temp_dir: /tmp/higlass-docker
Inferred filetype: cooler
Inferred datatype: matrix
state True
Inferred datatype: matrix
state True
state True
name_text:
hg_name: default
command: python higlass-server/manage.py ingest_tileset --filename /tmp/hic-resolutions.cool --filetype cooler --datatype matrix --uid Bz-gWBMMR1SMa6uvx6h9HA
Traceback (most recent call last):
File "/usr/local/bin/higlass-manage", line 8, in
sys.exit(cli())
File "/usr/lib/python3/dist-packages/click/core.py", line 764, in call
return self.main(*args, **kwargs)
File "/usr/lib/python3/dist-packages/click/core.py", line 717, in main
rv = self.invoke(ctx)
File "/usr/lib/python3/dist-packages/click/core.py", line 1137, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/lib/python3/dist-packages/click/core.py", line 956, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/lib/python3/dist-packages/click/core.py", line 555, in invoke
return callback(*args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/higlass_manage/view.py", line 157, in view
from higlass.client import Track, View, ViewConf
ModuleNotFoundError: No module named 'higlass'

Symbol not found: ____chkstk_darwin

Here's the stack trace:

Traceback (most recent call last):
  File "/Users/pkerpedjiev/miniconda3/bin/higlass-manage", line 11, in <module>
    load_entry_point('higlass-manage', 'console_scripts', 'higlass-manage')()
  File "/Users/pkerpedjiev/miniconda3/lib/python3.7/site-packages/pkg_resources/__init__.py", line 489, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/Users/pkerpedjiev/miniconda3/lib/python3.7/site-packages/pkg_resources/__init__.py", line 2843, in load_entry_point
    return ep.load()
  File "/Users/pkerpedjiev/miniconda3/lib/python3.7/site-packages/pkg_resources/__init__.py", line 2434, in load
    return self.resolve()
  File "/Users/pkerpedjiev/miniconda3/lib/python3.7/site-packages/pkg_resources/__init__.py", line 2440, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/Users/pkerpedjiev/projects/higlass-manage/higlass_manage/cli.py", line 3, in <module>
    from higlass_manage.ingest import ingest
  File "/Users/pkerpedjiev/projects/higlass-manage/higlass_manage/ingest.py", line 2, in <module>
    import clodius.cli.aggregate as cca
  File "/Users/pkerpedjiev/miniconda3/lib/python3.7/site-packages/clodius-0.15.2-py3.7.egg/clodius/cli/__init__.py", line 18, in <module>
  File "/Users/pkerpedjiev/miniconda3/lib/python3.7/site-packages/clodius-0.15.2-py3.7.egg/clodius/cli/aggregate.py", line 8, in <module>
  File "/Users/pkerpedjiev/miniconda3/lib/python3.7/site-packages/clodius-0.15.2-py3.7.egg/clodius/multivec.py", line 4, in <module>
  File "/Users/pkerpedjiev/miniconda3/lib/python3.7/site-packages/h5py-3.0.0rc1-py3.7-macosx-10.7-x86_64.egg/h5py/__init__.py", line 25, in <module>
    from . import _errors
ImportError: dlopen(/Users/pkerpedjiev/miniconda3/lib/python3.7/site-packages/h5py-3.0.0rc1-py3.7-macosx-10.7-x86_64.egg/h5py/_errors.cpython-37m-darwin.so, 2): Symbol not found: ____chkstk_darwin
  Referenced from: /Users/pkerpedjiev/miniconda3/lib/python3.7/site-packages/h5py-3.0.0rc1-py3.7-macosx-10.7-x86_64.egg/h5py/.dylibs/libsz.2.dylib (which was built for Mac OS X 10.15)
  Expected in: /usr/lib/libSystem.B.dylib
 in /Users/pkerpedjiev/miniconda3/lib/python3.7/site-packages/h5py-3.0.0rc1-py3.7-macosx-10.7-x86_64.egg/h5py/.dylibs/libsz.2.dylib

README suggestions

Hi,
I have noticed that you are using tempfile.TemporaryDirectory function which apparently is only available on python3. Given this restriction, I would suggest specifying python 3 as a requirement.

Regards,
RR

higlass-manage start: socket.timeout

Trying to use HiGlass for the first time, following the tutorial. Docker desktop for mac.

$ higlass-manage start
Pulling latest image...
done
Data directory: /Users/alex/hg-data
Temp directory: ()
Starting... default 8989
Docker started: higlass-manage-container-default
sending request 1
Waiting to start (tilesets)...
sending request 2
Waiting to start (tilesets)...
esending request 3
Waiting to start (tilesets)...
sending request 4
Waiting to start (tilesets)...
sending request 5
Non 200 status code returned (502), waiting...
sending request 6
Non 200 status code returned (502), waiting...
sending request 7
Non 200 status code returned (502), waiting...
sending request 8
Non 200 status code returned (502), waiting...
sending request 9
Non 200 status code returned (502), waiting...
sending request 10
Non 200 status code returned (502), waiting...
sending request 11
Non 200 status code returned (502), waiting...
sending request 12
Non 200 status code returned (502), waiting...
sending request 13
Non 200 status code returned (502), waiting...
sending request 14
Non 200 status code returned (502), waiting...
sending request 15
Non 200 status code returned (502), waiting...
sending request 16
Non 200 status code returned (502), waiting...
sending request 17
Non 200 status code returned (502), waiting...
sending request 18
Non 200 status code returned (502), waiting...
sending request 19
Non 200 status code returned (502), waiting...
sending request 20
Non 200 status code returned (502), waiting...
sending request 21
Non 200 status code returned (502), waiting...
sending request 22
Non 200 status code returned (502), waiting...
sending request 23
Non 200 status code returned (502), waiting...
sending request 24
Non 200 status code returned (502), waiting...
sending request 25
Non 200 status code returned (502), waiting...
sending request 26
Non 200 status code returned (502), waiting...
sending request 27
Non 200 status code returned (502), waiting...
sending request 28
Non 200 status code returned (502), waiting...
sending request 29
Non 200 status code returned (502), waiting...
sending request 30
Non 200 status code returned (502), waiting...
sending request 31
Non 200 status code returned (502), waiting...
sending request 32
Non 200 status code returned (502), waiting...
sending request 33
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 384, in _make_request
    six.raise_from(e, None)
  File "<string>", line 2, in raise_from
  File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 380, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/local/Cellar/[email protected]/3.7.10_3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 1369, in getresponse
    response.begin()
  File "/usr/local/Cellar/[email protected]/3.7.10_3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 310, in begin
    version, status, reason = self._read_status()
  File "/usr/local/Cellar/[email protected]/3.7.10_3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 271, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/local/Cellar/[email protected]/3.7.10_3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/socket.py", line 589, in readinto
    return self._sock.recv_into(b)
socket.timeout: timed out

higlass-manage ingest error

I recently updated to MacOSX Catalina and higlass-manage ingest stopped working, and is giving me this error:

xinyangbing$ higlass-manage ingest --assembly dm6 XYZ.mcool
state True
Inferred filetype: cooler
Inferred datatype: matrix
state True
state True
Traceback (most recent call last):
File "/Users/xinyangbing/miniconda3/bin/higlass-manage", line 8, in
sys.exit(cli())
File "/Users/xinyangbing/miniconda3/lib/python3.8/site-packages/click/core.py", line 829, in call
return self.main(*args, **kwargs)
File "/Users/xinyangbing/miniconda3/lib/python3.8/site-packages/click/core.py", line 782, in main
rv = self.invoke(ctx)
File "/Users/xinyangbing/miniconda3/lib/python3.8/site-packages/click/core.py", line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/Users/xinyangbing/miniconda3/lib/python3.8/site-packages/click/core.py", line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/Users/xinyangbing/miniconda3/lib/python3.8/site-packages/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "/Users/xinyangbing/miniconda3/lib/python3.8/site-packages/higlass_manage/ingest.py", line 69, in ingest
_ingest(
File "/Users/xinyangbing/miniconda3/lib/python3.8/site-packages/higlass_manage/ingest.py", line 122, in _ingest
return import_file(
File "/Users/xinyangbing/miniconda3/lib/python3.8/site-packages/higlass_manage/common.py", line 204, in import_file
os.makedirs(temp_dir)
File "/Users/xinyangbing/miniconda3/lib/python3.8/os.py", line 213, in makedirs
makedirs(head, exist_ok=exist_ok)
File "/Users/xinyangbing/miniconda3/lib/python3.8/os.py", line 213, in makedirs
makedirs(head, exist_ok=exist_ok)
File "/Users/xinyangbing/miniconda3/lib/python3.8/os.py", line 213, in makedirs
makedirs(head, exist_ok=exist_ok)
File "/Users/xinyangbing/miniconda3/lib/python3.8/os.py", line 223, in makedirs
mkdir(name, mode)
OSError: [Errno 30] Read-only file system: '/host_mnt'

Please help thank you!

--OSError: [Errno 30] Read-only file system: '/host_mnt' -- Error in quickly viewing a dataset

Hi ,

I followed tutorial in https://docs.higlass.io/tutorial.html and when I run "higlass-manage view /tmp/sample.mcool '' command , I always face same error :
(OSError: [Errno 30] Read-only file system: '/host_mnt' )

and all result of after run "higlass-manage view /tmp/sample.mcool '' command :
Pulling latest image...
done
Data directory: /Users/kbq5786/hg-data
Temp directory: ()
Starting... default 8989
Docker started: higlass-manage-container-default
sending request 1
Waiting to start (tilesets)...
sending request 2
Waiting to start (tilesets)...
sending request 3
Waiting to start (tilesets)...
sending request 4
Non 200 status code returned (502), waiting...
sending request 5
Non 200 status code returned (502), waiting...
sending request 6
Non 200 status code returned (502), waiting...
sending request 7
Non 200 status code returned (502), waiting...
sending request 8
Non 200 status code returned (502), waiting...
sending request 9
Non 200 status code returned (502), waiting...
sending request 10
Non 200 status code returned (502), waiting...
sending request 11
Non 200 status code returned (502), waiting...
sending request 12
public_data: True
Replaced js file
Started
Inferred filetype: cooler
Inferred datatype: matrix
state True
Inferred datatype: matrix
state True
state True
Traceback (most recent call last):
File "/Users/kbq5786/opt/anaconda3/bin/higlass-manage", line 8, in
sys.exit(cli())
File "/Users/kbq5786/opt/anaconda3/lib/python3.8/site-packages/click/core.py", line 829, in call
return self.main(*args, **kwargs)
File "/Users/kbq5786/opt/anaconda3/lib/python3.8/site-packages/click/core.py", line 782, in main
rv = self.invoke(ctx)
File "/Users/kbq5786/opt/anaconda3/lib/python3.8/site-packages/click/core.py", line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/Users/kbq5786/opt/anaconda3/lib/python3.8/site-packages/click/core.py", line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/Users/kbq5786/opt/anaconda3/lib/python3.8/site-packages/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "/Users/kbq5786/opt/anaconda3/lib/python3.8/site-packages/higlass_manage/view.py", line 144, in view
uuid = _ingest(
File "/Users/kbq5786/opt/anaconda3/lib/python3.8/site-packages/higlass_manage/ingest.py", line 122, in _ingest
return import_file(
File "/Users/kbq5786/opt/anaconda3/lib/python3.8/site-packages/higlass_manage/common.py", line 204, in import_file
os.makedirs(temp_dir)
File "/Users/kbq5786/opt/anaconda3/lib/python3.8/os.py", line 213, in makedirs
makedirs(head, exist_ok=exist_ok)
File "/Users/kbq5786/opt/anaconda3/lib/python3.8/os.py", line 213, in makedirs
makedirs(head, exist_ok=exist_ok)
File "/Users/kbq5786/opt/anaconda3/lib/python3.8/os.py", line 213, in makedirs
makedirs(head, exist_ok=exist_ok)
File "/Users/kbq5786/opt/anaconda3/lib/python3.8/os.py", line 223, in makedirs
mkdir(name, mode)
OSError: [Errno 30] Read-only file system: '/host_mnt'

What shoul I do? I check higlass-manage instances and firstly stop all instances then I try but again gave same error ?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.