ceteri/exelixi

Exelixi is a distributed framework based on Apache Mesos, mostly implemented in Python using gevent for high-performance concurrency. It is intended to run cluster computing jobs (partitioned batch jobs, which include some messaging) in pure Python. By default, it runs genetic algorithms at scale.

Wash Warm, Tumble Dry 👋

• 🔭☕🛠️ Currently working on ...

  • entity resolved knowledge graphs
  • natural language + entity resolution + graph technologies + machine learning, see https://huggingface.co/pacoid
  • devrel @ senzing.com
  • managing partner @ derwen.ai
  • advisor + board member @ argilla.io; advisor @ kungfu.ai; advisor @ dataspartan ; advisor @ kurve.ai
  • graph data science in general -- see: https://www.linkedin.com/groups/6725785/
  • enhancements and integrations for pytextrank, kglab, mkrefs, disparity_filter
  • API development with fastapi, pydantic, pydantic-aioredis
  • "Thinking Sparse and Dense" about how hardware evolution drives AI

• 💬

  • fluently: EN, Py, C++, JavaScript, Bash, LPG, RDF/OWL/SKOS/SHACL, NLP, ML, Neural Networks, Distrib Systems, Optimization
  • conversationally: FR, ES
  • attempted: FA, CY, JA, AR, RU, Abstract Algebra, Topology
  • etc: Scala, Clojure, R, Java, C, FORTRAN, LISP, Pascal, JCL, Spice, Verilog, various assembly
  • pronouns: he/him
  • 📫 reach me: https://derwen.ai/paco, @[email protected], Signal

• 🙌 collaborates on: integrating Python libraries that provide different kinds of language or graph capabilities

  • see also: https://stackoverflow.com/users/1698443/paco

• 👍 animist, bioregionalist, /|\

• 🌳 fun fact: lives on an apple orchard in the Redwoods near the coast, making gelato, sorbet, granita, cider, mead, limoncello, and so on

• 🎬 favorite animation series: Llamas with Hats

• 🏙️ member: ACM, PSF, NumFOCUS, ESIP

• orcid.org/0000-0003-3167-1539 | 📚

• 🚴 activities: cycling, kayaking, hiking

exelixi's Issues

hat-trie added dependency on Cython

After you replaced datrie with hat_trie, cython became a requirement. One idea is to add the dependency, such as "sudo pip install cython" in "bin/local_install.sh". Another idea is to go back to datrie. Thank you.

ImportError: cannot import name shutdown

Hi,
I'm on a MacBook Pro running 10.10.3 with an Anaconda stack.
David-Laxers-MacBook-Pro:incubator-zeppelin davidlaxer$ conda -V
conda 3.13.0

gevent==1.0.1 does not seem to have a shutdown method.
Any ideas?

David-Laxers-MacBook-Pro: davidlaxer$ ipython
Python 2.7.10 |Anaconda 2.2.0 (x86_64)| (default, May 28 2015, 17:04:42)
Type "copyright", "credits" or "license" for more information.

IPython 3.1.0 -- An enhanced Interactive Python.
Anaconda is brought to you by Continuum Analytics.
Please check out: http://continuum.io/thanks and https://binstar.org
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object', use 'object??' for extra details.

In [1]: from gevent import shutdown

ImportError Traceback (most recent call last)
in ()
----> 1 from gevent import shutdown

ImportError: cannot import name shutdown

In [2]: from gevent import *

In [3]: quit
David-Laxers-MacBook-Pro:incubator-zeppelin davidlaxer$ pip freeze | grep gevent
gevent==1.0.1
gevent-websocket==0.9.3
David-Laxers-MacBook-Pro:incubator-zeppelin davidlaxer$

is HDFS a hard requirement to setup/run exelixi framework?

Is HDFS a hard requirement to setup/run exelixi?

Hadoop is not currently part of playa-mesos box image, so install.sh fails the hadoop fs commands.

playa-mesos team is thinking to add support for a single node hadoop instance configured with pseudo-distributed operation. Would that work?