radlab/sparrow

Sparrow scheduling platform (U.C. Berkeley).

sparrow's Issues

Honor Placement Constraints in Scheduling

speaks for itself.

LICENSE

Please add a LICENSE to this project. Perhaps the same as other AMPLab projects?

https://github.com/mesos/spark/blob/master/LICENSE
https://github.com/amplab/snap/blob/master/LICENSE

Update simulation parameter framework

Change the simulation to use the same parameter framework as the deploy scripts, since that one is much more robust.

I doubt where is the package or jar of "edu.berkeley.sparrow.thrift._" ?

att. Thx~

Have Sparrow Pass Messages Between Backends and Frontends

Certain applications, such as Spark, rely on the scheduling framework to pass messages between worker tasks and the frontend. Sparrow should allow a generic message passing functionality allowing application backends to send messages to fronends and vice versa.

Create Mesos-Compliant Backend in Java

To inter-operate with existing Mesos-based applications (e.g. Spark), we want to create a back-end that looks like a Mesos slave.

The interface between these applications and Mesos on the backend side are the following classes:
https://github.com/apache/mesos/blob/trunk/src/java/src/org/apache/mesos/ExecutorDriver.java
https://github.com/apache/mesos/blob/trunk/src/java/src/org/apache/mesos/Executor.java

We want to create a subclass of ExecutorDriver that also acts as a Sparrow backend. This will both respond to function calls from the application and delegate function calls to the application (it will require a pointer to an Executor when instantiated).

Readme has a broken link to the research paper

The link in "To read more about Sparrow, check out our " seems to be broken.

http://delivery.acm.org/10.1145/2530000/2522716/p69-ousterhout.pdf

Clean up logging

Right now we have a bunch of audit logging and debug logging that all gets spit out by default. We should make logging configurable via existing sparrow configuration and also mostly off by default.

Break simulation graphing functionality into different files

Moving the various functions to run extensive simulations into individual files would make much more sense, and make the code easier to read.

Update simulation terminology

Update the terminology used in the simulation to be consistent with terminology we now use in publications (so Server --> Slave and FrontEnd --> Scheduler).

Make buildbot run simulation tests

Title says it all...