Comments (7)
are you implementing Celery?
from osparc-simcore.
are you implementing Celery?
Not at all.
I have evaluated celery
and scheduler
. None were sufficient for this application.
I have to build a lightweight scheduler with the requirements mentioned above.
from osparc-simcore.
are you implementing Celery?
Not at all.
I have evaluated
celery
andscheduler
. None were sufficient for this application. I have to build a lightweight scheduler with the requirements mentioned above.
Very interesting! But it also looks like quite a mouthful! Would it not be possible to build this on top of Celery or some other tool supported by a large community?
from osparc-simcore.
I do agree with @bisgaard-itis on this. if possible using an already well tested tool and build on top is a better option I think than creating everything from scratch.
from osparc-simcore.
@bisgaard-itis @sanderegg
I would like to use a pre baked solution.
My main difficulty here is that, while the service is running I need to do the following:
- schedule a new async handler
- modify the schedule period of an existing async handler
- remove and an async handler
With celery
is a bit complex to do so. It's a bit more statically configured.
I will have to implement the trie logic for scheduling and the only advantage I would get is run an event after "x" seconds done by celery. I can already have that with RabbitMQ
with much, much less complexity.
And as a final argument against celery
, it does not play nice with async
.
Any other suggestions that I can investigate are welcomed.
from osparc-simcore.
@bisgaard-itis @sanderegg I would like to use a pre baked solution.
My main difficulty here is that, while the service is running I need to do the following:
- schedule a new async handler
- modify the schedule period of an existing async handler
- remove and an async handler
With
celery
is a bit complex to do so. It's a bit more statically configured. I will have to implement the trie logic for scheduling and the only advantage I would get is run an event after "x" seconds done by celery. I can already have that withRabbitMQ
with much, much less complexity. And as a final argument againstcelery
, it does not play nice withasync
.Any other suggestions that I can investigate are welcomed.
Maybe check https://pypi.org/project/aio-celery/ whether it would be useful?
from osparc-simcore.
@bisgaard-itis @sanderegg I would like to use a pre baked solution.
My main difficulty here is that, while the service is running I need to do the following:
- schedule a new async handler
- modify the schedule period of an existing async handler
- remove and an async handler
With
celery
is a bit complex to do so. It's a bit more statically configured. I will have to implement the trie logic for scheduling and the only advantage I would get is run an event after "x" seconds done by celery. I can already have that withRabbitMQ
with much, much less complexity. And as a final argument againstcelery
, it does not play nice withasync
.
Any other suggestions that I can investigate are welcomed.Maybe check https://pypi.org/project/aio-celery/ whether it would be useful?
Being able to use async is just a minor detail.
I already said that I have discarded celery
. It is not suited for this application. I'm searching for something that can fit to the requirements and celery
is not it.
from osparc-simcore.
Related Issues (20)
- Autoscaling: use labels instead of draining nodes HOT 1
- 🚀 Release v1.66.0 HOT 3
- Switch to `debugpy` everywhere
- 🚀 Pre-release master -> [sim4life] staging_Schoggilebe1 HOT 2
- Introduce httpx exception handler in api server
- Flaky test_studies_dispatcher_studies_access
- 🚀 Pre-release master -> staging_Schoggilebe3 HOT 1
- Add all status codes to openapi specs for api-server
- Introduce multiple replicas of the api-server
- Add proper log messages when raising `HTTPException` in api-server
- UI: request account country field should have enum HOT 1
- Duration shows wrongly in Usage overview
- Fix exception handler in api server
- `storage` creates only http links
- Create new url that points to NIH staging sim4life product
- Fix progress report when starting sim4life on a fresh machine with pre-pulling
- 🚀 [sim4life] Release v1.67.0 HOT 2
- API keys tied to computational service life cycle
- Project cannot be saved (project update API call replies with 401 Unauthorised) HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from osparc-simcore.