Comments (15)
Flower gets celery events in real time and doesn't store anything on a disk. If you restart flower server you get an initial state.
from flower.
yep, I got that right then :)
do you see this feature coming in the future of flower ? (i might give some help developing the feature in that case).
from flower.
Celery can produce a huge amount of events and storing them all on the disk may be undesirable. What is you use case?
from flower.
I think it would be cool to keep data about throughput per instance (the monitor graph) over time.
from flower.
I would like this as well. I think it would be enough to store let's say the last 30 days worth of data. If flower has been running for a while you get nice graphs and tons of data about how your workers are performing, etc but if you need to make a configuration change and restart the server, all of that is lost. What if your supervisord process that keeps flower alive decides to go to lunch? Just a thought...
from flower.
For redis-backed brokers and storage backends, it may be a good idea to persist the events for a configurable amount of time specified by the user.
Setting an expiration on the already existing keys seems a logical option. Redis can then be configured http://redis.io/topics/config to cap the memory usage of those keys by either evicting old keys or denying the writing of new keys.
from flower.
I've implemented --persistent option which allows to save/load the current state. Be default flower.db file is used which can be changed with --db option.
--max_task option allows to limit the number of tasks in memory.
from flower.
@mher When you navigate to /monitor is the expected behaviour that it will show you past executed tasks or is it only real updated in real time?
Thanks.
from flower.
I have the same question: does persistent mode have any effect on the monitor's charts?
from flower.
/monitor only shows real-time data.
from flower.
@mher Is there a way where I can edit the contents of flower.db
such that if the file gets too big then I can edit it to save N newest tasks and delete the rest? I'm not sure whether --max_tasks
will accomplish this
from flower.
@mher I am new to flower and I am not able to figure out that from where does flower read celery tasks/events to show on dashboard? If I am not setting persistence flag as True that means there is no database file being created, then where does it read the data from?
from flower.
Flower reads data from events. When celery got message from broker, it fires the events: received
, started
, processed
and flower listenings for these events and update it's internal state accordingly and represent it using API: dashboard, tasks, monitors etc... Therefore if celery started and flower not, then when you start the latter, you do not get previous data as flower process events in real time.
To keep data about workers and counters, I tried to persist counter data in this PR: #926
Dashboard now became restored. Will check monitor as well later on
from flower.
Any update on PR #926? would be great to see this data persisted
from flower.
Monitor tab is deprecated, use Prometheus instead.
from flower.
Related Issues (20)
- Flower loads indefinitely HOT 1
- Add a configuration to mask some of the information that is shown in the config tab of the worker view
- [FEATURE] Option to retry a failed task HOT 1
- Unable to connect to rabbitmq broker that runs with self-signed certificates
- No data available in table HOT 1
- Migration to TailwinCSS HOT 1
- Add support for poetry
- Error 500 on Broker page HOT 4
- Search arg in List tasks API - more than one state
- Task is ignored
- Questions about Copilot + Open Source Software Hierarchy
- celery flower db path from s3 file or external source HOT 1
- number_of_prefetched_tasks not decrementing HOT 4
- Command line starts normally,but unable to access flower ui HOT 1
- amqps:// scheme in Broker URL breaks Broker Tab HOT 5
- Output numbers human-friendly
- InvalidRequirements error
- Broker scheduled tasks metrics
- Getting error `No module named '"flower' when running in Kubernetes HOT 1
- There should be a way to disable /metrics
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from flower.