Comments (12)
Great points! Thanks for taking the time to reply!
- How about nodes that don't run any jobs? (e.g. then connecting to a server via iex to interact with the system (including enqueueing new jobs as well) but not to run any jobs by itself)? Very typical scenario for me, there are hacks to make it work with current version but these don't work very well
I've done this using the distillery remote_console
command and it works really well. Another way that can be done is setting the queues configuration to be empty, but I would consider this a hacky way. Interested in other ideas
But yeah I think I need to think more about the client side of the Verk functions like enqueue
so that no hacks are needed. Thanks for bringing this up
- "all the jobs being run in the cluster" as mentioned in references
I'm not sure if I understood this second point, but I'm really excited the the view of all running jobs will be accessible through Verk Web (or any tools that use Verk.Queue
)
- Any thoughts on self-introspection for a job? i.e. within a job get it's current ID, queue, retry number. atm no such information is available but could be useful if custom logic is necessary
This is already achievable by using Verk.Worker.current_job()
from the process that is running the job. More here: https://github.com/edgurgel/verk/blob/master/lib/verk/worker.ex#L14
We should probably add to the README.md as it's clearly not easy to find this information?
from verk.
Another way that can be done is setting the queues configuration to be empty
Yes, i'm using it this way. Somehow doesn't work every time, so yeah, "hacky way"
-
Just agreeing on points mentioned in initial post
-
Good to know, thanks! Yep, didn't find the info, adding generally useful commands into README would be very useful
👍 maybe adding stuff likeVerk.Queue.count!("queue_foo")
as well
from verk.
Hi @edgurgel ,
Hope you are doing well, it would be great to have an idea of what happened with the plan (and the branch for that matter) for the 2.0 version ?
Just trying to gauge if this is something that will still happen at some point or not.
Thanks in advance.
from verk.
Hi @edgurgel ,
Thanks for getting back to me. Would it be possible to envision a v2 without verk Web but supporting telemetry so people can plug on to it to monitor verk ? Maybe that would be a pretty decent intermediate state without asking you for too much ? It seemed to me that the v2 branch you had was pretty close to being stable or was there much more work to do ?
In any event, thanks for all your contributions and I wish you a merry Christmas.
from verk.
Overall - great work!
-
How about nodes that don't run any jobs? (e.g. then connecting to a server via iex to interact with the system (including enqueueing new jobs as well) but not to run any jobs by itself)? Very typical scenario for me, there are hacks to make it work with current version but these don't work very well
-
"all the jobs being run in the cluster" as mentioned in references
-
Any thoughts on self-introspection for a job? i.e. within a job get it's current ID, queue, retry number. atm no such information is available but could be useful if custom logic is necessary
from verk.
Is a way to unique a specific worker or queue possible in the new update?
from verk.
Verk 1.0 and Verk 2.0 can support unique workers. We just never had traction enough to build it I think.
from verk.
Not sure where to ask about it, so I'm gonna ask here.
Would it be possible to change it to warn/error level?
My reasoning is that debug is used to debug the code, but this is unknown error that we are silencing and it's not visible in a log file, neither can Sentry report it. I don't want to turn on debug level, because imo debug isn't meant for this. Wdyt?
verk/lib/verk/workers_manager.ex
Line 165 in acda6d8
from verk.
@DiodonHystrix, it's a bit unrelated to Verk 2.0 but I will answer here!
Why would like to know this type of information? What kind of information do you expect from it?
This is internal to the workers manager. If you want to track retries on your workers just go to the existing tracking events that you can receive: https://github.com/edgurgel/verk/tree/v1.6.3#error-tracking
from verk.
Like I said, I'd like to see the errors in my logs (without a need to switch to the debug level).
Currently, I'm using Sentry to report any incidents and it doesn't get notified about the error, thus I have no clue that there was an error in my queue. It could be fixed by setting it to a debug level, but I don't think that's a correct solution.
I didn't notice error-tracking section, it could probably solve my problem, but I think it's gonna be an over-engineered solution.
from verk.
Hi @edgurgel,
Hope you are doing well, is there any update you could share regarding your plan for the 2.0 ?
Thanks !
from verk.
Hi @tlvenn,
At this stage it's indefinitely on hold because I don't have the time to invest on getting Verk 2.0 out with all the changes required to Verk Web.
from verk.
Related Issues (20)
- Strange behaviour with integer argument, potential bug HOT 17
- Add mix task to reset queues HOT 9
- Add pending jobs to queue stats HOT 2
- Unable to overwrite `max_retry_count` via config.exs HOT 2
- Event for dead job HOT 2
- Question. Can Verk check more frequently for new jobs? HOT 2
- Undocumented `max_dead_jobs` HOT 1
- function Verk.Supervisor.start_link/1 is undefined or private HOT 3
- Im surely missing something. HOT 4
- Coupling of producers and consumers? HOT 1
- Upgrade GenStage to current version HOT 5
- Version Conflict with Confex 3.4.0 HOT 2
- Loosen dependency on redis HOT 2
- Skip queuing jobs during tests. HOT 1
- Batch processing? HOT 2
- Defining contextual data for worker processes HOT 4
- [question] What are the risks of the "experimental" generate_node_id feature? HOT 4
- How do I setup Verk as a producer only? HOT 1
- Connect to Redix via ipv6? HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from verk.