Giter Site home page Giter Site logo

eojobqueuebundle's Introduction

EoJobQueueBundle

Dependencies Status

Mongodb ODM implementation for JMSJobQueueBundle which allows to schedule Symfony2 console commands as jobs.

Prerequisites

This version of the bundle requires Symfony 2.1+

Installation

Step 1: Download EoJobQueueBundle using composer

Add EoJobQueueBundle in your composer.json:

{
    "require": {
        "eo/job-queue-bundle": "dev-master"
    }
}

Now tell composer to download the bundle by running the command:

$ php composer.phar update eo/job-queue-bundle

Composer will install the bundle to your project's vendor/eo directory.

Step 2: Enable the bundle

Enable the bundle in the kernel:

<?php
// app/AppKernel.php

public function registerBundles()
{
    $bundles = array(
        // ...
        new Eo\JobQueueBundle\EoJobQueueBundle(),
    );
}

Step 3: Change console base application class

Have your app/console use EoJobQueueBundle's Application:

// use Symfony\Bundle\FrameworkBundle\Console\Application;
use Eo\JobQueueBundle\Console\Application;

Step 4 (Optional): Configure bundle

Now that you have properly installed and enabled EoJobQueueBundle, the next step is to configure the bundle to work with the specific needs of your application.

To change the default job class used by the bundle, add the following configuration to your config.yml file

# app/config/config.yml
eo_job_queue:
    job_class: JuliusJobBundle:Job

Setting Up supervisord

For this bundle to work, you have to make sure that one (and only one) instance of the console command eo-job-queue:run is running at all times. You can easily achieve this by using supervisord.

A sample supervisord config might look like this:

[program:eo_job_queue_runner]
command=php %kernel.root_dir%/console eo-job-queue:run --env=prod --verbose
process_name=%(program_name)s
numprocs=1
directory=/tmp
autostart=true
autorestart=true
startsecs=5
startretries=10
user=www-data
redirect_stderr=false
stdout_logfile=%capistrano.shared_dir%/eo_job_queue_runner.out.log
stdout_capture_maxbytes=1MB
stderr_logfile=%capistrano.shared_dir%/eo_job_queue_runner.error.log
stderr_capture_maxbytes=1MB

For testing, or development, you can of course also run the command manually, but it will auto-exit after 15 minutes by default (you can change this with the --max-runtime=seconds option).

Usage

Creating Jobs

Creating jobs is super simple, you just need to persist an instance of Job:

<?php

$job = new Job('my-symfony2:command', array('some-args', 'or', '--options="foo"'));
$dm->persist($job);
$dm->flush($job);

Adding Dependencies Between Jobs

If you want to have a job run after another job finishes, you can also achieve this quite easily:

<?php

$job = new Job('a');
$dependentJob = new Job('b');
$dependentJob->addJobDependency($job);
$dm->persist($job);
$dm->persist($dependentJob);
$dm->flush();

Adding Related Documents to Jobs

If you want to link a job to another document, for example to find the job more easily, the job provides a special many-to-any association:

<?php

$job = new Job('a');
$job->addRelatedDocument($anyDocument);
$dm->persist($job);
$dm->flush();

$dm->getRepository('EoJobQueueBundle:Job')->findJobForRelatedDocument('a', $anyDocument);

Schedule a Jobs

If you want to schedule a job :

<?php

$job = new Job('a');
$date = new DateTime();
$date->add(new DateInterval('PT30M'));
$job->setExecuteAfter($date);
$dm->persist($job);
$dm->flush();

License

This bundle is under the Apache2 license. See the complete license in the bundle:

Resources/meta/LICENSE

Reporting an issue or a feature request

Issues and feature requests related to this bundle are tracked in the Github issue tracker https://github.com/eymengunay/EoJobQueueBundle/issues

eojobqueuebundle's People

Contributors

ecentinela avatar eymengunay avatar kryniol avatar nicolas-brousse avatar nicolasbadey avatar pierredup avatar schmittjoh avatar tobiassjosten avatar

Stargazers

 avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

eojobqueuebundle's Issues

Job Queue is running on different php version

We've PHP 5.5 version on system configuration and we are running eojobqueue run command using php7.0. When eoJobqueue run our command it execute on php5.5 instead of php7.0. Because of it run using exec php command it select system configured php version. We've added PR to change hardcoded php command to PHP_BINARY. #6

cleanUpStaleJobs mark as incomplete running jobs

Hi,

I'm experiencing something strange. It was working properly, but suddenly it is not.

I have supervisor configured for running 13 workers, each one runs the EoJobQueueBundle command eo-job-queue:run with different --queue argument (that is, each worker is listening one queue), some of them have -j argument with value 2 and some others with value 1.
(The machine where supervisor is executed has 16 Cores and 120Gb of RAM).

The problem is about eo-job-queue:run command from where the cleanUpStaleJobs function is called. This function marks running jobs to incomplete. The problem is that suddently all my jobs are marked as incomplete.

I had to comment the line $this->cleanUpStaleJobs() and then from time to time I see some stale jobs (which should be caused by some SIGTERM signal and it is ok)

Any idea what could be happening? Thank you

composer install breaks EoJobQueueBundle

Following the installation instructions,

After updating composer.json with

"require": {
"eo/job-queue-bundle": "dev-master"
}

the version installed is not the last one.

i.e., the Job class is implementing JobInterface and it was removed in the last PR (#1)

Application.php onTick() uses JobStatistic which was removed in recent merge

So whole thing stopped working entrely

Job 57d97ace279871e7718b4567: [Symfony\Component\Debug\Exception\ClassNotFoundException]
Job 57d97ace279871e7718b4567: Attempted to load class "JobStatistic" from namespace "Eo\JobQueueBundle\Document".
Job 57d97ace279871e7718b4567: Did you forget a "use" statement for another namespace?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.