yiicod/yii2-jobqueue

Multi processes job queue

Yii Job Queue based on Illuminate Queue

Provides Illuminate queues implementation for Yii 2 using mongodb as main storage.

Base config:

    'bootstrap' => [
        'jobqueue'
    ],
    'components' => [
        'jobqueue' => [
            'class' => \yiicod\jobqueue\JobQueue::class
        ],
        'mongodb' => [
            'class' => '\yii\mongodb\Connection',
            'dsn' => 'mongodb://@localhost:27017/mydatabase',
        ],        
    ]

Console config (simple fork)

    'bootstrap' => [
        'jobqueue'
    ],
    'controllerMap' => [
        'job-queue' => [
            'class' => \yiicod\jobqueue\commands\JobQueueCommand::class,
        ]
    ],
    'components' => [
        'jobqueue' => [
            'class' => \yiicod\jobqueue\JobQueue::class
        ],
        'mongodb' => [
            'class' => '\yii\mongodb\Connection',
            'dsn' => 'mongodb://@localhost:27017/mydatabase',
        ],        
    ]    

Start worker:

Run worker daemon with console command:

$ php yii job-queue/start

Stop worker daemon:

$ php yii job-queue/stop

Console config + PM2(http://pm2.keymetrics.io/). This variant more preferable for console configuration.

    'bootstrap' => [
        'jobqueue'
    ],
    'controllerMap' => [
        'job-queue' => [
            'class' => \yiicod\jobqueue\commands\WorkerCommand::class,
        ]
    ],
    'components' => [
        'jobqueue' => [
            'class' => \yiicod\jobqueue\JobQueue::class
        ],
        'mongodb' => [
            'class' => '\yii\mongodb\Connection',
            'dsn' => 'mongodb://@localhost:27017/mydatabase',
        ],        
    ]        

pm2 config:

    {
      "apps": [
        {
          "name": "job-queue",
          "script": "yii",
          "args": [
            "job-queue/work"
          ],
          "exec_interpreter": "php",
          "exec_mode": "fork_mode",
          "max_memory_restart": "1G",
          "watch": false,
          "merge_logs": true,
          "out_file": "runtime/logs/job_queue.log",
          "error_file": "runtime/logs/job_queue.log"
        }
      ]
    }

Run PM2 daemons

pm2 start daemons-app.json

Note: Don't forget configure mongodb

Adding jobs to queue:

Create your own handler which implements yiicod\jobqueue\base\JobQueueInterface OR extends yiicod\jobqueue\handlers\JobQueue and run parent::fire($job, $data) to restart db connection before job process

JobQueue::push(<--YOUR JOB QUEUE CLASS NAME->>, $data, $queue, $connection);
// Optional: $queue, $connection

Note: $data - additional data to your handler

Queue configuration:

Add jobqueue component with connections parameters, specially with "MongoThreadQueue" driver and connection name ("default" in example)

'jobqueue' => [
    'class' => \yiicod\jobqueue\JobQueue::class,
    'connections' => [
        'default' => [
            'driver' => 'mongo-thread',
            'table' => 'yii-jobs',
            'queue' => 'default',
            'connection' => 'mongodb', // Default mongodb connection 
            'expire' => 60,
            'limit' => 1, // How many parallel process should run at the same time            
        ],
    ]
]

Worker will take jobs from mongo database and run them by thread with defined driver using "mongo-thread" command in the background

Available events:

---

In Worker::class:

EVENT_RAISE_BEFORE_JOB = 'raiseBeforeJobEvent';
EVENT_RAISE_AFTER_JOB = 'raiseAfterJobEvent';
EVENT_RAISE_EXCEPTION_OCCURED_JOB = 'raiseExceptionOccurredJobEvent';
EVENT_RAISE_FAILED_JOB = 'raiseFailedJobEvent';
EVENT_STOP = 'stop';

yii2-jobqueue's Issues

PHP Fatal error: Trait 'yiicod\jobqueue\commands\WorkerTrait' not found

Hi, I'm getting this error when I try to start the queue daemon, any ideas? thanks!