stitchng/adonis-queue

An addon/plugin package to provide driver-based job queueing services in AdonisJS 4.0+

adonis-queue's Issues

the queue is not running on the redis

I have a simple job like below

'use strict'

const Job = use('Job')

class Test extends Job {
  
    constructor(){
      super(arguments)
      
      this.timeOut = 2000; // seconds: time out for queue
      this.retryCount = 0; // number of times to retry
      this.retryUntil = 200; // seconds: retry
      this.delayUntil = 0; // seconds: delay
    }

    get queue () {
        return 'high'
    }

	async handle(link, done) {
		// ...
		console.log(`Job [${this.constructor.name}] - handler called: status=running; id=${this.id} `)
    }

    progress(progress) {
        // ...
        console.log(`Job [${this.constructor.name}] - progress:${progress}%: status=running; id=${this.id} `)
    }
  
    failed(error) {
        // ...
		console.log(`Job [${this.constructor.name}] - status:failed; id=${this.id} `, error.message)
	}
	
	retrying(error){
        // ...
		console.log(`Job [${this.constructor.name}] - status:retrying; id=${this.id} `, error.message)
	}

    succeeded(result){
        // ...
        console.log(`Job [${this.constructor.name}] - status:succeeded; id=${this.id} `, result)
    }
}

module.exports = Test

and simple in my controller just write the code for test the queue
queue.dispatch(new Test())
when this action is call in the console I got the below log

@@adonisjs/Queue: [Redis] Queue [high] now ready

but the handler method is not running and when I'm checking the redis the keys is empty.

the configoration of the queue is default config without any changes.

How should I run the queue ?

Make:Job command failure

Prerequisites

Follow README file instructions in a brand new Adonis 4.1 project.

Package version

0.1.4

Node.js and npm version

Node 10.15.3 - Npm 6.9.0

Sample Code (to reproduce the issue)

I followed the installation as README file describe. When I ran the command adonis make:job ProcessTask --queue high I received the error:

Error: Cannot find module '../app/Commands/MakeJob'

1 Object.app.bind [as closure]
  /Users/finger/trackmob/hub-core/node_modules/adonisjs-queue/providers/JobCommandsProvider.js:15

2 Ioc._resolveBinding
  /Users/finger/trackmob/hub-core/node_modules/@adonisjs/fold/src/Ioc/index.js:233

3 Ioc.use
  /Users/finger/trackmob/hub-core/node_modules/@adonisjs/fold/src/Ioc/index.js:731

4 Kernel.addCommand
  /Users/finger/trackmob/hub-core/node_modules/@adonisjs/ace/src/Kernel/index.js:182

5 JobCommandsProvider.boot
  /Users/finger/trackmob/hub-core/node_modules/adonisjs-queue/providers/JobCommandsProvider.js:38

6 _.filter.map
  /Users/finger/trackmob/hub-core/node_modules/@adonisjs/fold/src/Registrar/index.js:147

7 arrayMap
  /Users/finger/trackmob/hub-core/node_modules/lodash/lodash.js:639

8 Function.map
  /Users/finger/trackmob/hub-core/node_modules/lodash/lodash.js:9556

9 interceptor
  /Users/finger/trackmob/hub-core/node_modules/lodash/lodash.js:16993

10 thru
  /Users/finger/trackmob/hub-core/node_modules/lodash/lodash.js:8797

11 anonymous
  /Users/finger/trackmob/hub-core/node_modules/lodash/lodash.js:4374

12 arrayReduce
  /Users/finger/trackmob/hub-core/node_modules/lodash/lodash.js:683

After fixing this bug in the package, I faced the following error running the same command:

invalid value for "--queue" flag: value is either 'high' or 'low'

Pull Request to fix both problems

released 0.1.5

Redis in queue

Do It is necessary to use REDIS for the jobs to work?

Providers missing AdonisJS 4.1

Package version

Adonis JS 4.1 - 0.1.7

Node.js and npm version

9.11.2

Sample Code (to reproduce the issue)

After running adonis install adonisjs-queue I don't have the commands to create jobs, and when I try and run "adonis serve", I'm getting the error Error: Cannot find module 'Queue' after creating the events.js

To fix, I had to manually add:

  'adonisjs-queue/providers/QueueProvider',
  'adonisjs-queue/providers/JobProvider',
  'adonisjs-queue/providers/JobCommandsProvider',

To the providers array. I think you might need to just update the docs.

It also didn't create the "config/queue.js" file.

It's also throwing an error
Error: Cannot call Queue#process twice
when using:

 // dispatch to the "high" priority queue
  await Queue.select('high').andDispatch(new SendNotification(id))

in the events.js file.

Did I miss a step on installation?

Error: Cannot call Queue#process twice

I have this error while calling the Queue from the controller Error: Cannot call Queue#process twice
It works when I call the Queue once every after the server has been restarted, but it shows that error every after I call it the apie with that queue in the second time,
this is my code in controller where I called the queue

await Queue.select('low').andDispatch(new EmailVoucher(voucher_id, emailContents))

[v0.1.6] HttpException causes the destroyAll method of queue to throw error

Package version

v0.1.5

Node.js and npm version

v10 - NodeJS
v6 - NPM

Actual Code (that causes the issue)

Line 165 , Col 28 (src/Queue/index.js)

  async destroyAll () {
    // See: https://stackoverflow.com/questions/44410119/in-javascript-does-using-await-inside-a-loop-block-the-loop/44410481

    for (let queue of this._queuesPool) { // TypeError: this.queuesPool is not iterable
      await this.close(queue)
      await this.destroy(queue)
    }
  }

How do I send multiple Jobs in one Queue or run multiple Queues

Hi there, first of all I'm new to node js and adonis so bear with me please. So I was trying to make some heavy process running on queue, including send emails and write some history logs.

Event.on('approve::spending_house', async (parameters)=>{
console.log('HISTORY LOG & EMAIL EVENT FIRED!')
await Queue.select('high').andDispatch(new SendEmailApprovePerHouse(parameters.arrayDeptIdEmail, parameters.arrayHouseIdEmail, parameters.year, parameters.logModel, parameters.sendEmailPerBO, parameters.divisi_id))
})

as you can see I created a Job named SendEmailApprovePerHouse it was working fine. But when I add another Job and put it in different event, it shows error Cannot call Queue#process twice.

This is how I defined the new Job in another event:

Event.on('approve::spending_house', async (parameters)=>{
console.log('HISTORY LOG & EMAIL EVENT FIRED!')
await Queue.select('high').andDispatch(new SendEmailApprovePerHouse(parameters.arrayDeptIdEmail, parameters.arrayHouseIdEmail, parameters.year, parameters.logModel, parameters.sendEmailPerBO, parameters.divisi_id))
})
Event.on('reject::spending_house', async (parameters)=>{
console.log('HISTORY LOG & EMAIL EVENT FIRED!')
await Queue.select('high').andDispatch(new SendEmailRejectPerHouse(parameters.arrayDeptIdEmail, parameters.arrayHouseIdEmail, parameters.year, parameters.logModel, parameters.sendEmailPerBO, parameters.divisi_id))
})`

How can I add another Job or Queue?
Thanks.

Doesn't Auto create queue.js file in config directory

After installation using
adonis install adonisjs-queue

The package doesn't create queue.js file in the config directory. So also documentation doesn't provide information on how to create, setup or configure the queue.js config file

Update Yarn and Npm

Please update package at Npm and Yarn

[ error ] "make:job" command not found

[ error ] "make:job" command not found

Email sent but callback is failed

Package version

NPM 6.12.1

Node.js and npm version

v10.16.0

Adonis version

4.1

Sample Code (to reproduce the issue)

JOB

/** @type {typeof import('adonisjs-queue/src/Job')} */
const Job = use('Job');

/** @type {typeof import('@adonisjs/mail/src/Mail')} */
const Mail = use('Mail');

class SendEmail extends Job {
  get queue() {
    return 'low';
  }

  constructor(emailAddress, emailFrom, emailSubject, emailTemplate, emailBody) {
    super(arguments);

    this.timeOut = 50; // seconds
    this.retryCount = 1;
    this.retryUntil = 200; // seconds
    // this.delayUntil = Date.parse('2038-01-19T03:14:08.000Z'); // optional, omit this line if not required
  }

  async handle(link, done) {
    console.log(
      `Job [${this.constructor.name}] - handler called: status=running; id=${this.id} `
    );

    link.reportProgress(1);

    let data = link.data; // arguments passed into the constructor
    let error = null;
    let result = null;

    try {
      result = await Mail.send(data.emailTemplate, data.emailBody, message => {
        message.to(data.emailAddress);
        message.from(data.emailFrom);
        message.subject(data.emailSubject);
      });
      link.reportProgress(45);
    } catch (err) {
      link.reportProgress(65);
      throw err;
    } finally {
      link.reportProgress(100);
    }

    return result;
  }

  failed(link, error) {
    console.log(
      `Job [${this.constructor.name}] - status:failed; id=${this.id} `,
      error
    );

    this.detach(); // remove the job from the queue (when the job fails after 3 retries)
  }

  retrying(link, error) {
    console.log(
      `Job [${this.constructor.name}] - status:retrying; id=${this.id} `,
      error
    );
  }

  succeeded(link) {
    console.log(
      `Job [${this.constructor.name}] - status:succeeded; id=${this.id} `
    );
  }
}

module.exports = SendEmail;

Event

/** @type {typeof import('@adonisjs/framework/src/Event')} */
const Event = use('Event');

/** @type {typeof import('adonisjs-queue/src/Queue')} */
const Queue = use('Queue');

const SendEmail = use('App/Jobs/SendEmail');

Event.on('user_registered', async (_email, _body) => {
  await Queue.dispatch(
    new SendEmail(
      _email,
      '[email protected]',
      'Registro',
      'emails.UserStore',
      _body
    )
  );
});

Response

@@adonisjs/Queue: [Redis] Queue [low] now ready
Job [SendEmail] - handler called: status=running; id=1 
Job [SendEmail] - status:retrying; id=1  undefined
Job [SendEmail] - handler called: status=running; id=1 
Job [SendEmail] - status:failed; id=1  undefined

The email is sent successfully but the callback that is made is invalid

E_INVALID_QUEUE_DRIVER: rabbitmq is not a valid queue driver

Package version

0.1.10

Node.js and npm version

Node V - 14.15.4
Npm - 6.14.10

Sample Code (to reproduce the issue)

 'use strict'

const Env = use('Env')

module.exports = {
  driver: 'rabbitmq', // :> 'redis' , 'rabbitmq'
  redis: {
    url: '_',
    high: {
      prefix: 'mainh-',
      stallInterval: 5000,
      nearTermWindow: 1200000,
      delayedDebounce: 1000,
      redis: {
        host: Env.get('REDIS_HOST', '127.0.0.1'),
        port: Env.get('REDIS_PORT', '6379'),
        db: 1,
        options: { attempt: 20 },
        retry_strategy (options) {
          return Math.min(options.attempt * 100, 3000)
        }
      },
      isWorker: true,
      getEvents: true,
      sendEvents: true,
      storeJobs: true,
      ensureScripts: true,
      maxConcurrencyJobs: 4,
      activateDelayedJobs: true,
      removeOnSuccess: true,
      removeOnFailure: false,
      redisScanCount: 100
    },
    low: {
      prefix: 'mainl-',
      stallInterval: 8000,
      nearTermWindow: 1200000,
      delayedDebounce: 2000,
      redis: {
        host: Env.get('REDIS_HOST', '127.0.0.1'),
        port: Env.get('REDIS_PORT', '6379'),
        db: 2,
        options: { attempt: 20 },
        retry_strategy (options) {
          return Math.min(options.attempt * 100, 3000)
        }
      },
      isWorker: true,
      getEvents: true,
      sendEvents: true,
      storeJobs: true,
      ensureScripts: true,
      maxConcurrencyJobs: 4,
      activateDelayedJobs: true,
      removeOnSuccess: true,
      removeOnFailure: false,
      redisScanCount: 150
    }
  },
  rabbitmq: {
    url: Env.get('RABBIT_MQ_URL', 'amqp://localhost'),
    high: {
      prefetch: 1, // default prefetch from queue
      replyPattern: false, // if reply pattern is enabled an exclusive queue is created
      scheduledPublish: false,
      prefix: 'mainh-', // prefix all queues with an application name
      socketOptions: {} // socketOptions will be passed as a second param to amqp.connect and from ther to the socket library (net or tls)
    },
    low: {
      prefetch: 1, // default prefetch from queue
      replyPattern: false, // if reply pattern is enabled an exclusive queue is created
      scheduledPublish: false,
      prefix: 'mainl-', // prefix all queues with an application name
      socketOptions: {} // socketOptions will be passed as a second param to amqp.connect and from ther to the socket library (net or tls)
    }
  }
}

My issue is that i keep getting this error in my console when i start my application. Followed up the instructions to the letter and still getting this error. Can you please help ?