enioka-haute-couture / jqm Goto Github PK

An asynchronous job manager

License: Apache License 2.0

Python 0.12% Java 81.28% PowerShell 0.81% Shell 0.73% HTML 0.95% CSS 0.05% JavaScript 0.03% Dockerfile 0.11% Batchfile 0.21% TypeScript 15.71%

jqm's Introduction

JQM

Full documentation and feature list

Quickstart tutorial

Release notes

The aptly named Job Queue Manager, or JQM for short, is a queue manager. It has three goals:

to optimize and streamline the execution of jobs, whatever they may be, by using queues with rich configuration
to make job administration simple
to be easy to include or embed in most environments.

The result is a small-footprint, easy to use grid execution system that takes care of everything which would be boilerplate code or missing otherwise: configuring logs, throttling processes, handling priorities between different classes of jobs, distributing the load over multiple servers, distributing the files created by the jobs, and much more...

It is able to run anything that can be run on the command line without modifications. It also has a very rich Java integration which make it an ideal "job application server" for Java users - with no modifications required, allowing to directly use code from plain Main to Spring Batch and other frameworks...

Jobs and users also optionally benefit from rich REST APIs exposing all JQM data and operations.

There are many use cases for JQM. Common real-world examples include:

replacing another job execution manager, like the one inside OS/400 - a reference as far as job queuing is concerned
adding a distributed execution capability to a scheduler or any application
removing load from a paid application server
removing asynchronous executions from a web application server, not designed to deal with long running threads
throttling jobs, like allowing only one instance of the same job at a time

Also of note that JQM was created with compatibility in mind:

uses either PostgreSQL, Oracle, MySQL, DB2 or an embedded HSQLDB
one of the client API implementations is a REST-like API, callable from everywhere, not only Java but also .NET or shell scripts
the Java implementation of the client API is usable in all application servers and JSE code (tested with WebSphere 8.x, Glassfish 3.x, Tomcat 7.x, JBoss 7+...)
under an Apache 2 license, which basically allows you to do anything you want with the product and its code in any situation.

Finally, JQM is a free (as beer) and open source product backed by the IT consulting firm Enioka which first developed it for an international conglomerate. Enquiries about support, development of extensions, integration with other products, consulting and other commercial questions are more than welcome at [email protected]. Community support is of course freely offered on GitHub using the bug-tracker.

jqm's People

Contributors

Stargazers

Watchers

jqm's Issues

Try again using resourcefactory for jdbc datasources

Issue: currently we use DriverManager to load drivers. It means there can only be a single version of each driver Inside a running engine (JVM).
We should try to allow per application driver loading, through the ObjectFactory system.
However, a first try showed issues with driver registration that could not be deregsitred, and therefore classloader leaks.
The current situation is acceptable, however an enhancement may be in order.

Enhance default args parser

Would be nice to have a simple --help and better args parse to start a job from CLI

HIbernate upgrade for HHH-7479

Add payload launch without dependencies towards jqm-api

Use injection for communication from payload to engine

Test performance enhancement

Add a helper function that will wait for a certain number of results with timeout.

Prevent payload from calling System.exit()

Use a Security Manager.

Sonar hunt

HHH000388 on node creation under Oracle

Version 1.1.4.
Not blocking, but an index creation fails with Oracle:

HHH000388: Unsuccessful: create index idx_fk_jobparameter_jobinstance on JobParameter (jobinstance_id)

test payloads references cleanup

Why copy them?

Use build order instead of dependencies for referencing them Inside engine

This will also remove the "jqm-engine" directory created by tests.

Offer to reset cache

Useful when changing a jar - currently the whole server has to be restarted.

Bad error message when database connection is not well defined

If database connection is not well defined (example bad password); jqm engine give a not mysterious message (see below). One should catch that and return a proper error message to complain about database configuration.

Exception in thread "main" java.lang.ExceptionInInitializerError
        at java.lang.J9VMInternals.initialize(J9VMInternals.java:263)
        at com.enioka.jqm.tools.JqmEngine.<init>(JqmEngine.java:55)
        at com.enioka.jqm.tools.Main.main(Main.java:42)
Caused by: javax.persistence.PersistenceException: [PersistenceUnit: jobqueue-api-pu] Unable to build EntityManagerFactory
        at org.hibernate.ejb.Ejb3Configuration.buildEntityManagerFactory(Ejb3Configuration.java:924)
        at org.hibernate.ejb.Ejb3Configuration.buildEntityManagerFactory(Ejb3Configuration.java:899)
        at org.hibernate.ejb.HibernatePersistence.createEntityManagerFactory(HibernatePersistence.java:59)
        at javax.persistence.Persistence.createEntityManagerFactory(Persistence.java:63)
        at com.enioka.jqm.tools.Helpers.createFactory(Helpers.java:68)
        at com.enioka.jqm.tools.Helpers.<clinit>(Helpers.java:47)
        at java.lang.J9VMInternals.initializeImpl(Native Method)
        at java.lang.J9VMInternals.initialize(J9VMInternals.java:241)
        ... 2 more
Caused by: org.hibernate.HibernateException: Connection cannot be null when 'hibernate.dialect' not set
        at org.hibernate.service.jdbc.dialect.internal.DialectFactoryImpl.determineDialect(DialectFactoryImpl.java:98)
        at org.hibernate.service.jdbc.dialect.internal.DialectFactoryImpl.buildDialect(DialectFactoryImpl.java:68)
        at org.hibernate.engine.jdbc.internal.JdbcServicesImpl.configure(JdbcServicesImpl.java:170)
        at org.hibernate.service.internal.StandardServiceRegistryImpl.configureService(StandardServiceRegistryImpl.java:76)
        at org.hibernate.service.internal.AbstractServiceRegistryImpl.initializeService(AbstractServiceRegistryImpl.java:160)
        at org.hibernate.service.internal.AbstractServiceRegistryImpl.getService(AbstractServiceRegistryImpl.java:132)
        at org.hibernate.cfg.Configuration.buildTypeRegistrations(Configuration.java:1822)
        at org.hibernate.cfg.Configuration.buildSessionFactory(Configuration.java:1780)
        at org.hibernate.ejb.EntityManagerFactoryImpl.<init>(EntityManagerFactoryImpl.java:96)
        at org.hibernate.ejb.Ejb3Configuration.buildEntityManagerFactory(Ejb3Configuration.java:914)
        ... 9 more

Set Oracle program name to help DBAs

See http://stackoverflow.com/questions/3389180/configure-jdbc-oracle-specific-property-vsession-program-using-jboss-and-jpa-h

Refactor fixfile: should be replaced by commons io.

and therefore be made consistent with the rest of the code.

Legacy engine API does not support inheriting JobBase

If a job extends a class thta extends JobBase, the class is not recognized as a valid JQM job.
Works perfectly if first level.

Add generic query client API method

SO that all fields, such as keywords, may be used.

Two logs: one internal (engine) and one concerning payload run

Multiple start of the same node should not be allowed

Moreover, it seems that multiple same node leads to some kind of dead lock: jobs are not processed anymore. Killing all nodes and starting just one make its start again.

A node listens... on an interface named like the node name

.. basically, does not work !

Refactor client API Hibernate

Lots of duplicated code, EM handling inconsistent

History keywords are from JobDef and not from JobInstance

JobDef keywords are stable - they should not be put into history, as they can be retrieved easily. The goal of keyworks is to sort instances - but keywords have disappeared from JI. Put them back.

New payload type: static main(args)

Currenlty, JQM can launch classes inheriting from our API and Runnables. This should make it possible to launch classes with a classic main function.

Args should be the parameters sorted by key (alpha sort, culture insensitive). Only value given, not key.

Existing lib directory is not used (only used inside jar)

Linux: reroute stdout/err to logs directory, not root directory

Windows is already OK.

Too many warnings

Missing getInstanceId in legacy engine API

Add client API through WS instead of JPA

And remove the horrible Hibernate dependency from EE6 application using JQM

Add ping "i am alive" to poller JMX monitoring

Display version at startup

Random failure of test MultiNodeTest#testHighlanderMode

Only observed when the test is inside a suite (test alone works - even if it may be out of sheer luck)
Log below.

223318 [main] INFO com.enioka.jqm.api.Dispatcher - New EMF will be created
223768 [main] DEBUG com.enioka.jqm.tools.JobBaseTest - **********************************************************
223770 [main] DEBUG com.enioka.jqm.tools.JobBaseTest - **********************************************************
223770 [main] DEBUG com.enioka.jqm.tools.JobBaseTest - Starting test testHighlanderMode
223801 [main] DEBUG com.enioka.jqm.api.Dispatcher - BEGINING ENQUEUE
223801 [main] DEBUG com.enioka.jqm.api.Dispatcher - A new EM will be created
223804 [main] DEBUG com.enioka.jqm.api.Dispatcher - Job to enqueue is from JobDef 85
223804 [main] DEBUG com.enioka.jqm.api.Dispatcher - Highlander mode analysis is begining
223807 [main] DEBUG com.enioka.jqm.api.Dispatcher - Highlander mode will return: null
223807 [main] DEBUG com.enioka.jqm.api.Dispatcher - Not in highlander mode or no currently enqued instance
223808 [main] DEBUG com.enioka.jqm.api.Dispatcher - JI just created: 456
223808 [main] DEBUG com.enioka.jqm.api.Dispatcher - History just created: 456
223809 [main] DEBUG com.enioka.jqm.api.Dispatcher - BEGINING ENQUEUE
223809 [main] DEBUG com.enioka.jqm.api.Dispatcher - A new EM will be created
223811 [main] DEBUG com.enioka.jqm.api.Dispatcher - Job to enqueue is from JobDef 85
223811 [main] DEBUG com.enioka.jqm.api.Dispatcher - Highlander mode analysis is begining
223812 [main] DEBUG com.enioka.jqm.api.Dispatcher - JI seen by highlander: 456SUBMITTED
223812 [main] DEBUG com.enioka.jqm.api.Dispatcher - In the highlander if
223812 [main] DEBUG com.enioka.jqm.api.Dispatcher - Highlander mode will return: 456
223812 [main] DEBUG com.enioka.jqm.api.Dispatcher - JI won't actually be enqueued because a job in highlander mode is currently submitted: 456
223812 [main] DEBUG com.enioka.jqm.api.Dispatcher - BEGINING ENQUEUE
223812 [main] DEBUG com.enioka.jqm.api.Dispatcher - A new EM will be created
223813 [main] DEBUG com.enioka.jqm.api.Dispatcher - Job to enqueue is from JobDef 85
223814 [main] DEBUG com.enioka.jqm.api.Dispatcher - Highlander mode analysis is begining
223815 [main] DEBUG com.enioka.jqm.api.Dispatcher - JI seen by highlander: 456SUBMITTED
223815 [main] DEBUG com.enioka.jqm.api.Dispatcher - In the highlander if
223815 [main] DEBUG com.enioka.jqm.api.Dispatcher - Highlander mode will return: 456
223815 [main] DEBUG com.enioka.jqm.api.Dispatcher - JI won't actually be enqueued because a job in highlander mode is currently submitted: 456
223815 [main] DEBUG com.enioka.jqm.api.Dispatcher - BEGINING ENQUEUE
223815 [main] DEBUG com.enioka.jqm.api.Dispatcher - A new EM will be created
223816 [main] DEBUG com.enioka.jqm.api.Dispatcher - Job to enqueue is from JobDef 85
223817 [main] DEBUG com.enioka.jqm.api.Dispatcher - Highlander mode analysis is begining
223817 [main] DEBUG com.enioka.jqm.api.Dispatcher - JI seen by highlander: 456SUBMITTED
223817 [main] DEBUG com.enioka.jqm.api.Dispatcher - In the highlander if
223818 [main] DEBUG com.enioka.jqm.api.Dispatcher - Highlander mode will return: 456
223818 [main] DEBUG com.enioka.jqm.api.Dispatcher - JI won't actually be enqueued because a job in highlander mode is currently submitted: 456
223818 [main] DEBUG com.enioka.jqm.api.Dispatcher - BEGINING ENQUEUE
223818 [main] DEBUG com.enioka.jqm.api.Dispatcher - A new EM will be created
223819 [main] DEBUG com.enioka.jqm.api.Dispatcher - Job to enqueue is from JobDef 85
223819 [main] DEBUG com.enioka.jqm.api.Dispatcher - Highlander mode analysis is begining
223820 [main] DEBUG com.enioka.jqm.api.Dispatcher - JI seen by highlander: 456SUBMITTED
223820 [main] DEBUG com.enioka.jqm.api.Dispatcher - In the highlander if
223820 [main] DEBUG com.enioka.jqm.api.Dispatcher - Highlander mode will return: 456
223820 [main] DEBUG com.enioka.jqm.api.Dispatcher - JI won't actually be enqueued because a job in highlander mode is currently submitted: 456
223821 [main] DEBUG com.enioka.jqm.api.Dispatcher - BEGINING ENQUEUE
223821 [main] DEBUG com.enioka.jqm.api.Dispatcher - A new EM will be created
223822 [main] DEBUG com.enioka.jqm.api.Dispatcher - Job to enqueue is from JobDef 85
223822 [main] DEBUG com.enioka.jqm.api.Dispatcher - Highlander mode analysis is begining
223823 [main] DEBUG com.enioka.jqm.api.Dispatcher - JI seen by highlander: 456SUBMITTED
223823 [main] DEBUG com.enioka.jqm.api.Dispatcher - In the highlander if
223823 [main] DEBUG com.enioka.jqm.api.Dispatcher - Highlander mode will return: 456
223823 [main] DEBUG com.enioka.jqm.api.Dispatcher - JI won't actually be enqueued because a job in highlander mode is currently submitted: 456
223823 [main] DEBUG com.enioka.jqm.api.Dispatcher - BEGINING ENQUEUE
223823 [main] DEBUG com.enioka.jqm.api.Dispatcher - A new EM will be created
223825 [main] DEBUG com.enioka.jqm.api.Dispatcher - Job to enqueue is from JobDef 85
223825 [main] DEBUG com.enioka.jqm.api.Dispatcher - Highlander mode analysis is begining
223826 [main] DEBUG com.enioka.jqm.api.Dispatcher - JI seen by highlander: 456SUBMITTED
223826 [main] DEBUG com.enioka.jqm.api.Dispatcher - In the highlander if
223826 [main] DEBUG com.enioka.jqm.api.Dispatcher - Highlander mode will return: 456
223826 [main] DEBUG com.enioka.jqm.api.Dispatcher - JI won't actually be enqueued because a job in highlander mode is currently submitted: 456
223826 [main] DEBUG com.enioka.jqm.api.Dispatcher - BEGINING ENQUEUE
223826 [main] DEBUG com.enioka.jqm.api.Dispatcher - A new EM will be created
223827 [main] DEBUG com.enioka.jqm.api.Dispatcher - Job to enqueue is from JobDef 85
223828 [main] DEBUG com.enioka.jqm.api.Dispatcher - Highlander mode analysis is begining
223828 [main] DEBUG com.enioka.jqm.api.Dispatcher - JI seen by highlander: 456SUBMITTED
223828 [main] DEBUG com.enioka.jqm.api.Dispatcher - In the highlander if
223829 [main] DEBUG com.enioka.jqm.api.Dispatcher - Highlander mode will return: 456
223829 [main] DEBUG com.enioka.jqm.api.Dispatcher - JI won't actually be enqueued because a job in highlander mode is currently submitted: 456
223829 [main] DEBUG com.enioka.jqm.api.Dispatcher - BEGINING ENQUEUE
223829 [main] DEBUG com.enioka.jqm.api.Dispatcher - A new EM will be created
223830 [main] DEBUG com.enioka.jqm.api.Dispatcher - Job to enqueue is from JobDef 85
223831 [main] DEBUG com.enioka.jqm.api.Dispatcher - Highlander mode analysis is begining
223831 [main] DEBUG com.enioka.jqm.api.Dispatcher - JI seen by highlander: 456SUBMITTED
223831 [main] DEBUG com.enioka.jqm.api.Dispatcher - In the highlander if
223831 [main] DEBUG com.enioka.jqm.api.Dispatcher - Highlander mode will return: 456
223831 [main] DEBUG com.enioka.jqm.api.Dispatcher - JI won't actually be enqueued because a job in highlander mode is currently submitted: 456
223834 [main] INFO com.enioka.jqm.tools.JqmEngine - Node localhost was found in the configuration
223835 [main] INFO com.enioka.jqm.tools.JqmEngine - There are 9 queues defined in the database
223839 [main] WARN com.enioka.jqm.tools.JqmEngine - Queue VIPQueue was modified to become the default queue as there was no default queue
223841 [main] INFO com.enioka.jqm.tools.JqmEngine - This GlobalParameter is already exists
223843 [main] INFO com.enioka.jqm.tools.JqmEngine - This node is already configured to take jobs from the default queue
223845 [main] INFO com.enioka.jqm.tools.JqmEngine - A JNDI alias towards the JQM db has been created. It references: jdbc:hsqldb:hsql://localhost/testdbengine
223846 [main] INFO com.enioka.jqm.tools.JqmEngine - Log level is set at DEBUG which translates as log4j level DEBUG
223846 [main] INFO com.enioka.jqm.tools.JqmEngine - Starting Jetty (port 0)
223848 [main] INFO com.enioka.jqm.tools.JqmEngine - Jetty has started on port 60688
223855 [main] DEBUG com.enioka.jqm.tools.Polling - Polling JobInstances with the Deployment Parameter: 1
223961 [main] DEBUG com.enioka.jqm.tools.Polling - Polling JobInstances with the Deployment Parameter: 1
223976 [Thread-788] DEBUG com.enioka.jqm.tools.Polling - JI number 456 will be selected by this poller loop (already 0/40 on VIPQueue)
223977 [Thread-788] DEBUG com.enioka.jqm.tools.Polling - ((((((((((((((((((()))))))))))))))))
223977 [Thread-788] DEBUG com.enioka.jqm.tools.Polling - Actual deploymentParameter: 287
223977 [Thread-788] DEBUG com.enioka.jqm.tools.Polling - Theorical max nbThread: 40
223977 [Thread-788] DEBUG com.enioka.jqm.tools.Polling - Actual nbThread: 0
223977 [Thread-788] DEBUG com.enioka.jqm.tools.Polling - JI that will be attributed: 456
223977 [Thread-788] DEBUG com.enioka.jqm.tools.Polling - ((((((((((((((((((()))))))))))))))))
223981 [Thread-788] DEBUG com.enioka.jqm.tools.Polling - The job & history 456 have been updated with the node: localhost
223981 [Thread-788] DEBUG com.enioka.jqm.tools.Polling - TPS QUEUE: 516
223981 [Thread-788] DEBUG com.enioka.jqm.tools.Polling - INCREMENTATION NBTHREAD: 1
223981 [Thread-788] DEBUG com.enioka.jqm.tools.Polling - POLLING QUEUE: 516
223981 [Thread-788] DEBUG com.enioka.jqm.tools.Polling - Should the node corresponding to the current job be stopped: false
223982 [Thread-788] INFO com.enioka.jqm.tools.ThreadPool - Job instance will be inserted inside a thread pool: 456
223982 [Thread-788] DEBUG com.enioka.jqm.tools.ThreadPool - ThreadPool ActualNbThread: 1
224069 [main] DEBUG com.enioka.jqm.tools.Polling - Polling JobInstances with the Deployment Parameter: 1
224086 [Thread-788] DEBUG com.enioka.jqm.tools.Polling - End of poller loop on queue VIPQueue
224087 [pool-175-thread-1] DEBUG com.enioka.jqm.tools.Loader - LOADER HAS JUST STARTED UP FOR JOB INSTANCE 456
224090 [pool-175-thread-1] DEBUG com.enioka.jqm.tools.Loader - Loader will try to launch jar /home/mag/git/jqm/jqm-all/jqm-engine/./testprojects/jqm-test-datetimesendmsg/jqm-test-datetimesendmsg.jar - App
224090 [pool-175-thread-1] INFO com.enioka.jqm.tools.Loader - Application MarsuApplication dependencies are not yet in cache
224090 [pool-175-thread-1] DEBUG com.enioka.jqm.tools.Loader - Loader will try to load POM /home/mag/git/jqm/jqm-all/jqm-engine/testprojects/jqm-test-datetimesendmsg/pom.xml
224090 [pool-175-thread-1] DEBUG com.enioka.jqm.tools.Loader - Reading a pom file
224091 [pool-175-thread-1] DEBUG com.enioka.jqm.tools.Dependencies - /home/mag/git/jqm/jqm-all/jqm-engine/testprojects/jqm-test-datetimesendmsg/pom.xml
224091 [pool-175-thread-1] DEBUG com.enioka.jqm.tools.Dependencies - Working Directory = /home/mag/git/jqm/jqm-all/jqm-engine
224103 [pool-175-thread-1] DEBUG com.enioka.jqm.tools.Loader - Resolving Maven dep com.enioka.jqm:jqm-tests:pom:1.1.4-SNAPSHOT
224114 [pool-175-thread-1] DEBUG com.enioka.jqm.tools.Loader - Resolving Maven dep org.hibernate:hibernate-entitymanager:3.5.6-Final
224155 [pool-175-thread-1] DEBUG com.enioka.jqm.tools.Loader - Resolving Maven dep org.hsqldb:hsqldb:2.3.0
224158 [pool-175-thread-1] DEBUG com.enioka.jqm.tools.Loader - Resolving Maven dep com.enioka.jqm:jqm-api:1.1.4-SNAPSHOT
224162 [pool-175-thread-1] DEBUG com.enioka.jqm.tools.Loader - Resolving Maven dep com.enioka.jqm:jqm-clientapi:1.1.4-SNAPSHOT
224177 [main] INFO com.enioka.jqm.tools.JqmEngine - End of JQM engine initialization
224178 [main] INFO com.enioka.jqm.tools.JqmEngine - Node localhost4 was found in the configuration
224178 [main] INFO com.enioka.jqm.tools.JqmEngine - There are 9 queues defined in the database
224179 [main] INFO com.enioka.jqm.tools.JqmEngine - Default queue is named VIPQueue
224179 [main] INFO com.enioka.jqm.tools.JqmEngine - This GlobalParameter is already exists
224179 [main] INFO com.enioka.jqm.tools.JqmEngine - This node is already configured to take jobs from the default queue
224180 [main] INFO com.enioka.jqm.tools.JqmEngine - The jdbc/jqm alias already exists and references jdbc:hsqldb:hsql://localhost/testdbengine
224180 [main] INFO com.enioka.jqm.tools.JqmEngine - Log level is set at DEBUG which translates as log4j level DEBUG
224181 [main] INFO com.enioka.jqm.tools.JqmEngine - Starting Jetty (port 0)
224193 [main] INFO com.enioka.jqm.tools.JqmEngine - Jetty has started on port 35556
224195 [pool-175-thread-1] DEBUG com.enioka.jqm.tools.Loader - Resolving Maven dep joda-time:joda-time:2.3
224196 [main] DEBUG com.enioka.jqm.tools.Polling - Polling JobInstances with the Deployment Parameter: 2
224201 [main] INFO com.enioka.jqm.tools.JqmEngine - End of JQM engine initialization
224201 [main] DEBUG com.enioka.jqm.api.Dispatcher - BEGINING ENQUEUE
224201 [main] DEBUG com.enioka.jqm.api.Dispatcher - A new EM will be created
224203 [main] DEBUG com.enioka.jqm.api.Dispatcher - Job to enqueue is from JobDef 85
224203 [main] DEBUG com.enioka.jqm.api.Dispatcher - Highlander mode analysis is begining
224204 [main] DEBUG com.enioka.jqm.api.Dispatcher - Highlander mode will return: null
224204 [main] DEBUG com.enioka.jqm.api.Dispatcher - Not in highlander mode or no currently enqued instance
224204 [main] DEBUG com.enioka.jqm.api.Dispatcher - JI just created: 457
224206 [main] DEBUG com.enioka.jqm.api.Dispatcher - History just created: 457
224209 [main] DEBUG com.enioka.jqm.api.Dispatcher - BEGINING ENQUEUE
224209 [main] DEBUG com.enioka.jqm.api.Dispatcher - A new EM will be created
224210 [main] DEBUG com.enioka.jqm.api.Dispatcher - Job to enqueue is from JobDef 85
224210 [main] DEBUG com.enioka.jqm.api.Dispatcher - Highlander mode analysis is begining
224213 [main] DEBUG com.enioka.jqm.api.Dispatcher - JI seen by highlander: 457SUBMITTED
224213 [main] DEBUG com.enioka.jqm.api.Dispatcher - In the highlander if
224213 [main] DEBUG com.enioka.jqm.api.Dispatcher - Highlander mode will return: 457
224213 [main] DEBUG com.enioka.jqm.api.Dispatcher - JI won't actually be enqueued because a job in highlander mode is currently submitted: 457
224215 [main] DEBUG com.enioka.jqm.api.Dispatcher - BEGINING ENQUEUE
224215 [main] DEBUG com.enioka.jqm.api.Dispatcher - A new EM will be created
224219 [main] DEBUG com.enioka.jqm.api.Dispatcher - Job to enqueue is from JobDef 85
224219 [main] DEBUG com.enioka.jqm.api.Dispatcher - Highlander mode analysis is begining
224220 [pool-175-thread-1] DEBUG com.enioka.jqm.tools.Loader - Artifact from pom: file:/home/mag/.m2/repository/org/hibernate/hibernate-entitymanager/3.5.6-Final/hibernate-entitymanager-3.5.6-Final.jar
224221 [pool-175-thread-1] DEBUG com.enioka.jqm.tools.Loader - Artifact from pom: file:/home/mag/.m2/repository/org/hibernate/hibernate-core/3.5.6-Final/hibernate-core-3.5.6-Final.jar
224221 [pool-175-thread-1] DEBUG com.enioka.jqm.tools.Loader - Artifact from pom: file:/home/mag/.m2/repository/antlr/antlr/2.7.6/antlr-2.7.6.jar
224222 [pool-175-thread-1] DEBUG com.enioka.jqm.tools.Loader - Artifact from pom: file:/home/mag/.m2/repository/commons-collections/commons-collections/3.1/commons-collections-3.1.jar
224222 [pool-175-thread-1] DEBUG com.enioka.jqm.tools.Loader - Artifact from pom: file:/home/mag/.m2/repository/dom4j/dom4j/1.6.1/dom4j-1.6.1.jar
224222 [pool-175-thread-1] DEBUG com.enioka.jqm.tools.Loader - Artifact from pom: file:/home/mag/.m2/repository/xml-apis/xml-apis/1.0.b2/xml-apis-1.0.b2.jar
224222 [pool-175-thread-1] DEBUG com.enioka.jqm.tools.Loader - Artifact from pom: file:/home/mag/.m2/repository/javax/transaction/jta/1.1/jta-1.1.jar
224222 [pool-175-thread-1] DEBUG com.enioka.jqm.tools.Loader - Artifact from pom: file:/home/mag/.m2/repository/org/hibernate/hibernate-annotations/3.5.6-Final/hibernate-annotations-3.5.6-Final.jar
224222 [pool-175-thread-1] DEBUG com.enioka.jqm.tools.Loader - Artifact from pom: file:/home/mag/.m2/repository/org/hibernate/hibernate-commons-annotations/3.2.0.Final/hibernate-commons-annotations-3.2.0.Final.jar
224222 [pool-175-thread-1] DEBUG com.enioka.jqm.tools.Loader - Artifact from pom: file:/home/mag/.m2/repository/cglib/cglib/2.2/cglib-2.2.jar
224223 [pool-175-thread-1] DEBUG com.enioka.jqm.tools.Loader - Artifact from pom: file:/home/mag/.m2/repository/asm/asm/3.1/asm-3.1.jar
224223 [pool-175-thread-1] DEBUG com.enioka.jqm.tools.Loader - Artifact from pom: file:/home/mag/.m2/repository/javassist/javassist/3.9.0.GA/javassist-3.9.0.GA.jar
224220 [main] DEBUG com.enioka.jqm.api.Dispatcher - JI seen by highlander: 457SUBMITTED
224223 [main] DEBUG com.enioka.jqm.api.Dispatcher - In the highlander if
224223 [main] DEBUG com.enioka.jqm.api.Dispatcher - Highlander mode will return: 457
224223 [main] DEBUG com.enioka.jqm.api.Dispatcher - JI won't actually be enqueued because a job in highlander mode is currently submitted: 457
224227 [main] DEBUG com.enioka.jqm.api.Dispatcher - BEGINING ENQUEUE
224227 [main] DEBUG com.enioka.jqm.api.Dispatcher - A new EM will be created
224228 [main] DEBUG com.enioka.jqm.api.Dispatcher - Job to enqueue is from JobDef 85
224231 [main] DEBUG com.enioka.jqm.api.Dispatcher - Highlander mode analysis is begining
224231 [pool-175-thread-1] DEBUG com.enioka.jqm.tools.Loader - Artifact from pom: file:/home/mag/.m2/repository/org/hibernate/javax/persistence/hibernate-jpa-2.0-api/1.0.0.Final/hibernate-jpa-2.0-api-1.0.0.Final.jar
224231 [pool-175-thread-1] DEBUG com.enioka.jqm.tools.Loader - Artifact from pom: file:/home/mag/.m2/repository/javax/validation/validation-api/1.0.0.GA/validation-api-1.0.0.GA.jar
224232 [pool-175-thread-1] DEBUG com.enioka.jqm.tools.Loader - Artifact from pom: file:/home/mag/.m2/repository/org/slf4j/slf4j-api/1.5.8/slf4j-api-1.5.8.jar
224232 [pool-175-thread-1] DEBUG com.enioka.jqm.tools.Loader - Artifact from pom: file:/home/mag/.m2/repository/org/hsqldb/hsqldb/2.3.0/hsqldb-2.3.0.jar
224232 [pool-175-thread-1] DEBUG com.enioka.jqm.tools.Loader - Artifact from pom: file:/home/mag/.m2/repository/com/enioka/jqm/jqm-api/1.1.4-SNAPSHOT/jqm-api-1.1.4-SNAPSHOT.jar
224232 [pool-175-thread-1] DEBUG com.enioka.jqm.tools.Loader - Artifact from pom: file:/home/mag/.m2/repository/com/enioka/jqm/jqm-clientapi/1.1.4-SNAPSHOT/jqm-clientapi-1.1.4-SNAPSHOT.jar
224233 [pool-175-thread-1] DEBUG com.enioka.jqm.tools.Loader - Artifact from pom: file:/home/mag/.m2/repository/com/enioka/jqm/jqm-model/1.1.4-SNAPSHOT/jqm-model-1.1.4-SNAPSHOT.jar
224233 [pool-175-thread-1] DEBUG com.enioka.jqm.tools.Loader - Artifact from pom: file:/home/mag/.m2/repository/commons-io/commons-io/2.4/commons-io-2.4.jar
224234 [pool-175-thread-1] DEBUG com.enioka.jqm.tools.Loader - Artifact from pom: file:/home/mag/.m2/repository/org/slf4j/slf4j-log4j12/1.7.5/slf4j-log4j12-1.7.5.jar
224234 [pool-175-thread-1] DEBUG com.enioka.jqm.tools.Loader - Artifact from pom: file:/home/mag/.m2/repository/org/slf4j/slf4j-api/1.7.5/slf4j-api-1.7.5.jar
224234 [pool-175-thread-1] DEBUG com.enioka.jqm.tools.Loader - Artifact from pom: file:/home/mag/.m2/repository/log4j/log4j/1.2.17/log4j-1.2.17.jar
224234 [pool-175-thread-1] DEBUG com.enioka.jqm.tools.Loader - Artifact from pom: file:/home/mag/.m2/repository/joda-time/joda-time/2.3/joda-time-2.3.jar
224234 [pool-175-thread-1] DEBUG com.enioka.jqm.tools.Loader - Artifact from pom: file:/home/mag/.m2/repository/org/joda/joda-convert/1.2/joda-convert-1.2.jar
224232 [main] DEBUG com.enioka.jqm.api.Dispatcher - JI seen by highlander: 457SUBMITTED
224234 [main] DEBUG com.enioka.jqm.api.Dispatcher - In the highlander if
224235 [main] DEBUG com.enioka.jqm.api.Dispatcher - Highlander mode will return: 457
224235 [main] DEBUG com.enioka.jqm.api.Dispatcher - JI won't actually be enqueued because a job in highlander mode is currently submitted: 457
224238 [main] DEBUG com.enioka.jqm.api.Dispatcher - BEGINING ENQUEUE
224238 [main] DEBUG com.enioka.jqm.api.Dispatcher - A new EM will be created
224239 [main] DEBUG com.enioka.jqm.api.Dispatcher - Job to enqueue is from JobDef 85
224241 [main] DEBUG com.enioka.jqm.api.Dispatcher - Highlander mode analysis is begining
224243 [main] DEBUG com.enioka.jqm.api.Dispatcher - JI seen by highlander: 457SUBMITTED
224244 [main] DEBUG com.enioka.jqm.api.Dispatcher - In the highlander if
224244 [main] DEBUG com.enioka.jqm.api.Dispatcher - Highlander mode will return: 457
224244 [main] DEBUG com.enioka.jqm.api.Dispatcher - JI won't actually be enqueued because a job in highlander mode is currently submitted: 457
224247 [main] DEBUG com.enioka.jqm.api.Dispatcher - BEGINING ENQUEUE
224248 [main] DEBUG com.enioka.jqm.api.Dispatcher - A new EM will be created
224249 [pool-175-thread-1] ERROR org.hibernate.engine.jdbc.spi.SqlExceptionHelper - transaction rollback: serialization failure
224249 [main] DEBUG com.enioka.jqm.api.Dispatcher - Job to enqueue is from JobDef 85
224250 [main] DEBUG com.enioka.jqm.api.Dispatcher - Highlander mode analysis is begining
224250 [main] DEBUG com.enioka.jqm.api.Dispatcher - JI seen by highlander: 457SUBMITTED
224251 [main] DEBUG com.enioka.jqm.api.Dispatcher - In the highlander if
224251 [main] DEBUG com.enioka.jqm.api.Dispatcher - Highlander mode will return: 457
224251 [main] DEBUG com.enioka.jqm.api.Dispatcher - JI won't actually be enqueued because a job in highlander mode is currently submitted: 457
224251 [main] DEBUG com.enioka.jqm.api.Dispatcher - BEGINING ENQUEUE
224251 [main] DEBUG com.enioka.jqm.api.Dispatcher - A new EM will be created
224257 [pool-175-thread-1] ERROR com.enioka.jqm.tools.Loader - Could not update internal elements
javax.persistence.RollbackException: Error while committing the transaction
at org.hibernate.ejb.TransactionImpl.commit(TransactionImpl.java:92)
at com.enioka.jqm.tools.Loader.run(Loader.java:444)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:701)
Caused by: javax.persistence.PersistenceException: org.hibernate.exception.LockAcquisitionException: could not execute statement
at org.hibernate.ejb.AbstractEntityManagerImpl.convert(AbstractEntityManagerImpl.java:1387)
at org.hibernate.ejb.AbstractEntityManagerImpl.convert(AbstractEntityManagerImpl.java:1310)
at org.hibernate.ejb.TransactionImpl.commit(TransactionImpl.java:80)
... 7 more
Caused by: org.hibernate.exception.LockAcquisitionException: could not execute statement
at org.hibernate.exception.internal.SQLExceptionTypeDelegate.convert(SQLExceptionTypeDelegate.java:94)
at org.hibernate.exception.internal.StandardSQLExceptionConverter.convert(StandardSQLExceptionConverter.java:49)
at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:125)
at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:110)
at org.hibernate.engine.jdbc.internal.ResultSetReturnImpl.executeUpdate(ResultSetReturnImpl.java:189)
at org.hibernate.engine.jdbc.batch.internal.NonBatchingBatch.addToBatch(NonBatchingBatch.java:58)
at org.hibernate.persister.entity.AbstractEntityPersister.update(AbstractEntityPersister.java:3236)
at org.hibernate.persister.entity.AbstractEntityPersister.updateOrInsert(AbstractEntityPersister.java:3138)
at org.hibernate.persister.entity.AbstractEntityPersister.update(AbstractEntityPersister.java:3468)
at org.hibernate.action.internal.EntityUpdateAction.execute(EntityUpdateAction.java:140)
at org.hibernate.engine.spi.ActionQueue.execute(ActionQueue.java:393)
at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:385)
at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:302)
at org.hibernate.event.internal.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:349)
at org.hibernate.event.internal.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:56)
at org.hibernate.internal.SessionImpl.flush(SessionImpl.java:1159)
at org.hibernate.internal.SessionImpl.managedFlush(SessionImpl.java:404)
at org.hibernate.engine.transaction.internal.jdbc.JdbcTransaction.beforeTransactionCommit(JdbcTransaction.java:101)
at org.hibernate.engine.transaction.spi.AbstractTransactionImpl.commit(AbstractTransactionImpl.java:175)
at org.hibernate.ejb.TransactionImpl.commit(TransactionImpl.java:75)
... 7 more
Caused by: java.sql.SQLTransactionRollbackException: transaction rollback: serialization failure
at org.hsqldb.jdbc.JDBCUtil.sqlException(Unknown Source)
at org.hsqldb.jdbc.JDBCUtil.sqlException(Unknown Source)
at org.hsqldb.jdbc.JDBCPreparedStatement.fetchResult(Unknown Source)
at org.hsqldb.jdbc.JDBCPreparedStatement.executeUpdate(Unknown Source)
at org.hibernate.engine.jdbc.internal.ResultSetReturnImpl.executeUpdate(ResultSetReturnImpl.java:186)
... 22 more
224259 [main] DEBUG com.enioka.jqm.api.Dispatcher - Job to enqueue is from JobDef 85
224260 [main] DEBUG com.enioka.jqm.api.Dispatcher - Highlander mode analysis is begining
224261 [main] DEBUG com.enioka.jqm.api.Dispatcher - JI seen by highlander: 457SUBMITTED
224262 [main] DEBUG com.enioka.jqm.api.Dispatcher - In the highlander if
224262 [main] DEBUG com.enioka.jqm.api.Dispatcher - Highlander mode will return: 457
224262 [main] DEBUG com.enioka.jqm.api.Dispatcher - JI won't actually be enqueued because a job in highlander mode is currently submitted: 457
224270 [main] DEBUG com.enioka.jqm.api.Dispatcher - BEGINING ENQUEUE
224270 [main] DEBUG com.enioka.jqm.api.Dispatcher - A new EM will be created
224271 [main] DEBUG com.enioka.jqm.api.Dispatcher - Job to enqueue is from JobDef 85
224272 [main] DEBUG com.enioka.jqm.api.Dispatcher - Highlander mode analysis is begining
224273 [main] DEBUG com.enioka.jqm.api.Dispatcher - JI seen by highlander: 457SUBMITTED
224274 [main] DEBUG com.enioka.jqm.api.Dispatcher - In the highlander if
224274 [main] DEBUG com.enioka.jqm.api.Dispatcher - Highlander mode will return: 457
224274 [main] DEBUG com.enioka.jqm.api.Dispatcher - JI won't actually be enqueued because a job in highlander mode is currently submitted: 457
224277 [main] DEBUG com.enioka.jqm.api.Dispatcher - BEGINING ENQUEUE
224277 [main] DEBUG com.enioka.jqm.api.Dispatcher - A new EM will be created
224285 [main] DEBUG com.enioka.jqm.api.Dispatcher - Job to enqueue is from JobDef 85
224285 [main] DEBUG com.enioka.jqm.api.Dispatcher - Highlander mode analysis is begining
224286 [main] DEBUG com.enioka.jqm.api.Dispatcher - JI seen by highlander: 457SUBMITTED
224286 [main] DEBUG com.enioka.jqm.api.Dispatcher - In the highlander if
224286 [main] DEBUG com.enioka.jqm.api.Dispatcher - Highlander mode will return: 457
224286 [main] DEBUG com.enioka.jqm.api.Dispatcher - JI won't actually be enqueued because a job in highlander mode is currently submitted: 457
234404 [main] DEBUG com.enioka.jqm.tools.JqmEngine - JQM engine has received a stop order
234404 [main] DEBUG com.enioka.jqm.tools.JqmEngine - Engine localhost asks queue VIPQueue to stop.
234404 [main] DEBUG com.enioka.jqm.tools.JqmEngine - Engine localhost asks queue NormalQueue to stop.
234404 [main] DEBUG com.enioka.jqm.tools.JqmEngine - Engine localhost asks queue SlowQueue to stop.
234407 [Thread-788] DEBUG com.enioka.jqm.tools.Polling - Poller loop on queue VIPQueue is stopping [engine localhost]
234407 [Thread-788] DEBUG com.enioka.jqm.tools.ThreadPool - A thread pool will now try to stop
234407 [Thread-788] DEBUG com.enioka.jqm.tools.ThreadPool - A thread pool has stopped properly
234407 [Thread-788] INFO com.enioka.jqm.tools.Polling - Poller on queue VIPQueue has ended
234435 [Thread-789] DEBUG com.enioka.jqm.tools.Polling - Poller loop on queue NormalQueue is stopping [engine localhost]
234435 [Thread-789] DEBUG com.enioka.jqm.tools.ThreadPool - A thread pool will now try to stop
234435 [Thread-789] DEBUG com.enioka.jqm.tools.ThreadPool - A thread pool has stopped properly
234435 [Thread-789] INFO com.enioka.jqm.tools.Polling - Poller on queue NormalQueue has ended
235343 [Thread-790] DEBUG com.enioka.jqm.tools.Polling - Poller loop on queue SlowQueue is stopping [engine localhost]
235344 [Thread-790] DEBUG com.enioka.jqm.tools.ThreadPool - A thread pool will now try to stop
235344 [Thread-790] DEBUG com.enioka.jqm.tools.ThreadPool - A thread pool has stopped properly
235344 [Thread-790] INFO com.enioka.jqm.tools.Polling - Poller on queue SlowQueue has ended
235344 [Thread-790] DEBUG com.enioka.jqm.tools.JqmEngine - Jetty will be stopped
235417 [Thread-790] INFO com.enioka.jqm.tools.JqmEngine - JQM engine has stopped
235417 [main] DEBUG com.enioka.jqm.tools.JqmEngine - Stop order was correctly handled
235417 [main] DEBUG com.enioka.jqm.tools.JqmEngine - JQM engine has received a stop order
235417 [main] DEBUG com.enioka.jqm.tools.JqmEngine - Engine localhost4 asks queue VIPQueue to stop.
235419 [Thread-801] DEBUG com.enioka.jqm.tools.Polling - Poller loop on queue VIPQueue is stopping [engine localhost4]
235420 [Thread-801] DEBUG com.enioka.jqm.tools.ThreadPool - A thread pool will now try to stop
235420 [Thread-801] DEBUG com.enioka.jqm.tools.ThreadPool - A thread pool has stopped properly
235420 [Thread-801] INFO com.enioka.jqm.tools.Polling - Poller on queue VIPQueue has ended
235420 [Thread-801] DEBUG com.enioka.jqm.tools.JqmEngine - Jetty will be stopped
235493 [main] DEBUG com.enioka.jqm.tools.JqmEngine - Stop order was correctly handled
235497 [Thread-801] INFO com.enioka.jqm.tools.JqmEngine - JQM engine has stopped

Clean running/crashed jobs on startup

Review log levels & messages

Many messages should be TRACE.

JQM securisation - master ticket

Mostly for getdeliverable, enqueue...

XML validation with IBM J9 virtual machine 2.4 is broken

Refactor mailing: should be static

Execution date should be just at payload start

It is currently at thread creation, which is very different (~1s) if this is the first time this type of payload is started.

Engine tests + test-em rely on a hardcoded model jar => breaks every release

Due to: model jar name contains version. Either fix the name or (better) find a way to make the reference dynamic.

Tomcat style JNDI support

pom.xml resolution: add full pom support, including properties interpolation

allow full pom resolution (including properties interpolation)

This requires changing jcabi for shrinkwrap resolver.

Add Windows and Linux service support

Create Github pages from doc

Allow to move logs directory

Engine API enqueue does not work

Version 1.1.4, Oracle or MySQL

Impossible to create the EMF at enqueue time from a payload.
Due to: creating an EMF in the payload classloader.
Fix is: use the existing EMF, don't create a new one. This requires an additional client API for creating a client.

Untangle History from JobInstance

Currently there is a total // between JO update and history update. This is stupid - the separation exists only for performance reasons (frequent updates/lookup on a small unindexed table, big history table insert-only).
The insert-only property of History should be restored.

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.