Worker and Scheduler constantly restarting

[puckel]

Worker and Scheduler constantly restarting

[puckel]

Scheduler restart every 5 DAG run (ie sheduler command).

Did you deploy a new DAG ?

[ariksidney]

No I didn't deploy a new DAG. I tried to remove all the DAGs but the error still occurs.
I also discovered that if I go to the Graph view of a DAG, the Task is marked as 'no Status' but if I go to the DAG runs I can see that the Status is set to running (but it appears that it's not getting executed).

Thanks in advance

[ariksidney]

Ok I think I found the issue. The problem was that after the last build Airflow took Celery 4.0.0 which caused the above error. I switched back to Celery 3.1.23 by changeing the airflow_base Dockerfile like this:

RUN    pip install airflow[postgresql,hive,password]==$AIRFLOW_VERSION \
    && pip install celery==3.1.23

[puckel]

Thanks for reporting this issue. I fixed Celery version in the release 1.7.1.3-4