support for notebook tasks

[rafaelpierre]

support for notebook tasks

[aaravind100]

Hey @rafaelpierre, I was looking to add the spark python task support and there is some overlap between notebook and spark python task.

The default Task (pyjaws.pyjaws.api.base.Task) creates a python wheel task. Would it be better to have a class which models the various task types(notebook_task, spark_python_task, dbt_task etc) and generic class/function to create these different tasks in the workflow?

I understand in this quite early on, do you have something in mind?

[rafaelpierre]

Hey @aaravind100 - first of all thanks for chiming in :)

That's a great idea indeed. We could have a main Task class, and different child classes inhereting from it - e.g. PythonWheelTask, NotebookTask etc.

What do you think?

[aaravind100]

Great!! I can give it go :D

[rafaelpierre]

Awesome! Looking forward to your PR :)

[rafaelpierre]

Resolved with #12