Should we switch this project to Poetry?

[mrpowers]

Should we switch this project to Poetry?

[MrPowers]

I pulled the trigger on this one and switched over. Poetry is the only package management system I know how to use in Python. It's working out alright so far ;)

[afranzi]

Hey @MrPowers, I have some teammates who tried in other projects and they are quite happy, so maybe it could be great to give a shot, however, do you really need it? What could it bring new to the project?

[MrPowers]

@afranzi - Great question. Here are the benefits I see:

• I want this repo to be a good example of what PySpark projects should looks like / the best practices that should be followed.
• Want to make it easier for other folks to clone this repo and install the exact same dependencies I'm using on my machine
• The current deploy process seems like a hack. Think Poetry will allow for a single poetry publish command.

Interested in your thoughts. Will plan on trying out Poetry in a branch to get additional feedback on some working code 🤓

[MrPowers]

@afranzi - Here's the WIP PR: #25

Let me know if you're able to run poetry install and successfully run all the tests with poetry run pytest tests!