Add unit tests

[guidopetri]

Add unit tests

[guidopetri]

The big question here is: how do you do unit tests on a Luigi Task? In the past, I offloaded the actual processing into a method in the Task class, then ran that on some dummy input. However, this seems a little silly and artificial. Could we come up with tests that work directly on a Task's run method?

[guidopetri]

Something to look into: Great Expectations, a sort of testing library for your data, apparently.

[guidopetri]

Looks like Great Expectations was not really what I was expecting. Haha. I guess what I can do is set up some pytest type framework and just init the Luigi Tasks and then pass custom input into them, then see what their output is. I'd have to make sure the output comes back inside the test, though, and not as a file - or maybe I could actually output it as files and then just verify inside the test by loading the file and seeing what the output is? I'm not exactly sure how this would work so it'll take some experimenting.

[guidopetri]

Looks like the best way to do this is by offloading "business logic" into functions that are called by luigi.Tasks, and then test those with a small data sample (probably via pytest?).

I'll keep a list of things I need to offload here, as well as a list of tests to write.

[guidopetri]

Functions to write:

• parse headers fade7ac
• provisional rating manip ca50eaa
• sf evaluation filter not sure that this is needed
• clock time manip 8ce3534
• FEN manip d7df467
• all the GetGameInfos dataframe manipulation 32b3861
• newsletter SQL cfc15e8
• newsletter tables 20392b0 , f451bef
• newsletter plots (test the fact that they worked? maybe md5 of saved image?) cdbac9f , df86077

[guidopetri]

Tests to write:

• parse headers 965face
• provisional rating manip bbbb1b3
• clock time manip 272f254
• FEN manip 8223b37
• all the GetGameInfos dataframe manipulation 05b5fee
• newsletter SQL c5bce7a
• newsletter tables 42bb4a8 , eac7e7f
• newsletter plots (test the fact that they worked? maybe md5 of saved image?) 240b9dd , 8268527
• visitors 00f5993
• column queries (mock with a .sqlite somehow?) skipping this for now
• stockfish evaluation (must fix to a specific version of stockfish) 2b358c9 (testing on stockfish 10 working)
• sf evaluation filter

[guidopetri]

Got most of the tests written that I wanted to. I'm not sure what I meant by "sf evaluation filter" but at some point I'm sure it will come up.

I'm skipping the column query since it would require a more extensive rewrite and would probably require mocking the test with a .sqlite, which is a little bit more complicated than I'm willing to do right now, especially for such a simple SQL query.

Next step is setting up TravisCI or something similar, I guess.