Fix new_data output to remove unnamed columns (index=False)

[alphagov]

Fix new_data output to remove unnamed columns (index=False)

[ellieking17]

Also there is no text data in combined text from the old_taxons in new_content.csv. This needs fixing.

[ivyleavedtoadflax]

Has this been completed now @ellieking17?

[ellieking17]

not yet. We have a trello card for it

[ellieking17]

this turns out to be problem in the pipeline, where in some scripts, files are being written out as zipped and the folloiwng script being written in as csv. The makefile also is feeding unzipped files between scripts. @ff-l is working on standardising the pipeline to zipped file types as these are being read by notebooks too.

[ff-l]

@ellieking17 I've set up the scripts so that all outputs are compressed (gzip), final thing to change would be the output from the python crawl pipeline (to get gzipped json files).

[ellieking17]

this is now fixed.