Currently gin is set to run in Release mode all the time via main.go. That needs to be optional and left out for the user, inorder to debug in development.
The bug occurred in 10-04-2020.pdf since while parsing the pdf, there exists inconsistency in the 'date' text we are trying to extract for timestamp. It's given as '010/04/2020', where the 010 is erroneous for date. Manual string conversion to iso is problematic when day is not single digit.
The table data has very poorly laid out contents. Either skipping this particular date or manually adding the details from Annex1 is currently a quick fix since the website is not updating to latest data!
Small refactoring of the code would be a necessity i think. Currently since the scripts 'js' variable is populated using the json already existing in data/data.json, the arbitrary file cant have a filename which resembles(eg:- 04-05-2020.pdf or 27-03-2020.pdf) dates/timestamps existing as part of the data.json. That also needs to be fixed.
Currently important stats like number of deaths, number of discharged are missing from our response data since annex-2 table hasn't been extracted.
Extracting data from annex-2 table will be a challenge. But the data from it can help to create a timeline data from starting till latest dates with number of deaths per district, total number of deaths, number of people discharged per district, number of total discharged etc.
Currently the script works pretty well. But since the pdfs that we get from dhs kerala is very inconsistent, I had to use a lot of hacks to make the script work as intended. Also the pdftotext library used, is not very effective to all pdfs(say example pdf of date 02/12/2020). So what are your thoughts on refactoring the whole script? What other alternative libraries are there to extract tabular data from pdfs in a much better way? Also just manually adding the data for missing dates is the only solution(which destroys the purpose of this repo, but sitll)?