tonyfujs/ddhqa

ddhqa's Issues

Check for zero-length files in NAS

field_exception_s_ considered recommended field

field_exception_s_ is only mandatory for datasets that are not public. Currently, empty field_exception_s_ is always flagged as an issue.

check_overall() should include url

Add a column to output dataframe with the nids' urls.

check_resource_link() beyond 200

Currently check_resource_link() only passes for 200 messages, will probably need to think of a more robust output depending on the status code.

example
node 140181 fails due to 400 status message, however, the resource link works fine

Check that official use/confidential datasets don't have public resources

check_privacy_status() missing conditional

Add if-statement in check_privacy_status() for the dataset's "field_wbddh_data_class". If its public, then check the resource level's data_class.

example

dataset_metadata <- ddhconnect::get_metadata(dataset_nid)
data_class_tid <- unlist(dataset_metadata[["field_wbddh_data_class"]], use.names = FALSE)
data_class_ui <- lovs[lovs$tid == data_class_tid & lovs$machine_name == "field_wbddh_data_class", ]$list_value_name

if (data_class_ui == "Public") {
  check_privacy_status(resource_metadata)
}

check_unpublished checking wrong field?

I think this function is not checking the correct field... It is checking that status = 1. This is the node status, not the publication status.

Sanity check data classification against its path

Public resources should be in ddhfiles/public/ and others should be in ddhfiles/internal/.

Check if next update date has passed

add new check

• compare last update date vs update frequency
• next update date vs the current date

get_field_format(): Lookup of tids returns wrong values

tid #17 return "pdf" when it should return "vector"

Check if embargo date has passed

add new check

check_resource_link(): Does not handle files on ESRI server

Files hosted on https://geowb.worldbank.org should not return an error. Add some logic to skip the check for these files.

check_recom_fields(): look into "field_wbddh_data_dates"

Data Collection Dates is a required Microdata field. The ddh form maps the "Data Collection Dates" form field to the field_wbddh_data_dates machine name on the add data form. Including this field as a recommended field leads to a lot of fails.

• data collection collection_date is mapped to startDate and endDate in mdlibtoddh
• startDate and endDate in mdlibtoddh's lookup are associated with field_wbddh_start_date and field_wbddh_end_date respectively
• field_wbddh_start_date and field_wbddh_end_date are not populated for some Microdata datasets (ex: 113249, 114196, 108513)

It might make sense to include field_wbddh_start_date and field_wbddh_end_date as recommended fields instead.

Check for non-public dataset with exceptions

Add check to see if field_exception_s_ is populated if the dataset is not public