Field naming, biobank ETL

[gpbatra]

Field naming, biobank ETL

[carbocation]

Can you unpack this? I am not sure what this is asking.

[gpbatra]

Sure -- when we download the hesin table from the UKBiobank Showcase, the raw field name that comes out is 'eid', instead of 'sample_id'. The hesin.json (which defines the schema) also maps the field name to 'eid'.

I've got no problem if we want to rename that field as 'sample_id', but would be best if we did it there, directly in that hesin.json (and the other hesin.json files). Then, anytime we create a derived table, 'eid' would disappear.

In other words, make this change as far upstream as possible.

[carbocation]

Since hesin, hesin_icd10, etc are all direct copies of UK Biobank output without modification, I haven't been replacing eid with sample_id. I have only been doing that renaming step in derived tables. If we are going to change the naming of fields in the raw data, we will just need to review which tools will need to be updated to expect that rename.

[lucidtronix]

I think the current practice of keeping the raw imports as is and only doing the renaming in derived tables is working. If there are objections re-open this issue and let's discuss.