Standardize MRN cleaning

[erikr]

Standardize MRN cleaning

[paolodi]

Agreed! Let's merge the explore PR and I'll follow-up with #251 quickly to standardize the plots

[erikr]

@paolodi is this something we should implement soon since our focus is heavily on QC these days? It would remove a degree of freedom from analyses.

[paolodi]

I feel that we still do not have enough information from the EHRs to justify some extensive re-cleaning (so far, only very few MRNs of interest seem to be clearly malformed). In addition, the approach you implemented in tensorization seems pretty good!

Did you observe any discrepancies in STS or Apollo we could build upon?

[erikr]

I agree that extensive re-cleaning is not worth the effort.

Rather, given:

In addition, the approach you implemented in tensorization seems pretty good!

I propose the MRN cleaning function we use in tensorization should be used throughout the codebase any time MRN cleaning is required. The purpose is to standardize.

[erikr]

@paolodi @StevieSong
The function used to clean MRNs in tensor_writer_partners.py is

https://github.com/broadinstitute/ml/blob/e8e69b8c9b5be0cbac8cb6017812c24d0728886b/ml4cvd/tensor_writer_partners.py#L80-L90

so within that script, MRN cleaning is standardized.

Do you know if MRNs are cleaned anywhere else in ml4cvd? If yes, _clean_mrn should be made into a public function and used.