Research querying Jaime's API and put forward recommendations on how to carry out the required filtering

[yalla-coop]

Research querying Jaime's API and put forward recommendations on how to carry out the required filtering

[thejoefriel]

@mr-bagglesworth as discussed please use this issue to:

• put together recommendations on how to filter the data following our criteria
• build filtering query functions that we can use in the app
• highlight any issues we need to quickly address with @sesola / Jamie

[martinbagshaw]

Results.js

• Enters default search terms into the api, which can be overwritten by details entered into the form in BasicInfo.js
• default search terms are postcode of B152TH, gender of m, age of 70.

[thejoefriel]

Hi @mr-bagglesworth - in this issue you can see how we're expecting the data from the medical record to look: #4 which can inform on what we're then looking to query on Jamie's API.

As you've mentioned before currently in Jamie's API we would be looking for the following criteria to carry out filtering :

• Age - minAge / maxAge
• Gender
• Type of cancer (I believe this is stored in the conditions)
• Status - we want to filter out any trials that are completed (we need to think whether there's a consistent term/keywords to look out for)

To confirm, we won't filter out based on location, except to rule out non-UK trials

In addition, Jamie is modifying the API so we should also have:

• ECOG status
• Gleason score
• Disease within prostate
• Disease outside prostate

We need more clarification though as to how this would look (i.e. will it be a key value pair of Disease within prostate: ineligible ) . We are waiting for @sesola to confirm this with Jamie.

[martinbagshaw]

This PR will be the source of all query functions. It contains plenty of my musings and notes on the query functions and what we may want to filter out and search by. It filters the JSON copied over from the previous API, and savedhere(on Susan's Github).

Up until this morning (29 / 9 / 2019) I have filtered the top level properties of the above object, which contains info such as gender, age, and conditions using reduce, which for those who haven't used this array method much before, can be treated as a filter within a map - a more efficient way of running .filter then .map, which will loop over the data twice before returning a result.

So far, I have filtered out results with invalid values by writing a validation object that gets referenced from the filter function. The type of search conducted - e.g. 'Conditions' sometimes has the value undefined, which will break the filter function. This validation filters out such results.

Moving forward

• Locations property contains an array of objects that need to be filtered. After speaking to @thejoefriel we want to include just UK locations for now. A Location is not provided for the overall trial - just sites where the trial is conducted, which can be in multiple countries. So I guess we want to filter out all non-uk results, for now at least?
• The Trial result has a property OverallStatus, which says whether the trial is [recruiting, not recruiting, finished, etc]. Each location also has a property, Status which provides us with the same info. This information is not always consistent with the overall trial status. Do we want to filter trial status on a per-location basis, or a per-trial basis? See screenshot:

- Some results come back without any contact details, and others come back without a `Name` (the hospital / trial location) that is easy for anyone to even look up, for example, "GSK Investigational Site". **Do we want to filter out locations without contact details, or a generic name?** See screenshots of useful and non-useful results:

Useful:

Not Useful:

[thejoefriel]

This is great work @mr-bagglesworth - thanks for laying it out.

The Trial result has a property OverallStatus, which says whether the trial is recruiting, not recruiting, finished, etc. Each location also has a property, Status which provides us with the same info. This information is not always consistent with the overall trial status. Do we want to filter trial status on a per-location basis, or a per-trial basis?

What you recommend is the simplest? Perhaps we just take the overall status as that's the data that is most consistently in there?

• Yes we want to filter out all non-UK results

• Yes if they have no name AND no contact details then we should exclude them, but I think we should keep them in if we at least have one of these things?

[thejoefriel]

By the way @mr-bagglesworth - going forward it's important this week that the rest of the team can start working on this. I think @MohammedYehia will be working on it as he's looking at the clinical trials rendering at the moment.

Has the latest work been pushed up (as of 1st Oct) ?

What is the best way for this to proceed @MohammedYehia @mr-bagglesworth ?

[martinbagshaw]

@joe I pushed up my latest commit last night. Regarding recruiting status, I think we need to search by locations within each trial - as you can see from the example above an invalid overall status may contain locations that are recruiting, which we shouldn't miss.

My next steps will be:

• add some tests
• make a recruiting status function that takes into account each location. If none of a trials locations are recruiting perhaps use the overall status of the trial.

The whole data processing cycle may look like:
1 - filter out non uk results
2 - filter out non recruiting locations
3 - run user defined filter functions

[martinbagshaw]

A further thought:

• perhaps we need to detail both location recruiting status and overall trial recruiting status in the ui
• the aim of the above being that potential trial matches do not get overlooked due to recruiting status

@thejoefriel @MohammedYehia @susanX @wright1 @othman-shamla
Please discuss

[MohammedYehia]

I agree with @mr-bagglesworth we should display it on the UI
so we are filtering for location recruiting status within the UK first and then displaying both the location status and the overall status

[MohammedYehia]

@thejoefriel
after reviewing the result array I found that maybe this process will work for us

1. Filter the API result by OverallStatus by these conditions [recruiting, unknow status]
2. Save the object as a reference for the rest of the filtering
3. filter for [age, gender, phase, conditions(cancer type), ECOG..]
4. include [only UK and Location status as recruiting]
5. exclude data that doesn't contain the name AND the contact detail

the problem now with the Nearly Eligible Trials how to decide this?
for example, age is older by 1 and Gleason bigger by 1 or what are the criteria for this thing?

[thejoefriel]

Hi @MohammedYehia - we need @sesola to get back to us on thresholds for nearly eligible trials. For now let's go with the following:

• Age - either 5 years older or younger (so if eligibility is 18+ and our patient is 14 then they would be eligible)

• ECOG status - it can be one higher (e.g. if eligibility is 0-2 and the patient has ecog 3, then they would be eligible)

Also to confirm for ECOG score I believe the patient is eligible as long as their score is less than the maximum number found

That is all we will do now for nearly matches until @sesola provides us further guidance. @sesola if you could get back to us on this today/tomorrow that would be hugely appreciated.

[thejoefriel]

Please also refer to #81 which shows what the additional criteria will look like in the clinical trials JSON data sets

[MohammedYehia]

Is there going to be any filtering on the backend API? or is he just returning the whole JSON and all the filter is going to be on the frontend side this way all that the find clinical trials button doing is sending a request to get the whole data from Jamie's API then we are doing the filter is this right?