ohdsi/ares

A Research Exploration System

ARES

ARES is A Research Exploration System designed to improved the transparency of observational data research. ARES is an opinionated framework that delineates three levels of observational data assessment.

Network

The network level includes all data sources contributing information for a particular research agenda or community.

Data Source

The data source level refers to a data source that originates from an individual vendor, provider, payer or other data aggregator.

Release

The release level refers to a specific release of a data source. Releases are identified by the combination of attributes that makes them unique. These include data source version, vocabulary version, OMOP CDM version, and ETL version.

Features

ARES provides functionality to address the following research topics across all three levels of observational data assessment.

• Data Characterization
• Data Quality Assessment

ares's Issues

clear table / return to loading status on cards when toggling data source selection

for the following reports

• data source release / unmapped source codes
• data source release / conditions or drug exposures or any other domain

when the user changes data source there is a delay while the data loads before the table updates with data. if this delay is longer it can be confusing that the data hasn't changed yet.

please clear the table on the change of data source and reset the data loading flag so the loading indicator is displayed until the new data is completed loading.

implement database comparison report

standardize location of legend to right of chart

there are some charts on the measurement report (and potentially others) where the legend is listed at the top of the chart. we need to standardize the location of the legend to be on the right of all charts.

visualize data missingness patterns across network

pending analysis of missingness in DQD and then at the network level.

Sort the Observation periods per person report by % of people

Describe the solution you'd like
The report that lists the number of observation periods per person is sorted by number of observation periods rather than by % of people, making it difficult to read.

Describe alternatives you've considered
Either changing the default sort or allowing the user to sort on the columns would be helpful

Additional context

ESLint and Prettier

It would be a good idea to configure eslint and prettier to simplify code formatting and make all contributors adhere to certain code style rules.

Data Source Data Quality History - Right align titles

Right align the the titles, "# Passed", "# Failed" and "# Total" on the Data Source > Data Quality History report so that they are aligned with the values that they head. Becomes difficult to read with wide screens.

add payor reporting

add link from data quality history to specific release report

ares/#/_datasource/<SOURCE_KEY>/quality-history

The table on the quality-history report should provide links to the specific release data quality report.

/ares/#/_cdm/<SOURCE_KEY>/20211015/quality?tab=overview

Additionally there should be links to the result failures directly from the fail count on the initial report similar to:

/ares/#/_cdm/<SOURCE_KEY>/20211015/quality?tab=results&domainFailFilter=condition_occurrence

But without the domain specific fail filter, just a fail filter instead.

write setup documentation / articles

add scan report visualization to ares

Scan Reports are generated using the Rabbit in a Hat tool. This will provide a report on the contents of the native database before ETL processing. Will require additions to the AresIndexer.

Add Plausible Value Labels to measurement reports

In DQD, there are biologically plausible values given for measurement/unit pairs. It would be nice to overlay these values on the ARES measurement reports.

Observation Period - New Visualization

Add Achilles Analysis 113 to the observation period report in ARES visualization. Conveying how many observation periods individuals within the data source have at a macro level.

improve observation period report

upgrade the Observation Periods per Person table to use the vuetify datatable component. set default sort to the % of People column, descending. Limit the number of rows by default to 5.

integrate min/max p10/p90 functionality

There is an existing report in which I added a feature to toggle between two value ranges found here:

network/concept/measurement/3016723/summary

You get to this report by clicking on the 'Check measurement value distributions across the network' link on this report:

/cdm/DATA_SOURCE/20211023/measurement/3016723

The reason this feature was implemented was because often the min/max have very large, possibly junk data causing a view like this:

Changing to the p10/p90 range provides a better view of the data excluding the extreme ranges:

Please copy this functionality to the concept measurement report plot and default to the p10/p90.

Whisker plot layout error.

Height of whisker plots appear to be fixed. Jumbles text on y-axis when there are too many values.

explorer throws error when navigating directly to report

It looks like the explorer is trying to access a value that is not yet loaded when navigating directly to a report, or refreshing the page instead of navigating from the start of the application.

chunk-vendors.83c2dbc4.js:6303 TypeError: Cannot read properties of undefined (reading 'releases')
at o.getReleases (Explorer.vue:316)
at rr.get (chunk-vendors.83c2dbc4.js:8642)
at rr.evaluate (chunk-vendors.83c2dbc4.js:8744)
at o.getReleases (chunk-vendors.83c2dbc4.js:8942)
at o.getSelectedRelease (Explorer.vue:319)
at rr.get (chunk-vendors.83c2dbc4.js:8642)
at rr.evaluate (chunk-vendors.83c2dbc4.js:8744)
at o.getSelectedRelease (chunk-vendors.83c2dbc4.js:8942)
at o.va (Explorer.vue?d662:3)

Caching issue - DQD Failures

When navigating through Data Source Release Reports there are icons in the top right corner that identify how many DQD failures were related to the domain report you are reviewing. When navigating from a report that has failures to a report that does NOT have failures, the alert icon in the top right does not refresh.

Example: If the condition report has 11 errors and you then navigate to the condition eras report (and that report has 0 errors), the screen will continue to show 11 errors. If you refresh the page the alert disappears.

Transparency in Analyses

For each ARES visualization, there should be a way to identify which Achilles analysis it corresponds to. In trying to further clarify questions, this becomes quite important and is incredibly difficult to traceback (ARES > AresIndexer > Achilles).

Utilize VueX

As most components share a lot of similar state values it would be a good idea to utilize global store so that the app could become more modular. For example, the error component is currently used in most of components and is loaded if the local componentFailed value equals "true". The problem is that we're forced to use the Error component in each of components that could fail to load data.

A better approach would be to make this value global, create an additional wrapper component that will display the loaded data or the required component according to the global componentFailed value.

Visit Occurrence Report - Table Order

The data source release visit occurrence report has a table that is not ordered in any specific sequence. Data should be order in descending order by '% People'.

Data Source Release Comparison

When attempting to 'Review this Time-Series across data source releases', the application hangs if the specific concept_id a person is looking for does not exist in all of the data source release directories. A 404 error is created. Concept 37311061 is a good example.

update unmapped source codes report

add commas to the number formatting of the # records column

package updates

Consider updates to packages:

npm WARN deprecated [email protected]: This loader has been deprecated. Please use eslint-webpack-plugin
npm WARN deprecated [email protected]: this library is no longer supported
npm WARN deprecated @hapi/[email protected]: This version has been deprecated and is no longer supported or maintained
npm WARN deprecated @hapi/[email protected]: This version has been deprecated and is no longer supported or maintained
npm WARN deprecated [email protected]: Please see https://github.com/lydell/urix#deprecated
npm WARN deprecated @types/[email protected]: This is a stub types definition. fast-json-stable-stringify provides its own type definitions, so you do not need this installed.
npm WARN deprecated [email protected]: https://github.com/lydell/resolve-url#deprecated
npm WARN deprecated [email protected]: 3.x is no longer supported
npm WARN deprecated [email protected]: Chokidar 2 will break on node v14+. Upgrade to chokidar 3 with 15x less dependencies.
npm WARN deprecated [email protected]: Chokidar 2 will break on node v14+. Upgrade to chokidar 3 with 15x less dependencies.
npm WARN deprecated @hapi/[email protected]: Moved to 'npm install @sideway/address'
npm WARN deprecated [email protected]: The querystring API is considered Legacy. new code should use the URLSearchParams API instead.
npm WARN deprecated [email protected]: Please upgrade to version 7 or higher. Older versions may use Math.random() in certain circumstances, which is known to be problematic. See https://v8.dev/blog/math-random for details.
npm WARN deprecated [email protected]: Please upgrade to version 7 or higher. Older versions may use Math.random() in certain circumstances, which is known to be problematic. See https://v8.dev/blog/math-random for details.
npm WARN deprecated [email protected]: Please upgrade to version 7 or higher. Older versions may use Math.random() in certain circumstances, which is known to be problematic. See https://v8.dev/blog/math-random for details.
npm WARN deprecated [email protected]: request has been deprecated, see request/request#3142
npm WARN deprecated [email protected]: this package is now deprecated
npm WARN deprecated [email protected]: babel-eslint is now @babel/eslint-parser. This package will no longer receive updates.
npm WARN deprecated @hapi/[email protected]: This version has been deprecated and is no longer supported or maintained
npm WARN deprecated @hapi/[email protected]: Switch to 'npm install joi'
npm WARN deprecated [email protected]: This SVGO version is no longer supported. Upgrade to v2.x.x.
npm WARN deprecated [email protected]: You can find the new Popper v2 at @popperjs/core, this package is dedicated to the legacy v1

Logic and configs separation

At the moment project's components include markup, logic, configs and styles which is hard to read and maintain. It would be a good idea to at least separate Vega configs and routes into separate files.

add tooltip and resize to measurement record count proportion chart

the current record count proportion by age,sex and year report does not have tooltips. Tooltip should show year, age decile and proportion per 1000 people by sex.

Tooltip could use the nearest operator from vegalite or the line approach showing both series values. (example code: https://vega.github.io/vega-lite-v3/docs/nearest.html)

add type stratification to domain level report

Will need to be added to Achilles export package, a group by on the current type stratification that includes concept_id where it should not for the domain level characterization

add step height scaling to Years of Observation by Age on observation period report

ARES Visual Descriptions and clarification

There's a number of graph's that look at the first occurrence or the first observation of something and then visualize that but it's not always clear within ARES. Each ARES visualization should have a small description about what it is actually visualizing.

charts do not resize dynamically on horizontal resize

ares/#/_network/dataquality

data source: data quality history report

Add interactivity to legend and tooltip to charts.

network population report: update legend to right side display, add layer tooltip functionality

The network population overview report has the legend on the bottom, please switch to right side display.

/#/network/population

Please include legend interactivity for focusing on an individual series as implemented on the data density report.

Please add layer tooltip to show the value for each series as suggested here:

and mentioned in #39

add age at first observation distribution to observation period and/or person report

achilles_results_dist where analysis_id=103;

add the box and whisker plot below existing histogram either as a layered image or separate image

Changing data source does not refresh report

Expected Result

When viewing a report (such as drug era) you should be able to switch data source to review different data source results.

Actual Results

Switching the data source does not seem to refresh the report to reflect the newly selected source.

Steps to Reproduce

1. Pick Drug Era report for one data source
2. Search for specific ingredient (ex: ibrutinib)
3. Select ingredient
4. Choose a different data source from top nav.
5. Observe: report does not change to reflect new source.

Data Source Switching

When a user switches to the 'Data Source Release' category after reviewing the reports under the 'Data Source' Report Category, the Data Source is changes and/or defaults to a difference source. In our case, it's always defaulting back to IBM CCAE for the Data Source under 'Data Source Releases'.

Please reach out for a live demo of this issue.

update vega specifications

The input spec uses Vega-Lite v4, but the current version of Vega-Lite is v5.1.1.

improve data source release data density plots

For both plots on report:

• Add dot marks to the series to aid in identifying individual data points.

Improve tooltip:

• add the x axis value of the date of the data point to the tooltip.
• add comma formatting to the value of number of records
• add label to each data point in tooltip (top plot: # Records / Date) (bottom plot: Records per Person / Date)

Update X and Y axis labels to capitalize first letter (Records, Date)

configure build / yml git workflow

Observation over time lacking year markers

The current display is lacking information on the year which makes this chart hard to interpret unless you mouseover the data points:

improve titles of tooltips on box plots

add support for provider specialty drilldown report

With the addition of the provider report this is next on the list.

Error on folder change

Describe the bug
"TypeError: this.getSelectedRelease is undefined" thrown trying to change the folder from the data source category after a page reload

To Reproduce
Steps to reproduce the behavior:

1. Choose the "Data source" category
2. Reload the page
3. Try changing to any other category
4. The page will not update, an error will be produced in the console

Expected behavior
The folder should change, the page should update accordingly

add network feasibility report

The network feasibility report will be at the Network level and perform a real-time assessment of the data sources in the network and their feasibility for particular research purpose. A more in depth discussion of this topic can be found here:

OHDSI/CohortDiagnostics#480

This report will all a user to interactively choose selection criteria which will then filter across the network results to determine estimates of the number of people in each data source that could potentially be used in a research study. The tool will only consider the latest release of each data source.

The user should be able to input the following criteria:

• which domain their study requires data from (show with switches in the ui mockup below)
• age range (shown in mockup)
• observation year (shown in mockup)
• Inpatient vs Outpatient ( supported by https://github.com/OHDSI/Achilles/blob/master/inst/sql/sql_server/analyses/201.sql exported in ares as domain-summary-visit_occurrence.csv)

The user should also be able to enter a concept identifier to add to the list of requirements. Entering the concept id will retrieve the percent of people with at least one record as provided in the individual concept report along with a count (if available) of data quality issues and whether or not there are temporal issues with the concept.

The final estimate provided by the tool will be a list of data sources and the estimated population that is available. The estimation calculation will be:

(Total People in Data Source) * (Percent with data overlap in required domains) * (Percent in Age Range) * (Percent in Observed Years) * (Percent with Required Concepts)

Acknowledging that this is not the same as a query that identifies the precise number of people sharing all these characteristics (which is possible with cohort definition processing) this will provide the only real time estimate available given the available precomputed statistics.

An initial prototype that was developed is represented in this screenshot:

@Mikhail-iontsev - we can discuss during our touch point tomorrow if you have additional questions
@clairblacketer - fyi if you would like to add anything to the approach.

records per unit chart fails to render with high number of values in data

20211023/concept/measurement/4008265/summary

add back to table button for drilldown reports

Now that the data source drop down correctly refreshes a report on the drill down report (for example, measurement reports) the browser's back button will take you back to the prior data source. We need a button that takes you back to the overview table report. For example, from /measurement/2212648 to /measurement. This should apply to all domains based on how the components are used.

expand markdown help system and integrate mermaid for etl

table search speed

Reports with the data table search filter such as:

_cdm/DATA_SOURCE/RELEASE_KEY/domain/measurement/summary

have slow performance, response on key press. In the past I've used javascript libraries, I think one called patience, that would wait for typing to stop before initiating the search instead of search with each keystroke which gave a better overall experience.

update metadata report to use data table

Use the standard data table component to display the information in the Metadata card section that is loaded from metadata.csv.

Load the metadata content into the vuex state as metadata will be integrated into a number of reports in the future.

Record Count Proportion by Age, Sex, and Year

Age Deciles don't order properly

Standardize charts functionality

At the moment charts' functionality is unpredictable: some of them provide a ruler, tooltip and interactive legend, some don't have anything at all, though the chart type is the same. This leads to inconsistent user experience.

We need to provide all charts (if possible) with the same functionality.