moaraj/geowizard

Automated Query, Retrieval and Analysis of GEO Datasets

GeoWizard

GeoWizard is an Shiny App that automates the search, retrieval and analysis of publicly available expression data available on the GEOdatabase. The following page outlines the steps required to analysis a given expression dataset and export the results.

1. Querying Datasets from GEO
2. Assigning Samples to Experimental Blocks
3. Downloading, Filtering and Visualizing Raw data
4. Normalizing Data and Performing Differential Expression Analysis
5. Exporting Reports and Data Tables

Querying Datasets

Parsing Samples into Experimental Groups

Processing of the annotations of the expression data and automatically detect experimental Groups

Generating the Design and Contrast Matrix

Model Generation

Downloading Data and QC

BioQC

PCA

Differential Expression Analysis

Export

geowizard's Issues

Make a Mosaic Plot with Design Matrix

[ ] - Determine how to structure the data
[ ] - Produce the plot

Matrix Selection Panel for Design Matrix Page

Boxplot Page loads very slowly

1. Trying to sample only 100 genes

Download Data from GEO

Download Page

• Download Factor DF
• Download Filtered Factor DF
• Download GCT
• Download Filtered GCT
• Download ESET Object
• Download Annotations

Design Matrix Page

• Add options for Seleting Dataset
• View factor table
• Formula Input
• Generate Design Matrix button
• Select Control Level Drop Downs
• Detected Experimental Design
• Use Detected Design

GetUsefulVars Does't removes factors that aren't classified

Design Matrix Page

• Add options for Seleting Dataset
• View factor table
• Formula Input
• Generate Design Matrix button
• Select Control Level Drop Downs
• Detected Experimental Design
• Use Detected Design

Upload CSV

• Button to Upload CSV
• Upload Options
• Row and Column Number requirements

Get Bookr Markdown to run Main.Test

Display All Experimental Variable Doesn't Work

Display All Experimental Variable Doesn't show the full list of Vars in the DesignDataTable

Formula Input Autocomplete

https://github.com/AnalytixWare/ShinySky

Download and QC Page

• Plan what is needed
• setup UI

Raw-data Tabset

• Download Dataset from GEO
• GMT Tab
• Signal Profile
• Sample and Block Plots

QC Tabset

• BioQC analysis
• PCA Plot
• Hist
• Box

GSE99293 Factor Detection

The Dected factors in characteristics in GSEGSM table not being translated to final Factor DF

Incomplete Table:

Fix Executing Scheduling Issues

change formula input with factor names

Remove a single row from Factor DF

NGS Approaches

ARCH4 Database - Download of NGS if from Illumina machine
otherwise - a pipeline from FastQ, maybe have some email interface

Getting GeoQuery to Work Again

Wierd URL is constructed when I give a GEO accession number

• The issue arises from R 3.3 that is only available on the R Studio installed on our computers
• Using BioClite installer to see if i can get it to work

Comma AND Semicolons used as sep causing overflow in FactorDF

Current fix
comment out the following lines in GeoParse.R

ExpVarColsDF['CatVarText'] <-  apply(X = ExpVarColsDF['CatVarText'], MARGIN = 1, function(text) {
gsub(pattern = ",", replacement = ";", x = text)})

Useful Variables removed Vars that shouldn't be removed

Remove Genes that don't have a gene symbol in the matrix, don't just keep them as wierd lookiing variables

Replace Ggplot with Base plotting

http://nathanieldphillips.com/2016/10/the-yarrr-package-is-now-on-cran/
or base

"Analyse Selected Dataset" button doesn't Go to next page

The Tabset does not update to the GSM Info page
once a dataset has been selected and the button is clicked

Speedup SQLite DB search using Table Join

Instead of opening a connection and using lapply to query the list of GSE against the MetaDB
It is probably faster to make a table with all of the GSE and use a right join to populate the rest of the data

Horizontal Scrolling on Design Matrix

Factor Selection should not have option for GSM if plotting by factor

Filter GSM Metadata page not working

Contrast Inputs

• rename Design Matrix
• Control factor level inputs for contrast
• rearrange matrix according to factor level inputs
• If no control factor level make all contrasts
• Generate Matrix
• Interaction Term Render UI
• Interaction Term Gernation Matrix
• Make Contrast Matrix object for input into Pipeline

Volcano Plot Inputs don't work

Volcano Plot P-value and Fold Changes with weird values
and only the right half of the graph is being filtered meaning negative foldchanges filter above threshold isn't working - maybe use abs functions instead of two > and <

Random Gene Sampling in Boxplot is confusing

GSE input

Disable Filter Page until one Study selected

Error Using Req in a recursive loop

Retreiving Datatable Data
Filtering SQL Query Res for Selected GSE
Making SQL Summary Table
Warning: Error in eval: 'nthcdr' needs a list to CDR down
Stack trace (innermost first):
84:
83: stop
82: ExperimentalDesign$ExpFactorDF
81: renderUI [/pstore/home/hasanm7/GeoWizard/ShinyApp-Dashboard/server.R#417]
80: func
79: origRenderFunc
78: output$PickFactorColumns
1: runApp
Retreiving Datatable Data
Filtering SQL Query Res for Selected GSE
Making SQL Summary Table

Fix Contrast Names

When Interaction term is used contrast matrix is incorrect
Zero Model needs to omit contrast

• Write if 0 model function
• remove first column from design matrix

Review Shiny Objects

Add helptext popups

Recently identified bugs

Hi Mo, recently I tried a dataset with GeoWizard (GSE18269). It worked by and large well, though I found two bugs:

1. In the download page, the downloaded signal file does not contain row names (e.g. feature names).
2. It was unfortunately not possible to download the RData file.

And thanks to this particular dataset, I realised the need to edit the design matrix sometimes manually based on the semi-automatic procedure of constructing them, say removing a column. Do you envision incorporating this?

In case of questions, please kindly let me know!

Use Tabsets instead of an option on a separate pane to determine the full and classified factor tables

Text input not functional needs to be rendered

Add Cell Line, Tissue and Disease Ontology to Molecule Libraries search

• Cell Line
• Tissue
• Disease Ontology
• add to classficiation

Plot TabSet

• Volcano Plot
• Clustering
• PCA
• BoxPlot

GSE is not Parsed Properly

• Not Found In DB
• GSE57208
• GSE43956
• GSE43957
• GSE65646
• GSE76950
• GSE94884
• GSE102953
• GSE57251
• GSE87669
• GSE76951
• GSE49426
• GSE11440
• GSE16082
• GSE16085
• GSE16089
• GSE16066
• GSE16080
• GSE16070
• GSE74058
• GSE55688
• GSE99357
• GSE75727
• [ ]
• Multiple Copies
• GSE57800
• GSE57800
• GSE57800
• [ ]
• GSE57822
• GSE57822
• GSE57822
• GSE57822
• GSE57822
• [ ]
• GSE27211
• GSE27211
• GSE27211
• [ ]
• invalid regular expression '
• GSE2162

Rename ExpVars

Cannot Rename Factors in Summary of Factors Table

Classification Results

Error When you Switch GSE

Processing SQL Table Output
Filtering SQL Query Res for Selected GSE
Classify the Summary and Return the Filtered GSE GSM DF
executing ClassSummary
Determining if study has a time series
This gene series does not contain a time series experiment
Warning: Error in [<-.data.frame: replacement has 1 row, data has 0
Stack trace (innermost first):
109: [<-.data.frame
108: [<- [/GeoWizard//GSMAnnotation.R#281]
107: AssignTimeSeriesInformation [/GeoWizard//GSMAnnotation.R#281]
106: ClassSummary [~/GeoWizard//GSMAnnotation.R#110]
105: reactive:ExperimentalDesign$ExpClassFactorDF [/pstore/home/hasanm7/GeoWizard/ShinyApp-Dashboard/server.R#375]
94: ExperimentalDesign$ExpClassFactorDF
93: reactive:ExperimentalDesign$ExpFactorDF [/pstore/home/hasanm7/GeoWizard/ShinyApp-Dashboard/server.R#385]
82: ExperimentalDesign$ExpFactorDF
81: renderUI [/pstore/home/hasanm7/GeoWizard/ShinyApp-Dashboard/server.R#416]
80: func
79: origRenderFunc
78: output$PickFactorColumns
1: runApp
Retreiving Datatable Data
Making SQL Summary Table
filtering DataTable with Default ExpVarSelection
Warning: Error in [<-.data.frame: replacement has 1 row, data has 0
Stack trace (innermost first):
99:
98: stop
97: ExperimentalDesign$ExpFactorDF
96: reactive:ExperimentalDesign$FactorDF [/pstore/home/hasanm7/GeoWizard/ShinyApp-Dashboard/server.R#440]
85: ExperimentalDesign$FactorDF
84: exprFunc [/pstore/home/hasanm7/GeoWizard/ShinyApp-Dashboard/server.R#532]
83: widgetFunc
82: func
81: origRenderFunc
80: renderFunc
79: origRenderFunc
78: output$FullFactorTable
1: runApp
Rendering Selectize Inputs for filtering factor levels
Warning: Error in [<-.data.frame: replacement has 1 row, data has 0
Stack trace (innermost first):
84:
83: stop
82: ExperimentalDesign$FactorDF
81: renderUI [/pstore/home/hasanm7/GeoWizard/ShinyApp-Dashboard/server.R#449]
80: func
79: origRenderFunc
78: output$FilterGSMbyFactor
1: runApp
Processing SQL Table Output
Filtering SQL Query Res for Selected GSE
Classify the Summary and Return the Filtered GSE GSM DF
executing ClassSummary
Determining if study has a time series
This gene series does not contain a time series experiment
Expands Characteristics Column
Expanding Columns with Factor Conataining Text
Columns containing Factors are gsm.title and characteristics_ch1
Importing ClassListDF
Fetching ClassDFList
executing ClassGSMText
Stemming Factor Text for Text Classification
Executing DescerningClassDF
Finding Useful Default ExpVars
No Useful Factor Classifications
Adding Time Factors
Adding Titration Series
Output Default ExpVarSelection
Rendering checkbox input for selecting which Factors to use
Rendering Selectize Inputs for filtering factor levels
filtering DataTable with Default ExpVarSelection
Retreiving Datatable Data
Making SQL Summary Table

Set Up UI

Extracting Gene Symbol feature annotations

[ ] - Broken GetGEO
[ ] - GeneSymbol annotation needs to be moved upstream of extracting count matrix
[ ] - all Feature annotation objects need to be optional
[ ] - there should be an object at beginning of the load that ONLY contains count matrix then the annotation can occur downstream

When GSE has more than one Matrix

• Determine GLP of Matrix
• Add option for selecting Matrix number