ensemblgsoc/chanaka-gsoc-2017

Project for Chanaka de Silva for GSOC 2017

chanaka-gsoc-2017's Issues

REST endpoint definition

As defined in #6

Define the structure and keys of JSON Schema

We have to define all the fields [keys and values] which we are going to use in the JSON Schema.
And also the new structure of the data.

Deployment to Heroku

• Load application into Heroku
• Use a Bonsai Elasticsearch instance to host the test data (https://elements.heroku.com/addons/bonsai)

Functional search GUI

As mentioned in #6

Provide an example of the current track configuration interface

Take a screenshot and annotate the current track configuration interface. We should point out

• The track name
• The description
• Any short descriptions used
• The nested facets
• Renderer configuration
• Any other options

Loading JSON documents into Elastic search

Using JSON data generated in #11

Provide schemas to help design the database and JSON schemas

In order to design good database and JSON data structures we need to provide the original Ensembl data schemas. We will provide the following

• The link to the core ER diagram
• SQL schema for core
• The connection details to connect to the public Ensembl MySQL servers
• An example of the web data hash and description of keys/meaning
• A trackDb example and explanation of the keys being used

In addition we will highlight the areas of the core scheme of interest. This will form the first use-case of the project

New SQL Schema and ERD

Created SQL schema.
Designed the Database ERD.

Installing and running Elastic Search locally

• Download elastic search
• Install elastic
• Load an example JSON track document into Elastic

Define REST interface for search

Exposing the Elastic Search interface directly to users is not good design as it means we cannot isolate downstream applications from underlying infrastructure changes. We will define the basic queries required and contribute them to the issue in chanakaDe/ensembl-elastic-rest#1 .

Create GitHub issues to track work on GSOC 2017

We will create a github issue for all action points raised during our weekly telcos.

Loading tracks into the database from Ensembl

Run this bash command to log into the server:

$ mysql -hensembldb.ensembl.org -uanonymous homo_sapiens_core_89_38

This schema's species is homo_sapiens (location_species), its schema type is core (location_dbtype) and the object type we are retrieving here is gene (location_object_type).

Then run this SQL and find all displayable tracks in an Ensembl core schema for genes e.g.

select a.analysis_id, logic_name, ad.description, ad.display_label, ad.web_data
FROM (
	select distinct(analysis_id) as analysis_id from gene 
) as a 
join analysis on (a.analysis_id = analysis.analysis_id) 
join analysis_description as ad on (analysis.analysis_id = ad.analysis_id)
where ad.displayable = 1;

• logic_name is the same as location_logic_name and should be linked as such
• description is the same as track description
• web_data is a Perl formatted data structure. Simple regex of => to : should mean you can use JSON deserialisers to get back to a data structure

Loop through the web data fields and look for where the field key is repeated. These entries should be combined into a single track.

Generate JSON documents for Elastic search

New issue as defined in #6

Defining search text and expected responses from a search interface

Search requires the search interface to return the expected results for a given phrase. We will define a set of search strings and what track should be returned in response.

Documentation Templates

All of these are sample templates you can use to write documentation with. Please adapt as you see fit.

Introduction Document

# Project Name

# Description
Describe what the project attempted to do (take the original project description and adapt)

# Application Links

Link to the running application and make them aware of any usage requirements to make it work

# Repository Links
List each repository you have committed code to publicly

Architecture Template

• Project is the name of this project
• Component here refers to one of your major areas of work e.g. REST API, Elastic, UI, MySQL importer.

# Project Name

# Overall Structure

Diagram to show the overall structure about how the components you're about to list fit together

# Component Name

## Role
Describe the role of the component here. What is it trying to do

## Interconnections
How the component fits into the architecture

## Composition
How is it made? Technology choices. Major packages/files you should be aware of. Do we have existing documentation e.g. JSON schemas, database schemas we can bring in (do it)

## Limitations
Any limitations in the implementation the reader should be aware of.

Usage Template

Use this to describe how someone should use your application

• Tool refers to a unit of code you're going to deploy or expect someone to run

# Tool name

## Synopsis

Brief description about how to run or deploy the application. Include it as an inline example e.g. if this was a command line tool

`./path/to/command --option 1 --option 2 file`

## Description

Brief description of what the tool is meant to do. Link to any other information about it e.g. architecture documents if needed

## How to Install/Deploy/Run

Step by step about how to do this. Don't include really basic information that's better described elsewhere e.g. don't say "Sign up to Heroku by going https://..... and then clicking on signup". Link to Heroku's documentation about how to do that.

If it is a command line application then go through how to install the tool and then run it. Refer to any options you might have given below.

## Options

Command line options, if you have any, or options to tweak the tool

Define what will be delivered in each of the coding periods

Ensure we define what we will produce for each coding period. On the last telco we agreed month 3 would be to produce the GUI interface. We will make sure this is discussed at the next telco

Documentation of all components

• Web application front-end design and adding new pages
• Web application deployment; how to run
• REST API application
• Ensembl MySQL to track db MySQL import documentation install and build
• Ensembl MySQL to track db MySQL import documentation; how to run
• Track db MySQL to ElasticSearch import documentation install and build
• Track db MySQL to ElasticSearch import documentation; how to run

Put these with the projects as a README.md on each GitHub repo. I suggest the following structure for this.

# NAME OF PROJECT
Short description

#SYNPOSIS
  `How to run the command`

# DESCRIPTION
Bigger description

# INSTALL
How to install

# RUN
How to run

Create a GitHub page for schema description

Create a page in GitHub so we can describe the various schemas and specification in response to issue #1

Working REST API on top of Elastic search

As defined in #6

Use semantic versioning

We should use semantic versioning to track the versioning of all components of the project as defined at http://semver.org/