bradnerlab/jenkinsjobs

JenkinsJobs

Python scripts and config.xml files for various jobs on the Bradner Lab Jenkins server.

Jenkins (http://jenkins-ci.org/) is a continous integration server. Jenkins is flexible and can also serve as a cron replacement and can run arbitrary jobs. Jenkins can provide a simple GUI wrapper around command line programs, and also provides easy ways to string multiple jobs together, email results or failure messages, archive results, and do logging.

Most of these Jenkins jobs will be centered around automating (as much as possible) the process from genome sequence data to useful plots to help Cancer researchers.

These jobs require at least the following plugins:

• http://wiki.jenkins-ci.org/display/JENKINS/Git+Plugin
• http://wiki.jenkins-ci.org/display/JENKINS/Email-ext+plugin
• http://wiki.jenkins-ci.org/display/JENKINS/Extended+Choice+Parameter+plugin

jenkinsjobs's Issues

Automate First 3 Steps of Processing

I need to follow up with Charles to understand what the first 3 steps are.

I will probably end up creating 3 Jenkins jobs, and have one job call the next.

Error handling is important. If a step fails, it should email someone where the failure occurred and stop. If there are no errors, then the next job should be triggered.

Allow Multiple Selections

Allow users to make multiple plots at once.

Each selection for a plot should have a color associate with it. For now, probably just use the same color for each item.

Have a drop down menu with multiple colors to choose from, or include some RGB examples with some colors.

The default mode should be to plot each selection separately, but a drop down allowing a single plot with multiple lines is fine as well.

Email Failed for Jenkins job

http://tod.dfci.harvard.edu/jenkins/job/Normalize_GPL16043/configure

"Error sending to the following VALID addresses:"

GPL16043 Upload Job

Charles explained the following in an email:

the user uploads a zip file and provides a name

jenkins can then call the following job

/ark/home/cl512/pipeline/GPL16043.sh ARG1 ARG2

e.g.
bash /ark/home/cl512/pipeline/GPL16043.sh /ark/home/cl512/ressrv19/raw/expression/JB20130926st.zip affy2

where ARG1 is the path to the zip file

and ARG2 is the name that the user provided (the name can't have any spaces)

In the last line of the std out of GPL16043.sh, the location of the output file is provided as follows (example):

"WROTE OUTPUT TO: /ark/temp/affy2_tmp_15578/affy2_output.zip"

this zip should then be sent by email to the user.

Hierarchy Selection

should be 3 sets of drop downs:

• first by genome, then by source, then by cell type, then by name
• hierarchy is important because there will probably be about ~1000 datasets by the end of the year

samba on tod

set up samba for /ark /raider and have it link to tod user accounts

Dynamic Loading of Table File

The choice options should be loaded dynamically from the following file:

/mnt/d0-0/share/bradnerlab/projects/masterBamTable.txt

So if the file changes, the job can immediately use it without any manual updates to the Jenkins job.