albhasan/amazongreenup2005

Reproduction of the computations on the article "Amazon Forest Green-Up During 2005 Drought"

amazonGreenUp2005

Reproduction of the computations on the article "Amazon Forest Green-Up During 2005 Drought"

Pre-requisites:

• Docker.io
• SSH
• Internet access

Files:

• LICENSE - License file.
• README.md - This file.

<li>Docker-related files
<ul>
	<li><code>setup.sh</code> - (host) Script for generating a docker image.</li>
	<li><code>Dockerfile</code> - (host) Docker file for building a Docker Image.</li>
</ul>
</li>
<li>SciDB-related files:
<ul>
	<li><code>conf</code> - SHIM configuration file. SHIM is SciDB's web service</li>
	<li><code>scidb_docker_XX.ini</code> - SciDB's configuration files. Here you can control things such as the number of instances (see table below).</li>
	<li><code>iquery.conf</code> - IQuery configuration file. IQuery is SciDB's native client able to process AQL and AFL queries.</li>
	<li><code>startScidb.sh</code> - Script for starting SciDB.</li>
	<li><code>stopScidb.sh</code> - Script for stopping SciDB.</li>
	<li><code>containerSetup.sh</code> - Commands for setting up SciDB inside a container. It also creates some test data.</li>
	<li><code>anomalyComputation.afl</code> - Array Functional Language instructions to calculate EVI2 anomalies.</li>		
</ul>
    </li>
    <li>R scripts
<ul>
            <li><code>anomalyHistogram.R</code> - Script for retrieving and ploting the resulting EVI2 anomaly histogram.</li>
            <li><code>exploreResults.R</code> - Script with general functions for retrieving and ploting the resulting EVI2 anomalies.</li>
            <li><code>installPackages.R</code> - R script for installing R packages.</li>
    </ul>
    </li>
<li>Other files
<ul>
	<li><code>installBoost_1570.shh</code> - Install Boost libraries.</li>
	<li><code>installGribModis2SciDB.sh</code> - Install a tool for exporting HDFs to SciDB's binary.</li>
            <li><code>installParallel.sh</code> - Script for installing parallel.Parallel allows to execute scripts at the same time.</li>
	<li><code>install_pyhdf.sh</code> - DEPRECATED - Install an interface for enabling python to handle HDFs.</li>
	<li><code>downloadData.R</code> - DEPRECATED - R script for downloading MODIS data from NASA website.</li>
	<li><code>downloadData.sh</code> - DEPRECATED - Script for downloading MODIS in parallel. It is a wrapper of <code>downloadData.R</code></li>
	<li><code>hdf2bin.sh</code> - DEPRECATED - Script for exporting HDFs to binary files. It is a wrapper of the python scripts available at <a href="http://github.com/albhasan/modis2scidb" target="_blank">modis2scidb</a>.</li>
</ul>
</li>

Instructions:

1. Clone this project git clone https://github.com/albhasan/amazonGreenUp2005.git
2. Setup SciDB on Docker and other required stuff:
   • Build a docker image ./setup.sh. This script will build the Docker image scidb_amazon_img and it will start the Docker container scidb_amazon1.
   • Login the SciDB Docker container scidb_amazon1 by using ssh -p 49911 root@localhost. The default password is xxxx.xxxx.xxxx
3. Run the container script using one of the SciDB configuration file names as a parameter, for example: /home/root/./containerSetup.sh scidb_docker_2a.ini.
4. The array with the results is MODIS_AMZ_EVI2_ANOM
5. You can use exploreResults.R to get the computation results in R, either form the container or the host.

SciDB setup configurations files:

Name	Instances per server	Max concurrent connections	CPU cores per server	GB per server
scidb_docker_1.ini	1	2	2	2
scidb_docker_2.ini	2	2	4	4
scidb_docker_2a.ini	2	2	4	8
scidb_docker_2b.ini	2	2	4	16
scidb_docker_4.ini	4	4	4	16
scidb_docker_8.ini	8	16	24	160