mlap/neon4cast-aquatics

The aquatics group repo for ESPM 288

RCN_freshwater

WIP! Beginning!

Test from local computer

Test from Git

neon4cast-aquatics's Issues

Place for us to put useful visualizations/plots

I set this up so we could have a central place to put any informative plots that we make; hopefully this will be useful when determining the most important drivers to take into account for our models.

Drivers

Potential Drivers of DO & Temp

• temp
• recent rain
• water depth
• water velocity
• vegetation
• turbidity
• strength & duration of sunlight --> for photosynthesis

Running List of Main Objectives

Deadlines:

• Thursday, April 22nd: Have all 3 drafts of our individual forecasts prepared to share.
• Thursday, April 29th: Have final draft completed and ready to go.
• May 3-15th (dead week & finals week): Finalize forecasting model. Prepare for final submission.
• May 15-31st: Input April data released on May 15th to make forecast for weeks of May 1-8th

Forecast submissions will due beginning May 31, 2021 at 11:59 Eastern Standard Time (UTC−05:00) for forecasts that start May 1. Final forecast submissions will be due on August 31, 2021 at 11:59 Eastern Standard Time (UTC−05:00) for forecasts that start August 1.

General:

• Read through Modeling issue tab to view classes of models we could use. Learn about different classes of models we could use: Bayesian models (CH 1,2,4,6), neural networks, Machine Learning (ML) Models, etc. Thursday we will go over Bayesian modeling on 3/8/2021 with Marcus. Maybe one month of Machine Learning & one month of Bayesian.
• Use literature examples of forecasting to try out models with different approaches.
• Figure out set up of sensors for Barco site: number, fixed/floating position, etc.

Lin:

• Was going to look into grading process for the model

Max:

• Finish setting up thelio & cloning repository
• Learn more about Neural networks and Bayesian models (monte carlo specifically)
• Plot relationships between drivers & include alongside plots showing Time vs. Drivers
• Create base forecasting model in R to build on and test out grading with. (Variables: diffusion, photosynthesis, respiration, mortality, and others)
• Use Fable R Package to do basic forecasting analysis. Main package site is here

Main links of project requirements and deadlines:

• EFI RCN NEON Ecological Forecast Challenge
• NEON Aquatics Forecasting Challenge_Targets and Other Environmental Variables

A list of some useful links I previously posted:

• Details of the targets, how they are calculated, descriptions of the target files, and examples of other environmental variables that could be used
• Detailed Information on site: Barco Lake
• Download raw data here
• Spatial data: here & here
• General info on our study site from Sarah

Main Objectives Before Monday 2/1/2021 & Thursday 2/4/2021

• Data exploration: Continue running the template data listed in our code section and check to see accuracy of results.
• Look for scientific literature & other sources online of people who have done similar projects.
• GIS work to take a look at spatial relationships. (Fires, vegetation, land cover)
• Take a look at the raw data and plot trends you think might matter
  -Plot changes in mean data per year
  -Show varied highs/lows between each site
  -etc.

Old Goals to Continue:

• Share paper links on drivers of DO and other factors to take into account for our forecast
• Plot data sources from Table 2 and look for trends
• Brainstorm some additions that we could make to the template code to make it more accurate
  - If you make changes to template code, make your own branch

Main Objectives Before Friday 1/29/2021

• Run the template data listed in our code section and check to see accuracy of results
• Share paper links on drivers of DO and other factors to take into account for our forecast
• Plot data sources from Table 2 and look for trends
• Brainstorm some additions that we could make to the template code to make it more accurate
  - If you make changes to template code, make your own branch

Modeling

The general classes of models that I imagine we will investigate over the term include the following, I'll provide a brief running example that hopefully conveys the gist of the different methods:

Mechanistic Models -- This is basically direct physical modeling of the system. For example, let's say I am making a forecasting model for the pressure of a gas in a box. A mechanistic model, which uses the ideal gas assumption, would be P = NRT / V. So if I am given the temperature forecast for the box, provided moles and volume are assumed to be constant, then I could use this model to predict what the pressure would be over time.

Bayesian Models -- The general idea here is that you are going to use probability distributions to model the data. So in the gas example above, let's say we think that pressure for the box is drawn from a normal/gaussian distribution that has a mean of NRT / V with some standard deviation. The task for bayesian modeling would be to first estimate what the parameters of the model are from the past data; e.g. what is the std. Once we have estimated this parameter, and provided we are given a forecast for temperature, we can then make our forecast by sampling from the distribution to get estimates on what the pressure will be.

Machine Learning (ML) Models -- ML is a huge field with tons of tons of different algorithms that can be starkly different. But the general main idea, not always true though!, is that ML models typically have a bunch of parameters. During the training phase for this above example, you would feed in past data of moles, temperature and volume into the ML model where the ML model outputs one real-valued number which we want to fit to pressure. Through an algorithm called back propagation, the ML model will update the parameters in its model so that its output will better approximate the pressure of the gas in the box. Once our model works well on past data, then we can make forecasts with the ML by say using a temperature forecast and plugging in the best estimate for moles and volume as inputs, the ML model will then directly output the prediction for pressure.

There is a lot more to the Bayesian and ML approach but hopefully this gives everyone a general roadmap of what is what to guide people in independent exploration. The progression of mechanistic to bayesian to ML I think is natural because often in Bayesian methods you need some understanding of the mechanisms to write down the model. ML meanwhile you don't need any mechanistic understanding to do ok, but having some mechanistic understanding can potentially help you a lot. Like in our aquatics examples, let's say from mechanistic modeling, we find that phytoplankton population is really important to DO but none of the drivers really capture it, if we can define some proxy variable to describe phytoplankton population then maybe inputting this proxy variable into our ML model would lead to much better predictions in a shorter amount of time. I'll scavenge around for some links on these various topics and add them in a bit.