datacarpentry / r-raster-vector-geospatial Goto Github PK
View Code? Open in Web Editor NEWIntroduction to Geospatial Raster and Vector Data with R
Home Page: https://datacarpentry.org/r-raster-vector-geospatial
License: Other
Introduction to Geospatial Raster and Vector Data with R
Home Page: https://datacarpentry.org/r-raster-vector-geospatial
License: Other
Can someone fill me in on the purpose of the pseudo-code chunk that begins Lesson 01 (line 70; http://data-lessons.github.io/NEON-R-Spatial-Raster/R/Plot-Rasters-In-R.R/ when rendered everything down to Plotting Raster Files).
Was it just the basic code that was then broken up below for teaching purposes. Most of it seems to repeat. I don't want to delete it if it is vital to the lesson, I'm just not seeing where it adds to the lesson.
@lwasser?
Left a comment in this lesson to include a discussion of CRS - this would be content in the INTRO TO SPATIAL Data part of the workshop if that is developed. So we should revisit where this will live. But regardless it needs to be created. We can largely link out to existing content as this topic is well covered in the GIS world. Does anyone have time to work on this little chunk of explanation?
http://data-lessons.github.io/NEON-R-Spatial-Raster/R/Introduction-to-Raster-Data-In-R/
Megan found a very nice overview of projections
https://www.nceas.ucsb.edu/~frazier/RSpatialGuides/OverviewCoordinateReferenceSystems.pdf
i haven't gone through it to carefully ensure it covers what we want to cover but it might.
From @brymz
What is going on with the 'Related Content: raster-data' at the bottom of the page? It doesn't seem to be in the .Rmd. It just animates (shrinks) when I click on it.
I think the tag used for the lessons (and main tag) is likely wrong or spelled wrong.
@mjones01 when you review these lessons, can you have a look to see if we are using the right mainTag for the raster lessons?
Thank you!
This is a good challenge but a bit out of place in this lesson. Consider moving it.
@lwasser suggested into the introductory data management section
To be consistent with other NEON sites and #WorkWithData lessons please use LiDAR instead of lidar or any other variants.
@lwasser which updated file needs to be adding to the repo so that the overview/TOC will appear in the sidebar. Currently the header shows up but nothing populates.
we might use this in our background content.
Leah needs to create a geotiff with
The other option is i could include band 4 as a separate tiff that is in a different CRS - then it needs to be projected?? @jhollist thoughts on this if i need to do this work anyway? i could give you band 4 as a geotiff in WGS 84 geographic CRS.
NOTE - we'd want to create NDVI here
http://data-lessons.github.io/NEON-R-Spatial-Raster/R/Multi-Band-Rasters-In-R.R/
l.
Hi Spatial Raster Group- @jhollist @msmorul @KristinaRiemer @JasonJWilliamsNY @brymz !
I'm very excited to see the material shelled out here in this repo!
I wanted to followup to see
Let's start a discussion in this issue. It is understandable if you are too busy to work on things - feel free to say that. I am just trying to plan resources on my end. Thank you guys for all of the work on this and for getting back to me on timeline and availability :)
Just for reference, I rendered out one repo already. You can see here what the lessons will eventually look like when we flesh them out!
http://data-lessons.github.io/NEON-R-Spatial-Vector/
You are all awesome for putting these materials together! Thank you for your time!
Leah
@mjones01 a while back we had a brief discussion where you use fun() and i was using fun in our code. let's move to what you were doing per @brymz latest PR . copying his comment below.
He also suggests creating a style guide (we are working on this but should provide it to people!) and a contributing guidelines .md file
Oops. 80 characters, absolutely. I keep my text editor at 80 characters so I can easily make the hard wrap, but sometimes I miss that step when I'm editing content. Thanks for the gentle reminder.
As for the bigger question, I developed a style guide for the Data Carpentry semester course (http://www.datacarpentry.org/semester-biology/docs/style-guide/) that I shamelessly used here. The function() formatting is also addressed there. I would definitely recommend a contributing.md and/or README.md that outlines these sort of topics.
Let's consider implementing this throughout. this isn't a huge high priority issue but it's a very good thing for us to implement and do when our brains our tired :)
Per @mjones01 a comment (i think this is your comment?)
The bottom line is that anytime you reproject, or do calculations on a raster you are modifying it. In the case of projection you are moving it from one grid to another so there is some interpolation that needs to happen .Thus you will notice small changes in the data. This can really compound over time.
However i'm not sure if we want to teach this or not to a beginner audience as there is a LOT in this section already and reprojection is confusing and very hard to teach quickly, well. open to ideas but i pulled the comment from the lesson whosever it was!.
Hi @mjones01 - all edits are merged! Please have a look at the rendered lessons and let me know if you see any issues. I have fixed the baseurl (removed the slash). Let's see if anything breaks. Things that could break include images! If you are happy with it as built, feel free to close this issue or assign it back to me to close :)
Once the code and steps for all the lessons are complete, make sure they fit into the broader "Jessica" story. That is, justify why these functions will be useful for answering scientific questions.
Do we want to/ need to explain slots in this module (or other module)?
Peripherally used in Lesson 00. Used elsewhere?
Check w/ vector/inteagration group and make sure their requirements are pre-loaded at the end of this lesson. Are the right rasters loaded in memory.
So first, when I made a pull request, it said there are merge conflicts and I have no idea how to deal with that.
Second, what is the difference between plot() and image(), and do we need to introduce image here?
Third, is there anything else that needs to be included in this lesson? I can't think of anything that is basic that might be useful.
Merge stub / lessons material and raster_module_code.R
into 03 and 04
The code for the this challenge uses something that hasn't been taught in the lesson. I tried to recreate the challenge code using what was taught in that and the previous lessons and was unable to. I think, even if we give a more efficient way, the challenge code should be something do able based on what we are teachings with (maybe) some minor Googling.
http://data-lessons.github.io/NEON-R-Spatial-Raster/R/Raster-Times-Series-Data-In-R/
@lwasser I think we should make the two stretch graphics a bit more inline with the others Colin's created in the last few days.
However, you two were the ones who initially created these so I wanted to run it by you before asking Colin.
http://data-lessons.github.io/NEON-R-Spatial-Raster/R/Multi-Band-Rasters-In-R/
or lines 379/389.
Currently Lessons 00, 01, 03-07 are titled "Raster 00:Title" while Lesson 02 ( and all lessons in the other NEON-DC repos) are "Lesson 00: Title".
Is there Lesson 00 important to the Data Carpentry format?
If not, perhaps switching to "Raster 00:", "Vector 00:" , "Time Series 00:", and "Spatial Intro:" would be most appropriate as we are now at the point where Time Series lessons refer to Vector Lessons and Spatial Intro lessons refer to all the different other repos yet "Lesson 00:" doesn't make that obvious.
Would this also facilitate talking about the different lessons when multiple individual series are taught together in one workshop?
A few folks have edited L01 - plotting rasters in R. Given histograms are also included there, let's add a few lines of code to show users how to make prettier histograms as well. We can add color, customize labels, etc. It's a simple addition!
http://data-lessons.github.io/NEON-R-Spatial-Raster/R/Plot-Rasters-In-R/
i rebuild but i think the output from the challenge needs to be cleaned up - potentially placed in a separate chunk just as an example. will revisit tomorrow.
All the NEON derived data (Lessons 00-04) have UTM Zone 18 in the geotags. The Landsat derived data (Lessons 05-07) have UTM Zone 19. Why the difference? MA is divided by the two. We at least need to make a note in the lessons about why the two different ones and why it does/ does not create a problem.
The landsat data downloaded have very long file names. The file name itself actually contains metadata including the Julian day collected, year etc. And you can parse it via the metadata. For this lesson, I actually created shorter file names for simplicity sake. However, should i use the original complex file names to allow users to explore the metadata as an activity?
When a challenge has an output of a figure (and most do), the figure causes the shadow element to get broken on the bottom. This leaves a little bit of shadow on the bottom left and a weird angular shadow on the right side.
@lwasser We could remove shadow element (data tip boxes are flat) or figure out how to insert a space.
L01 challenge (chunk {r challenge-hillshade-layering, echo=FALSE}) requires a SJER hillshade for DSM & DTM. Neither are in the current data available on FigShare.
Write lesson for performing a reprojecetion on the raw landsat.
I don't think the text is giving the correct reasoning for why we do what we are doing. I'm pretty sure I wrote that original code but not the indexing part in code or text. The origin day is 01 January 2011 so the extracted day is 01. The Julian day for this is also 01. If we need to cover Julian day 05 (which is the 5th of January), we cant add origin+Julian day because 1+5 is 6 or January 06, 2011, we have to add origin+JD-1 to get the correct day January 05 2011.
The explanation of zero-based indexing isn't wrong, but I think its application here is wrong. Can someone confirm this for me?
http://data-lessons.github.io/NEON-R-Spatial-Raster//R/Extract-NDVI-From-Rasters-In-R/
~line 231, code chunk: {r convert-jd}
CODE:
#create a date column; -1 added because origin is the 1st and indexing begins at 0.
#If not -1, 01/01/2011 + 5 = 01/06/2011 which is Julian day 6, not 5.
avg_NDVI_HARV$Date<- origin + (avg_NDVI_HARV$julianDay-1)
TEXT:
Note that when converting our character formatted Julian Day values to dates,
we subtracted one as follows:
avg_NDVI_HARV$Date <- origin + (avg_NDVI_HARV$julianDay-1)
This is because R
stores Julian days using 0-based indexing. That is the first
value is 0, rather than 1. Thus Julian Day 1, is stored as 0, Julian day 2 is
stored as 1, and so on.
The file CropNDVIFiles.Rmd code is currently in the knitr_processing_code folder. Should this be moved to Lesson Development Repo? Maybe create a new folder called NEON_external-data-processing-code that can be a repo for all the code created to extract/clean the data for the lessons. Some might be developed into lessons. Might be very useful if OS/IS might be using the code (or referring to it).
Once we've figured out landing pages and what their URLs will be this section needs to be updated (same is true for all the NEON-dc lessons).
it looks like the mainTag is set to raster-data rather than raster-data-series (or whatever the YAML list contains for this series!). I will fix this once i've merged zach's edits.
check how L03 renders - the .notice tag in the data tip below includes a function chunk however - i'm not sure this will render properly in kramdown as the {} tag is on a paragraph and doesn't support multiple paragraphs. Let's make sure this works before pushing to dev.
Data Tip: A custom function consists of a defined
-tasks performed on a input object. Functions are particularly useful for tasks +set of commands performed on a input object. Custom functions are particularly
-that need to be repeated over and over in the code. A simplified syntax for a +useful for tasks that need to be repeated over and over in the code. A simplified
-function in R is: +syntax for writing a custom function in R is:
-functionName <- function(variable1, variable2){WhatYouWantDone, WhatToReturn}
+```
+functionName <- function(variable1, variable2){
@mjones01 this is very weird. I just accepted you PR but it said "0" files changed. I saw your commits and see the changes but the files don't have the fixes. Now the headers don't work in this repo so this confirms we need to fix this.
Colin created a great extent graphic. where should it go?
@mjones01 you had originally suggested this graphic - where did you intend to place it?
At end of lesson chm and very end, save calculated chm geotiff and ndvi csv
Per @brymz
The rainbows color palette pops up oddly. I like the idea of demonstrating how color palettes differ, but I'd like to see a progression of plot map -> plot hill shade -> overlay map on hillshade, so that you can scroll back and see how they all fit together. Also, I wonder why it is not the hillshade that is overlayed on the map.
CLARIFICATION:
The progression I would like to see already exists (kind of) from the challenge map solution one map before 'Layering Rasters' to the 'DMS map with hillshade', IF the 'DSM map with hillshade' was plotted with terrain.colors() as with the earlier 'DMS' maps. This way you could scroll back to reference the 'hilshade' and 'DSM' maps to see how they line up when plotted together. If you want to introduce rainbows, I recommend repeating the sequence of 'DSM' -> 'hillshade' -> 'DSM with hillshade' with the rainbows color. Alternately, you can introduce rainbows earlier with the 'DSM_HARV' map just before introducing hillshade to match the sequence. Hope that helps clarify things.
I am merging the edits and we will keep this open for consideration when we work on the lesson more!
The histograms look fine when run in R but no data shows up when the lesson posts. Ideas?
Participants requested (and it would be helpful for teaching) diagrams on R data structures. Particularly for heterog. ones like spatial objects with nested components stored in slots (eg data.frames in .@DaTa slot, etc. Looking for some for today's teaching now.
Write up an example to use extract. The use case would be to find the temperature of an area where a specific point observation occured. Some overlap w/ the integration group, but this is an introduction to extract.
@lwasser originally noted:
#R cant render that as a stretch function. Not sure how to properly fix this
#via code, for the meantime writing a manual exception
CODE: 433-444 or http://data-lessons.github.io/NEON-R-Spatial-Raster/R/Extract-NDVI-From-Rasters-In-R/
for (aFile in rgb.allCropped.SJER)
{NDVI.rastStack <- stack(aFile)
if (aFile =="NEON-DS-Landsat-NDVI/SJER/2011/RGB/254_SJER_landRGB.tif")
{plotRGB(NDVI.rastStack) }
else { plotRGB(NDVI.rastStack, stretch="lin" ) }
}
7 Jan I couldn't get this manual fix to work. It does work if we remove ", stretch="lin" " but then all images in very blue cast.
NEED TO FIX!
I have code that will allow us to automatically list related lessons. Right now it's at the bottom of the page rather than the top. What is better?
I'm trying to shorten up the top grey block as it's long and for SEO it's ideal if we have unique content up towards the top. that list is useful but it could go at the bottom potentially with ONE link at the top to a "module" page that describes the entire lesson set like what SWC does. Thoughts? @mjones01
Please see L00 to see where i am going with this.
http://data-lessons.github.io/NEON-R-Spatial-Raster/R/Introduction-to-Raster-Data-In-R/
Hi Guys -- i've been working on what is now L03 - modifying rasters in R.
http://data-lessons.github.io/NEON-R-Spatial-Raster/R/Modify-Rasters-In-R/
if the above link doesn't work use this one (to be fixed soon)
http://data-lessons.github.io/NEON-R-Spatial-Raster/R/Modify-Rasters-In-R.R/
The challenge with this particular lesson is that the best approach to performing raster calculations is not direct raster math (rasterone-rastertwo) UNLESS it is a small - non complex operation. So i introduced overlay
. However i'm not realizing that we have not introduced functions and this section is getting heavy.
How should we proceed? Should we explain functions? Or should this be a separate advanced lesson - l04 that instructors can chose to skip if they want to?
the challenge is that this module is getting long potentially and i know many people get hung up on functions. BUT functions are critical to good coding.
Thoughts? @brymz @jhollist @msmorul @jduckles @jsta @KristinaRiemer
@mjones01 a note as we work on lessons in the future
@brymz noted:
We should add a space after # in comment lines of code as per http://adv-r.had.co.nz/Style.html.
I think this is a good idea to implement throughout. It's also a very easy quick update.
i'll implement this in the intro lessons which I will work on today. Zach is taking care of raster.
I can also do vector if you want to do time series when you have a bit of "i need to do something mindless" which i have often at the end of the day after coding all day! :)
Currently this is introduced in Lesson 04 (https://github.com/data-lessons/NEON-R-Spatial-Raster/blob/gh-pages/04-Multi-Band-Raster.Rmd) lines 299-326 but is poorly address and potentially unnecessary.
Does it need to be explained? or can we remove it from the code?
How best to teach / explain the concept?
Is a graphic needed? eg. http://www.nrcan.gc.ca/earth-sciences/geomatics/satellite-imagery-air-photos/satellite-imagery-products/educational-resources/9389
Does it fit better elsewhere in the series?
Need to add a geotiff w/ the following characteristics:
Identify and add data so the class can run an example where they calculate ndvi
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.