Ingest Marxan data files

[prioritizr]

Ingest Marxan data files

[ricschuster]

I think this is a crucial component for the package. I would say for starters we should support shapefiles (polygon), rasters and csv/txt files. I have put two basic functions that do this for shapefiles and csv files in a repo called 'pre_prioritizr'. The functions are located in the 'To_source.R' file and work with the ILPapp function. That function uses the ingested files and sets up a basic Shiny app using gurobi to solve the problems. I think the ingest functions in conjunction with the Shiny app creation function could really help first time users to buy into our approach. I agree, a function that ingests and then solves the problem would be good, but I think one option (maybe the standard) should be to create a Shiny app in the ingest process so users have the option to use a GUI. That being said, I did like the way that I set things up, which allows the ingest function to create file outputs for subsequent use in the solver app. I did this to allow for more flexibility and to allow storage of input files. Thinking further down the line, this could also avoid repeat processing if users want to customize the problem starting with the ingested dataset. Maybe another option in the ingest function, to allow for more flexibility?

…

On 2017-02-19 15:08, Jeff Hanson wrote: I think having the ability to ingest Marxan data files and solve it using ILP would be very useful. But how well supported should this be? I think this feature will be crucial for showing first time users the benefits of using ILP. After the release of the earlier version of the prioritizr package, I had trouble getting people to try it. So, I was thinking that the Marxan ingest function would ingest the files and then solve the problem. We intentionally would not allow users to customise the problem further by adding constraints using the standard syntax. This way users can see for themselves that ILP is awesome (well, hopefully haha), and then will want to learn how to create and customise problems using the standard syntax. Thoughts? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#4>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ADSRZPbfRnFRPN2-8kLcbc9ww1kzvCt_ks5reMtsgaJpZM4MFoYV>.

-- Richard Schuster Liber Ero Postdoctoral Fellow Geomatics and Landscape Ecology Laboratory Department of Biology Carleton University Email: [email protected] http://richard-schuster.com/ Twitter: @ricschuster ph: 250-635-2321

[jeffreyhanson]

The pre_prioritizr app looks great!

Yeah that sounds good. If I understand correctly, the pre_prioritizr app assumes users have the marxan data already compiled in a shapefile. We could definitely accommodate input like this with no problem.

We could have an R function that creates a marxan problem like this:

p <-marxan_problem(pu_polygons, cost=..., features=..., locked_in=..., 
                                    locked_out=..., penalty=..., edge_factor=....)

where the dots are column names or vectors of values. This returns a standard ConservationProblem object that would be the same as if the users did:

p <- problem(pu_polygons, features=c(......)) %>%
     add_minimum_set_objective(cost='cost') %>%
     add_locked_in('locked_in') %>%
     add_locked_out('locked_out') %>%
    add_boundary_constraint(penalty=.., edge_factor=...)

We could create a load_planning_data (idea for better name?) function that opens a shiny app that is essentially a wrapper for the marxan_problem function to help users import data.

After ingesting the data and setting some initial parameters, they could run the modify function that is the standard shiny interface for visualising solutions and playing around with settings.

p2 <- modify(p)

How does that sound?

My concern is how we should handle raw marxan input data. In this case, the data doesn't have a spatial component and so doing modify(p) to play around with different parameters and visualise the solution won't work. For this case, should we have it just output the solution as a logical (binary) vector?

Also, I wonder if the shiny apps should go in their own separate R package (eg. prioritizrshiny?), and the R interface should have a different package name (eg. prioritizrcli?), and the real "prioritizr" package would just be a package that imports the two packages (like tidyverse does for dplyr, plyr and etc)?

[ricschuster]

Correct, in this initial version I assumed compiled shapefiles. But the plan was that in subsequent iterations users could also do data processing with this approach, just like with the marxan package. Yes, I was thinking of a function like the marxan_problem example you had. re: load_planning_data. No idea for a better name yet, but parts of this can be recycled from Matt Watts' marxanui and marxan.io repos. I think your workflow idea sounds good, the only thing I would change is the function name 'modify'. This might be getting more into the package splits, but I think using 'modify' to start the standard Shiny interface might send a strange message to non-technical users. To me that sounds like the solution has already been generated and Shiny would only be used to modify it. I think the separate package approach would take care of this though. re: raw marxan input data. That's really easy to accommodate in Shiny. In the function call/app creation we just have a check about spatial data. If no spatial data is included we don't display the 'map' tab in the Shiny app and stick to e.g. the 'Edit Target' and 'Results + Download' tabs (http://forbasin.forestry.ubc.ca/CDFCP.v0.21.2). In future iterations we could also include output figures, such as something like the cost vs compactness tradeoff curve, but that's something down the line, after the initial package(s) stand. I think having a separate R package for the shiny apps would make a lot of sense and also keep the cli package with the problem formulation functions et al. cleaner. prioritizrshiny would always depend on prioritizrcli, but cli can easily be used without shiny.

…

On 2017-02-19 19:38, Jeff Hanson wrote: The pre_prioritizr app looks great! Yeah that sounds good. If I understand correctly, the pre_prioritizr app assumes users have the marxan data already compiled in a shapefile. We could definitely accommodate input like this with no problem. We could have an R function that creates a marxan problem like this: |p <-marxan_problem(pu_polygons, cost=..., features=..., locked_in=..., locked_out=..., penalty=..., edge_factor=....) | where the dots are column names or vectors of values. This returns a standard |ConservationProblem| object that would be the same as if the users did: |p <- problem(pu_polygons, features=c(......)) %>% add_minimum_set_objective(cost='cost') %>% add_locked_in('locked_in') %>% add_locked_out('locked_out') %>% add_boundary_constraint(penalty=.., edge_factor=...) | We could create a |load_planning_data| (idea for better name?) function that opens a shiny app that is essentially a wrapper for the |marxan_problem| function to help users import data. After ingesting the data and setting some initial parameters, they could run the |modify| function that is the standard shiny interface for visualising solutions and playing around with settings. |p2 <- modify(p) | How does that sound? My concern is how we should handle raw marxan input data. In this case, the data doesn't have a spatial component and so doing |modify(p)| to play around with different parameters and visualise the solution won't work. For this case, should we have it just output the solution as a logical (binary) vector? Also, I wonder if the shiny apps should go in their own separate R package (eg. prioritizrshiny?), and the R interface should have a different package name (eg. prioritizrcli?), and the real "prioritizr" package would just be a package that imports the two packages (like tidyverse does for dplyr, plyr and etc)? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#4 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ADSRZK9FWAdZT8SGiJn_EiyU_wT1GoFBks5reQqygaJpZM4MFoYV>.

-- Richard Schuster Liber Ero Postdoctoral Fellow Geomatics and Landscape Ecology Laboratory Department of Biology Carleton University Email: [email protected] http://richard-schuster.com/ Twitter: @ricschuster ph: 250-635-2321

[jeffreyhanson]

Ok, sounds good.

Ok, I'll create a new repo called "prioritizrutils" and put the code there (now that I think about it, cli sounds wrong because its still within R).

[jeffreyhanson]

I've been thinking more about how the different packages should be managed.

I still think it's good having the main R interface in a separate package from the shiny and data packages. But I realised that if we have the "prioritizrutils" in a separate package from the main "prioritizr" package this would mean that all the "useful" help files and package vignettes would be located in a different package. I worry that this might make it harder for users to navigate the help files. What do you think?

[ricschuster]

I agree. Lets stick to three packages: R interface Shiny interface(s) data package (unless that gets too big, the split into more than one) If we keep it at that we can hopefully minimize confusion.

…

On 2017-03-07 18:40, Jeff Hanson wrote: I've been thinking more about how the different packages should be managed. I still think it's good having the main R interface in a separate package from the shiny and data packages. But I realised that if we have the "prioritizrutils" in a separate package from the main "prioritizr" package this would mean that all the "useful" help files and package vignettes would be located in a different package. I worry that this might make it harder for users to navigate the help files. What do you think? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#4 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ADSRZDMNd3trOJ8VWlZ6uonYT08-MVDQks5rjer0gaJpZM4MFoYV>.

-- Richard Schuster Liber Ero Postdoctoral Fellow Geomatics and Landscape Ecology Laboratory Department of Biology Carleton University Email: [email protected] http://richard-schuster.com/ Twitter: @ricschuster ph: 250-635-2321

[jeffreyhanson]

I've just pushed the bulk of priorititzrutils here and everything seems to work. I'll delete prioritizrutils now that it's not needed?

[ricschuster]

Sounds good. Thanks very much Jeff. ⁣Sent front my android device​

…

On Mar 10, 2017, 01:20, at 01:20, Jeff Hanson ***@***.***> wrote: I've just pushed the bulk of priorititzrutils here and everything seems to work. I'll delete prioritizrutils now that it's not needed? -- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: #4 (comment)

[jeffreyhanson]

The package has had the marxan_problem function to accept Marxan input data for a while now, so I'm closing this issue.