cugos/dropchop

:fork_and_knife: browser-based spatial operations

dropchop's Issues

incorporate browserify?

@thebigspoon I'm curious if you'd want to use browserify here or stick with using the Grunt wiredep for loading in all of our bower components?

handle multiple files

currently only grabbing the first file in the list if a user drags two with files[0]

How do we handle turf functions that return something other than geometry?

Currently all of our turf functions return geometry and we add as a new layer. How do we propose working with functions that are essentially mathematical functions that are more useful for attributes than new data layers?

Turf count is an example of a function that doesn't return new geometry. http://turfjs.org/static/docs/module-turf_count.html

Tests Tests Tests -- Union is broken

It's great that all this new development is happening, but why don't I see tests around the stuff we are merging into master?

Currenlty the union is broken

Standardize incoming geometry (handle FeatureCollections)

Right now feature collections are a bit iffy. The ops.geom.union() function, for example, tests to see if a passed object has object.features, which means it's a feature collection and then uses the first feature object.features[0] to do the operation. This doesn't seem very good!

TURF: Add extent operation

Follow these instructions for adding a new turf function.

Turf docs: http://turfjs.org/static/docs/module-turf_extent.html

build "view extent" button

this button would zoom the map to the extent of all of the visible layers

break project into modules, then compile

We can use grunt/gulp to do this. It would be really nice for keeping things organized on the development side

i.e. operations.js, handlers.js, etc. as their own files, but combined and minified on the app side.

ability to remove layers

A user should be able to remove layers. Doh!

_unCollect should run on files added, rather than executed

Right now if someone adds a featureCollection of a single feature from their desktop, it stays the same. When adding the feature to the map, we should _unCollect() instead of that function only existing withing GeoExecute.js

Allow a way to download/save a new file

Perhaps FileSaver.js (blog post) would be suitable

an HTML5 saveAs() FileSaver implementation

TURF: Add simplify operation

Follow these instructions for adding a new turf function.

Turf docs: http://turfjs.org/static/docs/module-turf_simplify.html

I suspect the true/false can be a select object for now!

turn shapefiles into geojson in browser

using https://github.com/calvinmetcalf/shapefile-js

Set `gh-pages` as default branch

I've created a gh-pages branch, making the project available at http://cugos.org/drop-n-chop/. This should be made the default branch (https://github.com/cugos/drop-n-chop/settings), as all PRs should be folded into this branch. Only members of the "Owners" group have the permissions to do this.

Travis PR Builds + Travis GH-Pages Deployment = Security Vulnerability

During the course of reviewing @aaronr's last PR, #48, some issues regarding the new continuous deployment method were exposed.

Backstory:
PR #44 added tooling to run the .travis/deploy-to-gh-pages.sh build script that analyzes if the build is taking place on the master branch and will merge master into the gh-pages branch. To allow Travis to make the merge, it was given a Github OAuth token (stored privately on Travis) with public_repo scope. The logic to determine which branch the build was occurring on was a bit incorrect, causing every build (including PRs) to be deployed to gh-pages. (This was discussed in greater detail in the comments of PR #48, and was fixed with PR #49)

Issue:
What was exposed during that process was that Travis will run whatever code it's given during a PR build. This allows for anyone to make a PR with malicious intent that will be executed with our Github credentials. For example, someone could make a PR to alter .travis/deploy-to-gh-pages.sh so that it erases all the files within the repo, rebases the repo, and then does a forced push to Github, effectively erasing all history. Furthermore, that same auth token could be used on any other public repo that the creator (myself) owns. Since Git is distributed, those changes could be reversed, but it's not a situation we want to get ourselves in.

Suggestions:
@aaronr and I agreed that, for a short term fix, we should disable builds on PRs. This was done that night, immediately after the vulnerability was realized. This means that currently, we must run tests on PRs locally, which isn't ideal.

It doesn't seem like there's any safe way to have the codebase dictate deployment steps if it involves granting every build with Github credentials. Travis doesn't seem to have the ability to manage deployment steps from the web-admin or to only grant Github access tokens to non-PR builds. Travis works really well for notifying us of PR's test results, so I think we should keep with using it for that. I think we're going to have to look elsewhere for the automated deployment step.

Does anyone know of free, hosted tools that will subscribe to a Git repo and can run arbitrary node/bash scripts? I'm going to start going through the links on this list and see which fit.

dist is showing up again, wtf!?

Our dist/ directory is showing up again ... not sure why??

TURF: Add envelope operation

Follow these instructions for adding a new turf function.

Turf docs: http://turfjs.org/static/docs/module-turf_envelope.html

Add a license

Adding a license would validate DnC as open source. But I think all contributors need to agree on which one.

http://choosealicense.com/

Change layer order (z-index)

Clicking and dragging the layers on layerList would be awesome.

TURF: Add destination operation

Follow these instructions for adding a new turf function.

Turf docs: http://turfjs.org/static/docs/module-turf_destination.html

Mv Menu configuration to L.DNC.js

Thinking about the design that came out of #30, I feel that instantiating / configuring the menu within menu/MenuBar.js was a mistake. I'm now thinking that all configuration / setup should be done in a centralized place, such as L.DNC.js. This way, hopefully developers should have less hunting to do when figuring out where elements are coming from.

Need test suite

As the project grows and code changes, it is important to have a set of unit tests ensuring that the system runs as expected.

I am not well versed in JS unit tests, can anyone offer suggestions as to how we should do this? tape seems popular.

set up an http server Grunt task that works with 'watch'

would be nice to get the default grunt command to roll a server and watch for changes in src/. What's the best way to do this?

create icon set for DNC

Having our own icons to use for DNC would be excellent. I'm thinking for now we can do SVG, but eventually turning them into a font would be killer. Some categories/needs for icons:

• Turf.js functions
• UI / Helper buttons
• LayerList & Data type (points vs. line vs. polygon vs. multipolygon)

For now, including icons as <img class="icon" src="icons/turf-buffer.svg"> seems to be good. But would be really awesome to be able to use CSS :before and escaped hex references in content:

Dropzone should be entire screen

Instead of dropzone only being on the left side, make it the entire screen to avoid this:

create script that builds from source and pushes into production

Now that we aren't watching /dist from #34 it would be good to get a shell script or some sort of automated way of building from the master branch and pushing into gh-pages for the production site. This presumably would only work for those who have write access and if they actually ran the script themselves. What does that workflow look like?

Add tooltips

Helpful tips should appear when hovering over Menu operations. The tooltip data could be inputted when instantiating the operation within the menu. In the beginning, we could support only text. In the long run, I'd like to see richly-formatted helptext with the potential to add in illustrative .gif/.webm examples.

This can be done entirely with CSS and HTML, which is nice.

This was mentioned to some extent within #30 (comment)

Bind dropzone to body

Currently, the "dropzone" is only the map area. I have made the mistake of dragging files to the filelist area (which seems like a natural place to drop them to me), which opens the file in my browser rather than loading it into DNC. By binding the dropzone to the entire body, this would be avoided.

include natural earth data sources as example data

Indiemapper (flash) has all of the natural earth data preset so you can add it from their application, rather than needing to download it manually. That seems pretty cool!

Add your own base layer

Now that @lyzidiamond added the feature to switch base layers, it would be awesome to implement a custom layer. This would essentially allow the user to input their own Mapbox tile ID and key?

@lyzidiamond do you know if this would require a key or would the ID be sufficient?

Allow for featureCollections to be run recursively

If someone runs bezier, for example, on a featureCollection of more than one feature and turf requires only a single "feature", what happens? We could help the user out here by recursively running along each feature and returning a new feature collection of updated features. This can be an addition to GeoExecute.js

Remove References from Unrelated Components

I would appreciate some feedback about the ideas I mention below and some of the solutions the commits tied to this ticket propose.

The App/Controller Idea

Originally the MapView class was called something like DncApp and it's general purpose was to glue and setup things:

• setup and bind components being used in DNC, like the map
• setup component event listeners, like for FileReader event handlers
• it was a place for slim applciation logic, but not component specific stuff

When we decided that MapView felt like it was it's own component, things got shuffled around and some glue stuff didn't make sense to have in that class anymore.

I've noticed that some of the glue logic was moved out into L.DNC where it currently lives. My guess is that we can expect that glue to grow in the future. L.DNC really feels like a simple namespace object for things to hang off, but not it's own class. Maybe that's just me.

More importantly, another side effect of the refactor was the propagation of some patterns that I feel like we should avoid:

• FileReader now reaches outside of it's scope globally to access other components here.
• The TurfOperation also reaches outside of it's scope to add layers here.
• Even more confusing, the FileReader component fires an event that it itself listens to here.

Proposal

It feels like we need a thing that does the glueing and setup again. This thing, lets call it a controller, would also hold component references so we don't force components that shouldn't know about each other to tightly reference each other.

What Changed?

• I managed to remove all these type of references except one, which I flagged as a possible removal later.
• For lack of a better term ( i'm open to anything ), I called the controller AppController. See the class here.

TURF: Add bezier operation

Follow these instructions for adding a new turf function.

Turf docs: http://turfjs.org/static/docs/module-turf_bezier.html

TURF: Add midpoint operation

Follow these instructions for adding a new turf function.

Turf docs: http://turfjs.org/static/docs/module-turf_midpoint.html

TURF: Add explode operation

Follow these instructions for adding a new turf function.

Turf docs: http://turfjs.org/static/docs/module-turf_explode.html

CUGOS Spring Fling "alpha"

The CUGOS Spring Fling is coming up in a few weeks, and DNC is a big piece to the program for the day. @powersa has us cranking on DNC for a whole afternoon with a good number of people. That means we have to make sure this thing works well - even if it's not at 100% functionality.

@alukach @thebigspoon what do you think are the most important features to finish up by then? My immediate reaction:

• prepare the menu space to take in all of the turf functions - this would be a great day to have people adding those in! Blockers: #5, #40
• establish a proper pattern for adding <form> elements for these turf operations. I'm not exactly sure how we go about doing this best, but it seems important to move forward. Exists in #2 already
• Style some basemaps? Choose a basemap? Thinking of things that someone without programming experience could work on and potentially choose good basemaps for the data.
• Clear documentation for install, #47
• SUPER clear documentation on the architecture of the project (can add to wiki)
• Establish a proper workflow for people, including for ourselves to test forks properly and add tests
• what else?

I think @thebigspoon will be there to help people out, but I'm a little worried that we're not going to be in a place to have 15 (total guess) people working on this project (that's insane, right?). @powersa you mentioned this will likely split into multiple groups working on stuff. Any idea what you expect those to be?

/dist conflicts ALL THE TIME

Is it just me or do we keep getting stupid branch conflicts because of the /dist builds changing all the time? Wondering if it's worth removing /dist in the .gitignore ...

Make ability to have grunt build release and dev instances of project

There is no current way (other than modifying a tracked index.html file) to change to use non-minified versions of the code. Also the index.html file is not placed in the dist dir like it should. I understand that there where reasons to do this... but there must be a better way:

1. Have index.html end up in dist like the rest of the project when built.

2. Change the gh-pages stuff to just publish the contents of dist.

• This gets rid of all the extra stuff that is being published that should not be (config files etc)
• Everything is at the right level then (i.e. index.html is no longer treated special)

3. Make index.html based on a template in the root folder, but use a grunt html processor to build out the real index.html based on build type. Some possible grunt tools already exist:

• https://github.com/spatools/grunt-html-build
• https://github.com/dciccale/grunt-processhtml

4. Make grunt serve and the connect stuff for grunt serve from the dist dir.

In the end you would be able to specify build type... each would pre-process the index.html to the proper form, populate the dist dir, and if updating master push a release version of the dist dir to gh-pages.

I plan to work on prototyping this (as a learning exercise for myself) in the aaronr_gruntwork branch

Layers are starting to generate on layerList out of nowhere after operations complete

Look at the layers that come out of the union in this gif:

sf.geojson and east.geojson are duplicated. Wtf!? Probably a result of some of the work on #63

Create view for geoprocessing input information

For example, when someone clicks "buffer" they get a screen asking for specific geometries. I'm unsure how to build this one right now. I wonder what the best way to do it is?

write instructions for installing project for PC/Linux

https://github.com/cugos/drop-n-chop/wiki/Installing-the-Project only exists as Mac

Update readme gif

Because those example bars bother me ;)

Need test data

Somewhat related to #22, it seems important to have a set of known supported files. These should be of various file types (JSON, KML, Shapefile) and data-types (Features, FeatureCollections, Point, LineString, Polygon, MultiPoint, MultiLineString, MultiPolygon).

TURF: Add tin operation

Follow these instructions for adding a new turf function.

Turf docs: http://turfjs.org/static/docs/module-turf_tin.html

The propertyName parameter can be avoided for now. Maybe someone can make another issue that deals with selecting available attributes from the data.

When buffering multiple geometries, get a TypeError

Uncaught TypeError: Cannot set property 'properties' of undefined

Build error handler and notification system

I'd like to start finding and standardizing potential errors that a user could run into (i.e. "incorrect file type", or "cannot union these two geometries"). I think logging errors is fine right away, but I'd like to connect it to a notification system within the application that is used for alerts/successes/etc.

There are some libraries that do this, such as notify.js. Honestly, I think I might roll our own because this seems like a fantastic UI exercise.

fitBounds should only run when the user is manually adding a new file

This seems a bit UI-unfriendly since you won't want to zoom back to the extent you're working at every time you run an operation. Workflow gif below:

Some potential options for fixing this:

1. Don't zoom at all, unless the new file has been added manually by the user
2. Zoom to the extent of the new feature, rather than the entire map

Let's choose a javascript spec and set some options in the linter

such as 2 space indents, no semi-colons, for/length prioritized over for/in, etc.

TURF: Add center operation

Follow these instructions for adding a new turf function.

Turf docs: http://turfjs.org/static/docs/module-turf_center.html

TURF: Add along operation

Follow these instructions for adding a new turf function.

Turf docs: http://turfjs.org/static/docs/module-turf_along.html