data-skeptic/home-data-gallery

A place for people to send pull requests for interesting examples they'd like to share

home-data-gallery's Issues

Remove the address search field until the API supports it

Standardize indends on the code

Right now it's a mix of tabs and spaces. Let's standardize (begrudgingly) on spaces. It would also be great if some sort of linter could check for this.

This change will touch a lot of files. Please ensure that you make NO LOGICAL code changes. It would be difficult to review the code since so many lines will be different in the diff. This pull request that satisfies this task should only convert the tabs to spaces.

Add spinner image on data refresh

When the input fields are changed, and the API is called again, only the plots page has a waiting spinner. Need to add that to map and to data table.

Initialize map to user's local area

Right now, code.js assigns a new user to be centered on the continental united states at a large zoom.

It would be preferred if (when possible) the user's actual locale was detected and the zoom and center was picked appropriate to the user for a better experience.

Add rollover info option to parsing page

http://openhouseproject.co/create-parse.html

Add a rollover with the text “Your email will be kept private but we may want to reach out with questions or to show our appreciation”

Store API results in K-D tree

Per @stweinn and @joytafty 's discussion

Create header React component

The header should include the logo, selectors, and curlBox from the existing codebase seen here:

https://github.com/data-skeptic/home-data-gallery/blob/master/contingency-plots/index.htm

Essentially, this task is about taking

and React-ifying it.

Feel free to take some design liberties if you have nice ideas!

Migrate slider.js to React component

Convert this existing function to a React component:

https://github.com/data-skeptic/home-data-gallery/blob/master/contingency-plots/scripts/slider.js

Fix Raw Data table update issue

Presently, this tab is not updating when new data comes back from the API

Asynchronously get "next" data

When the API is called, presently, a limit of 500 is used, and the API is called only once. Thus, a user only sees the first 500 uploads matching their criteria.

It would be preferable for this to happen:

1. Show indicator of waiting for API on first search (currently showing the box.png image)

2. When API returns, hide box.png and display results

3. If the API has no more results stop

4. Show some indicator different from box.png indicating that more results are being retrieved

5. Asynchronously call API to get more results

6. Go to step 3

Reorganize parsing page into 3 sections

http://openhouseproject.co/create-parse.html

First a header with logo, instructions, and skip button to a header. Eventually, we will have a login option here as well. When the user scrolls, this should minimize. Then allow user to choose between vertical or horizontal split screen of code / iframe.

Modernize email input box

http://openhouseproject.co/create-parse.html

Place the “ghost” text “Enter your email in the form and have it disappear when the user gives focus to the input box.

Header disappears in data table on refresh

Improve the heatmap visualization

https://github.com/data-skeptic/home-data-gallery/blob/master/contingency-plots/heatmap.htm

• Provide axis labels.
• Better color choice
• Legend for how to interpret color
• Change to probabilities, and maybe have the frequency appear on hover

Update README.md

Describe the purpose and goal fo the repo.

localStorage I/O

Improve the scatterplot visualization

https://github.com/data-skeptic/home-data-gallery/blob/master/contingency-plots/scatterplot.htm

• More attractive CSS; better line widths, for example
• Better tick frequency; show min and max always; not too many ticks

Update travis to deploy webpack

Improve the histogram visualization code

https://github.com/data-skeptic/home-data-gallery/blob/master/contingency-plots/histogram.htm

• Work on axis labels
• Include unknown (null or -1) as distinct column
• Thinner axis (via CSS?)

Create footer React component

Include information about our data license, which is "CC BY SA". There's an official logo we should include and link to an explanation page, similar to what I have on the footer of dataskeptic.com

Footer should have links to github repos as well, and back to the main domain: openhouseproject.co

add cleanup on localStorageFull

Add map marker popup

Presently, the OH map shows markers for where listings are, but nothing is done on the map with those markers when you click on them.

Please add a pop up window with some quick details on the property clicked.

Create "no results" React component

localStorage eviction

The amount of space that can be used in localStorage is limited. If a QuotaException is thrown, do smart eviction.

Currently, the API returns the `raw_address` value and `address_object.raw`. These fields are expensive and redundant thing to include. Let's remove them from API responses.

React.js migration

Following the excellent hierarchy created by @joytafty, let's convert the gallery portion of the site to be React based.

Feel free to break this into multiple tickets if you think it's too monolithic. Identifying distinct subsets could allow us to chunk off work and get more people involved if you see clean opportunities.

Migrate map.js to React component

Integrate with r-fiddle.org

There's a great website called http://www.r-fiddle.org/ that lets you do quick interactions in R (similar to JS Fiddle).

We currently have a "copy to clipboard" option in the header of the Gallery which allows you to get a curl request which returns the data shown in the UI.

Please add another button that opens a new browser tab and provides some starter code in R that calls our API and returns a data.table.

This could be a great way for an R user to be working with OpenHouse data within seconds of visiting our site.

Add convenient copy to clipboard option on parsing page

http://openhouseproject.co/create-parse.html

When the code window contains text, allow the user to hit a button to copy it to their clipboard. This functionality already exists in the OpenHouse Gallery. You can copy the javascript code from there if you need inspiration on how to do it.

Suggest a Javascript unit test framework

Presently we have a home grown method of doing unit tests for the front end. It's more of a quasi-unit test. The approach we have is useful and relatively painless, but it lacks a certain formality that you typically have with unit tests.

Our process today:

Create two javascript files. For example: localStorageIO.js and localStorageIO_test.js. Code goes in the first, tests in the second.

Ok, so how do these tests get run? Add them to tests.htm, which calls each one and reports the results.

Are there better frameworks that could also do things like code coverage and ru-running broken tests, etc?

Make our parsing page work like PythonAnywhere, allowing the user to develop bsoup parsing code in one simple interface. http://openhouseproject.co/create-parse.html

Research S3 auto-expire

Our crawling system uses S3 for storage

S3 offers some functionality to have objects expire. Our code sets expiration times for all objects, but it enforces these manually.

We are using the Python boto library do interact with S3.

Please research how to use S3's native expiration functionality. Specifically: how can we use boto to set an expiration and have AWS do the deletions for us?

This may help get you started?

https://aws.amazon.com/blogs/aws/amazon-s3-object-expiration/

Develop a Slack bot that posts to the #openhouse channel every morning letting people know what work needs to be done in our web interfaces

Randomize API response with a time based seed

Currently, the API returns results in an arbitrary order. This can introduce a bias in results. We should take the minute of the hour and use it to seed a random number generator. We should then use that seed to randomize. This way, continuously refreshing won't make every result random (only once per minute should anything change)

Responsible design / mobile friendly research

What do we need to do to get the site to be mobile friendly? Whoever takes up this task should review the codebase and come back with suggestions. It's not required that you implement them (although you are certainly welcome to!), just share with the group what you learned so we can create tasks to implement your plan.

Need code optimization in map updates

The function updateMapAfterViewportChange() in code.js can be very slow. If you zoom out to see the entire US, this takes almost a full 1 second to load. While this might sound fast, it makes the user interface feel VERY clunky.

The problem probably lies in the way we update the map. Presently, regardless of how many markers are on the map, we remove all the markers, and then add all the new ones. I suspect if you only add the markers not already on the map, that would solve it.

Repo clean up and restructure

The Gallery component has evolved a lot and has some legacy baggage. Let's get ride of the "contingency-plots" folder and migrate the React.js code base to the top level of the repo.