heavyai/heavyai-tweetmap-2

Play with and instantly visualize hundreds of millions of realtime tweets, from the global level all the way down to your neighborhood.

heavyai

This package enables using common Python data science toolkits with HeavyDB. It brings data frame support on CPU and GPU as well as support for arrow. See the documentation for more.

Quick Install (CPU)

Packages are available on conda-forge and PyPI:

# using conda-forge
conda install -c conda-forge heavyai

# using pip
pip install heavyai

Quick Install (GPU)

We recommend creating a fresh conda 3.8 or 3.9 environment when installing heavyai with GPU capabilities.

To install heavyai for GPU Dataframe support (conda-only):

mamba create -n heavyai-gpu -c rapidsai -c nvidia -c conda-forge -c defaults \
    --no-channel-priority \
    cudf heavyai pyheavydb pytest shapely geopandas pyarrow=*=*cuda

Documentation

Further documentation for heavyai usage is available at: http://heavyai.readthedocs.io/

heavyai-tweetmap-2's Issues

No fetching in components

https://github.com/mapd/mapd-tweetmap-2/blob/master/src/containers/ShareMenu.js#L34

Components shouldn't be handling network requests - this should instead be happening via an action. The action can dispatch further actions within the fetch logic, which also prevents this container from having to manage its own state.

Don't issue time or hashtag queries when those charts are minimized

As a user I should only pay a performance hit on charts I am actively looking at. In particular, the hashtag query is currently expensive as the Twitter dataset gets large (think > 1B rows) and so it would be nice for demo scenarios to be able to use a bigger dataset by collapsing the hashtag chart. Also on mobile the hashtag chart is not shown by default so its even a more common scenario of query wastage.

Search by location does nothing

🪲 If bug

• Operating System and version: OSX 10.14
• Browser and version: All browsers
• Steps to reproduce (including dashboard link): Search for location
• Description of issue:
  Search by location does nothing currently due to Google now requiring a Google Maps Javascript API key

Media Query component

I'm curious about the media query component we're using. What purpose is it serving? I skimmed through the documentation, and I can't see how it's doing anything more useful than plain old media queries in CSS - if nothing else, I'm worried that it bloats JSX code and muddles the separation of concerns between HTML, CSS, and JS.

Mind you, I get that JSX is already kind of a gray area (are we writing our HTML in JS? What is JSX?) But using a media query component is like a melange of all three - a JS component that renders HTML that has CSS implications.

Point and popup not following map on pan

🪲 If bug

• Operating System and version: iOS 11
• Browser and version: Safari
• Steps to reproduce (including dashboard link):
• Description of issue:

In mobile, selecting a point on the tweetmap and then panning results in the point and popup staying fixed in space, rather than moving along with the map.

map doesn't load

Allthe components load except the map. I get this error: "TypeError cannot read property getTable of null"

Sharing functionality broken

🪲 If bug

• Operating System and version: OSX 10.14
• Browser and version: All major browsers
• Steps to reproduce (including dashboard link): Click share icon. Nothing is produced in the text box. Click FaceBook or Twitter share button and notice that the information generated to share is incorrect for both. Errors in console regarding API request to http://external-apis.mapd.com/shorten
• Description of issue:
  A few things are going on here.

1. No twitter filters are captured from the map.
2. The description captured for Twitter doesn't generate a shareable link
3. The Facebook link is incorrect.

I believe the underlying issue is that the POST request to http://external-apis.mapd.com/shorten is not being completed and thus a shareable url is not created.

Switch favicon to OmniSci

Current favicon is still MapD logo

🪲 If bug

• Operating System and version:
• Browser and version:
• Steps to reproduce (including dashboard link):
• Description of issue:

🆕 If feature request

• User story:
• Specific requirements/functionality:
• Mocks (if applicable):

Need Mapbox attribution on basemap

See www.mapd.com/demos/tweetmap for an example (pretty small there, not sure if that complies with their guidelines). May need something more like we have in Immerse below:

Unclear state management

Currently, we have a few different types of state management going on. In some components, state is updated locally using setState, but others are properly connected to the redux store. Let's use redux across the application. I'll point out some examples here soon.

Count Widget

Todd mentioned that the count widget is very helpful during demos and sales pitches. We'll want to incorporate that back into Tweetmap somehow.

Unclear PropTypes

I just added a linting rule that forbids array and object proptypes - instead, let's use shape and arrayOf to make it clearer what props are being passed around. More on this rule here:

jsx-eslint/eslint-plugin-react#215

Rendering in process circle spinner mangled

Hard to capture this in a screenshot, but when switching between pointmap and heatmap, if the backend needs a few seconds and the spinning circle appears, it's not an actual circle but rather a mismatched on (i.e the line segments for each half-circle don't touch, looks like a flat tire)

🪲 If bug

• Operating System and version:
• Browser and version:
• Steps to reproduce (including dashboard link):
• Description of issue:

🆕 If feature request

• User story:
• Specific requirements/functionality:
• Mocks (if applicable):

Dates stays in horizontal position on Safari

🪲 If bug

• Operating System and version: OX 10.13

• Browser and version: Safari 11.0

• Steps to reproduce (including dashboard link):
  Create a brush filter

• Description of issue:
  *First great job it looks amazing and very smooth.

The date should be vertical on small range and horizontal on larger range.
The container for the date goes horizontal but the date stays vertical.
Then it goes crazy when you move the filter all the way to the right. (this last behavior happens on all browsers.)

Update dependencies to newest working versions

In merging #32, dependencies are pinned on some very old versions of npm packages. Specifically, mapd-connector is pinned to right before v3.0.0, which had breaking changes. mapd-connector at the time of this issue is at version 4.2.0, indicating that it's likely that additional breaking changes have occurred.

In evaluating what it would take to update Tweetmap 2 to newest versions, there are a handful of high-level projects:

• Update mapd-connector and related libraries to their newest versions
• Updating React from v15 to v16
• Updating webpack from v2 to v4
• Update other dependencies for security and/or functionality improvements

Using yarn upgrade-interactive, many updates to other dependencies could be done without the major overhauls identified in the first 3 bullet points above.

Source option on legend/pointmap

Allow the option of coloring the point map by device source and adjust the legend to match

Time Labels on line chart

User story:

Currently selecting a portion of the line chart does not convey which section of time has been selected. There should be special labels on the brush similar to old tweetmap the say what dates/times the markers are at.

Specific requirements/functionality:

Extend line chart to display the time range that has been selected on the brush

Mocks (if applicable):

Current brush:

We desire a brush similar to old tweetmap:

File structure improvements

Currently, everything is organized by type, but it feels inconsistent. Containers aren't always containers, and the relationship between container and component isn't always abundantly clear.

Check out this article: https://marmelab.com/blog/2015/12/17/react-directory-structure.html

I like the idea of organizing by domain, and seeing where that takes us. This can also help guide our testing efforts. (:

Automated Tests

We'll need to get some automated tests up and running so that we know Tweetmap isn't breaking as the libraries update. Let's talk to Mru about how automated tests work with our demos and get tweetmap set up similarly.

publish blog post!

post body is found here:

https://gist.github.com/edwickable/422445ae3eda78425ab415cef0213a4c

JSON CSS vs. CSS Modules vs. SASS vs. React Libraries?

I'm not a huge fan of how we're handling CSS in the tweetmap right now. I like the fact that we're using JSON in places, because it avoids issues with global scope and cascades, but it's incredibly repetitive and clutters up each component. It's particularly confusing, though, because sometimes we have inline JSON for styling, sometimes there are ids/classes that are tied to SCSS files... It's a lot to keep track of.

For something like tweetmap, I think that extracting the CSS into CSS modules could make a lot of sense. That being said, if we stick to the SCSS structure, I think that's fine as well. Let's just commit to it.

PS: fan of the BEM that seems to be going on. (:

Toolbar with go to location and zoom out

User story:

from an information architecture point of view, the "go to location" and "zoom out" functionalities in the nav don't fit in with the others. With consideration of the future arrival of choropleth and heat map functionalities, there should be a tool bar on the map that contains these functionalities.

Specific requirements/functionality:

The "go to location" and "zoom out" button should be gone from the nav. There should a new component floating right and center with "go to location" and "zoom out." Should easily allow extending with choropleth and heat map