Giter Site home page Giter Site logo

3rww / rainfall Goto Github PK

View Code? Open in Web Editor NEW
4.0 2.0 2.0 36.61 MB

Search, visualize, and download real-time and historic hyper-local rainfall data in Allegheny County, PA

Home Page: https://3rww.github.io/rainfall/

License: Apache License 2.0

JavaScript 96.85% HTML 0.93% Shell 0.09% SCSS 2.12%
rainfall gauges radar calibrated historic visualization animation spatial temporal geodata

rainfall's Introduction

3RWW Rainfall beta

Have you ever wondered how rainfall is actually measured? Technical instruments, called rain gauges, are designed to collect and accurately measure rainfall during wet weather events. However, a rain gauge can only provide a specific rainfall measurement for the limited geographic area where the gauge is located.

On the flip side, radar systems, often used in weather reports, do not measure rainfall directly, but rather they detect the intensity of microwave energy reflected by raindrops, called reflectivity. Through the use of a mathematical formula, the reflectivity of the raindrops can be converted by the radar system into rainfall estimates for a particular defined area.

Neither measurement technique is perfect, but when the two are combined—when radar estimates are calibrated with actual rain gauge data—a highly accurate and valuable source of rainfall data can be calculated over large geographic areas.

Because engineers and planners addressing the wet weather issue need this level of accuracy, 3 Rivers Wet Weather created the calibrated radar rainfall system in 2001. Communities throughout Allegheny County use this data—provided in both real-time and historical formats—to design more cost-effective solutions to reduce or eliminate sewage overflows and improve stormwater management.

The NEXRAD radar (located in Moon Township) data is calibrated with the rain gauge measurements collected during the same time period and rain event for every square kilometer in Allegheny County. The resulting rainfall data is equivalent in accuracy to having 2,276 rain gauges placed across the County.

Functionality

Currently, this site offers search, download, and mapping capability for rainfall data in three buckets:

  • Real-time Rainfall data: provisional rainfall data for both rain gauges and radar pixels. Real-time gauge data is sourced from Datawise,and pixels are sourced from Vieux Associates. This data is updated every 15 minutes.
  • Historical Rain Gauge data: QA/QC'd rain gauge data, usually available within 30-60 days. QA/QC is performed by ALCOSAN and 3RWW.
  • Calibrated Radar Rainfall data: QA/QC'd, gauge-adjusted radar rainfall observations, typically available within 30-60 days. Calibration is performed by Vieux Associates.

Powered By The 3RWW Data API

The rainfall data is served up from 3RWW's Data API. Currently a few low-level API functions are documented and accessible through https://trwwapi.herokuapp.com/rainfall/.

Status

This project is a work-in-progress. We're currently looking to replicate the workflows of the original rainfall downloads site. In the future and with feedback, we'll build out new ways to explore and view the rainfall data. We'll also provide resources for helping you integrate this data into your workflows and use it in your tools.

Some notes on development

This project was bootstrapped with Create React App.

Use Node NPM, Yarn, or PNPM to install dependencies. While we use PNPM, the commands shown the sections below are the same with any of those tools.

NOTE: dependencies as spec'd are currently only building with Node V16.

Clone and Installation

Run git clone https://github.com/3rww/rainfall.git to clone this codebase.

Running pnpm install will install dependencies.

Available Scripts

In the project directory, you can run:

pnpm start

Runs the app in the development mode.
Open http://localhost:3000 to view it in the browser.

The page will reload if you make edits.
You will also see any lint errors in the console.

pnpm test

Launches the test runner in the interactive watch mode.
See the section about running tests for more information.

pnpm build

Builds the app for production to the build folder.
It correctly bundles React in production mode and optimizes the build for the best performance.

The build is minified and the filenames include the hashes.
Your app is ready to be deployed!

pnpm serve

Locally serves the built site http://localhost:3000.

pnpm deploy

Runs the build script and commits the changes the gh-pages branch, and pushes it to GitHub pages.

pnpm eject

Note: this is a one-way operation. Once you eject, you can’t go back!

If you aren’t satisfied with the build tool and configuration choices, you can eject at any time. This command will remove the single build dependency from your project.

Instead, it will copy all the configuration files and the transitive dependencies (Webpack, Babel, ESLint, etc) right into your project so you have full control over them. All of the commands except eject will still work, but they will point to the copied scripts so you can tweak them. At this point you’re on your own.

You don’t have to ever use eject. The curated feature set is suitable for small and middle deployments, and you shouldn’t feel obligated to use this feature. However we understand that this tool wouldn’t be useful if you couldn’t customize it when you are ready for it.

Learn More

You can learn more in the Create React App documentation.

To learn React, check out the React documentation.

rainfall's People

Contributors

gassc avatar

Stargazers

 avatar  avatar  avatar

Watchers

 avatar  avatar

rainfall's Issues

Mobile adaptivity

Need to update the layout classes to adjust for appropriate screen breakpoints.

Check on RTRR vs RTRG timestamp conversion

Timestamps are showing a 3 hour difference between rain gauges and radar pixels (real-time data only).

image

This is likely an error in the timezone conversion on the real-time gauge data (it comes in as PST).

Historic rainfall charts

On map hover or click, in a map pop-up, show charts for:

  • Year-to-date
  • Annual averages for last 5 10, and 20 years
  • Amount for last recorded rainfall event
  • ???

Having any of this summary data at the ready will require

  • some new API endpoints,
  • semi-regularly updated materialized views in the database, since the queries behind these stats would be massive

decimal places

Investigate results with many, many decimal places (many zeroes). Shouldn't be more than what Datawise and Vieux show. This may be happening in the API, but at the very least we should truncate for download.

rainfall events search - dates, locations, characteristics, names

A key feature of this viewer is providing shortcuts to date-time ranges of known rainfall events. This information comes from Vieux (we just need to pull it into the database and provide through rainfall-api.

From that data, and with other data attributes on storm events (e.g., we know the name of the storm was Hurricane Ivan, or we know there was flooding reported), have text-based and visual ways to search and select events (e.g. timeline).

Legend w/ dynamic symbology updater

Create a map legend component; Control it with preset scale options and maybe a few color options--control in the nav bar.

Consider being able to switch from showing rainfall totals to DDF.

Realtime rainfall animations+playback

  • Mimic playback presets available in the legacy site
  • Include playback controls
  • Have a "quick" download option available
  • Auto-request this data with the page load
  • Make a killer animation!

Realtime rainfall charts

On hover or click, withing a map pop-up, show a chart for:

  • rainfall over the last 12 hours
  • ???

Re-shapeable results table

Enable the ability to convert from the current long format to a wide format, where sensors and metadata are in columns (like the legacy site)

Active vs. inactive gauges

Need to make the distinction between active and inactive gauges in the various download contexts.

  • Real-time rainfall should only be able to download from active gauges.
  • Historic should handle both active and inactive (because we have data for previous gauges).
  • The map symbology and labeling should make it clear what is active/inactive.

Excel-compatible timestamp formatting for rainfall results

Currently, timestamps (including timezone offset) are returned using the RFC 3339 standard (ISO-8061 format). Excel doesn't automatically parse this out, requiring an extra step for end-users of the CSV download. We can return a spec that Excel will automatically parse.

3D results

Get those rainfall results showing in 3D! Need to work-around the transparency issue for pixels with no-data (simply by eliminating those geometries for results, which means separating the results and hover layers in the stylesheet)

Longer term: consider using color and height to represent different things: e.g., DDF for a default return period (or a comparable one if we have an event on record) for color, and height for rainfall

Update rain gauge and pixel sources

Two gauges were moved this spring.

  • Bell Acres >>> West Deer
  • Pitt >>> Schenley

This has been updated in the 3RWW API. However, this app uses static copies of that data.

Change that so we use the dynamic sources.

new "Explorer" tab

Create a new context/view that is:

  • unifies the workflow for provisional and qa/qc'd data; a "best of"
  • puts the rainfall events front-and-center
  • emphasizes visualization of stats between sensors and over time
  • enables comparison and diffing between different events.
  • by default has no geographic constraints: all gauges, all pixels

This would use a lot of the existing and planned components under the hood, but re-arrange them to support a new, exploratory workflow.

This could also possibly be done (or at least prototyped) in a Dash app.

Question about the Calibrated Radar Rainfall Data

Hello Christian,
How are you.
Very pleased to read about your work, very fascinating. I have a question, how do you correct the radar data? Is there a description of the method, or a relevant published article?

best wishes,
Xing WANG

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.