Giter Site home page Giter Site logo

joannalcy / electricitymap-contrib Goto Github PK

View Code? Open in Web Editor NEW

This project forked from electricitymaps/electricitymaps-contrib

0.0 0.0 0.0 77.85 MB

A real-time visualisation of the CO2 emissions of electricity consumption

Home Page: https://www.electricitymap.org

License: MIT License

HTML 10.87% Python 58.49% JavaScript 26.43% CSS 3.50% Jupyter Notebook 0.36% Shell 0.28% Dockerfile 0.07%

electricitymap-contrib's Introduction

electricityMap Slack Status CircleCI Twitter Follow

A real-time visualisation of the Greenhouse Gas (in terms of CO2 equivalent) footprint of electricity consumption built with d3.js and mapbox GL, maintained by Tomorrow. Try it out at http://www.electricitymap.org, or download the app:

Get it on Google Play Get it on the Apple Store

image

You can contribute in the following ways:

You can also find a list of missing data displayed as warnings in the developer console, or question marks in the country panel:

image

Check the contributing section for more details. Join us on Slack if you wish to discuss development or need help to get started.

Frequently asked questions

How do you define real-time data? Real-time data is defined as a data source with an hourly (or better) frequency, delayed by less than 2hrs. It should provide a breakdown by generation type. Often fossil fuel generation (coal/gas/oil) is combined under a single heading like 'thermal' or 'conventional', this is not a problem.

Why do you calculate the carbon intensity of consumption? In short, citizens should not be responsible for the emissions associated with all the products they export, but only for what they consume. Consumption-based accounting (CBA) is a very important aspect of climate policy, and allows to assign responsibility to consumers instead of producers. Furthermore, this method is robust to governments relocating dirty production to neighbouring countries in order to green their image while still importing from it. We published our methodology here.

Why don't you show emissions per capita? A country that has few inhabitants but a lot of factories will appear high on CO2/capita. This means you can "trick" the numbers by moving your factory abroad and import the produced good instead of the electricity itself. That country now has a low CO2/capita number because we only count CO2 for electricity (not for imported/exported goods). The CO2/capita metric, by involving the size of the population, and by not integrating all CO2 emission sources, is thus an incomplete metric. CO2 intensity on the other hand only describes where is the best place to put that factory (and when it is best to use electricity), enabling proper decisions.

CO2 emission factors look high — what do they cover exactly? The carbon intensity of each type of power plant takes into account emissions arising from the whole life cycle of the plant (construction, fuel production, operational emissions, and decommissioning).

Is delayed data useful? While the map relies on having real-time data to work it's still useful to collect data from days/months past. This older data can be used to show past emissions and build up a better dataset. So if there's an hourly data source that lags several days behind you can still build a parser for it.

Can scheduled/assumed generation data be used? The electricityMap doesn't use scheduled generation data or make assumptions about unknown fuel mixes. This is to avoid introducing extra uncertainty into emissions calculations.

Can areas other than countries be shown? Yes providing there is a valid GeoJSON geometry (or another format that can be converted) for the area. As an example we already split several countries into states and grid regions.

How can I get access to historical data or the live API? All this and more can be found here.

Data sources

Carbon intensity calculation and data source

The carbon intensity of each country is measured from the perspective of a consumer. It represents the greenhouse gas footprint of 1 kWh consumed inside a given country. The footprint is measured in gCO2eq (grams CO2 equivalent), meaning each greenhouse gas is converted to its CO2 equivalent in terms of global warming potential over 100 year (for instance, 1 gram of methane emitted has the same global warming impact during 100 years as ~34 grams of CO2 over the same period).

The carbon intensity of each type of power plant takes into account emissions arising from the whole life cycle of the plant (construction, fuel production, operational emissions, and decommissioning). Carbon-intensity factors used in the map are detailed in co2eq_parameters.json. These numbers come mostly from the following scientific peer reviewed literature: IPCC (2014) Fifth Assessment Report is used as reference in most instances (see a summary in the wikipedia entry)

Each country has a CO2 mass flow that depends on neighbouring countries. In order to determine the carbon footprint of each country, the set of coupled CO2 mass flow balance equations of each countries must be solved simultaneously. This is done by solving the linear system of equations defining the network of greenhouse gas exchanges. Take a look at this notebook for a deeper explanation. We also published our methodology here.

Real-time electricity data sources

Real-time electricity data is obtained using parsers.  

Click to see the full list of sources

Production capacity data sources

Production capacities are centralized in the zones.json file. Values in the capacity maps are in MW.  

Click to see the full list of sources

Cross-border transmission capacity data sources

Cross-border transmission capacities between the zones are centralized in the exchanges.json file. Values in the capacity maps are in MW.  

Click to see the full list of sources

A ⇄ B: bidirectional operation, with power flow either "from A to B" or "from B to A"

A ⇉ B: unidirectional operation, only with power flow "from A to B"  

Electricity prices (day-ahead) data sources

Real-time weather data sources

We use the US National Weather Service's Global Forecast System (GFS)'s GFS 0.25 Degree Hourly data. Forecasts are made every 6 hours, with a 1 hour time step. The values extracted are wind speed and direction at 10m altitude, and ground solar irradiance (DSWRF - Downward Short-Wave Radiation Flux), which takes into account cloud coverage. In order to obtain an estimate of those values at current time, an interpolation is made between two forecasts (the one at the beginning of the hour, and the one at the end of the hour).

Topology data

We use the Natural Earth Data Cultural Vectors country subdivisions (map admin subunits).

Contribute

Want to help? First of all, thank you for your interest!

Join us on Slack at http://slack.tmrow.co to see what we are working on and to get in touch with other contributors. Also, if you get stuck, you can write a message in our channel #electricitymap.

For different ways to help improve electricityMap, have a look at the overview above. Here is how to get started:

Steps to making a code contribution

Follow these steps to make your contribution:

  1. Fork the repository.
  2. Clone your fork of the repository.
  3. Download Docker.
  4. Set up and start your local environment (see steps below).
  5. Make your code changes and test them in your local environment.
  6. Push your changes to your fork.
  7. Submit a pull request to bring your contribution into the production version.

Set up and start your local environment

Note: to test parsers, go to testing parsers locally

  1. First, you need to compile the frontend. Open a terminal in the root directory and run:

    docker-compose build

  2. Start the application by running:

    docker-compose up

    This will watch over source file changes, run nonstop and watch changes you make in the code to recompile the frontend if needed.

  3. Go to http://localhost:8000/ and you should now see the map!

Notes:

  • These steps only build with the English language (which will be faster as not all languages need to be built). To build all languages, change the command of the web-watch-en section of docker-compose.yml from command: npm run watch-en to command: npm run watch.
  • The backend handles the calculation of carbon emissions. The map data displayed comes from a mock server providing dummy data from the state file.

See Troubleshooting below for common issues and fixes when building the map locally.

Logger

If you want to add new or change existing parsers you can use our public logger to see if your parser(s) can fetch data successfully. The logs show warnings and errors for all parsers.

Update region capacities

If you want to update or add production capacities for a region, go to the zones file and make changes to the capacity map. Values are in MW. The zones use ISO 3166-1 codes as identifiers.

Add a new region

As a first step, do a search for the region on our GitHub, as contributors may have explored things before. It is very simple to add a new country. The electricityMap backend runs a list of so-called parsers every 5 minutes. Those parsers are responsible for fetching the generation mix of a given country. See the existing list in the parsers directory, or review work in progress parsers).

I have not found a new data source

Open a new issue for the relevant region and add the relevant individual or organization to contact. Please include email, Twitter and phone number for the contact where available.

This helps us and the community to contact them and make them aware of the electricityMap. Usually, energy agencies, governments, and transmission system operators are good potential sources. If you can't code, this is an amazing way to help us!

I have found a data source and I could help build a parser

A parser is a python3 script that defines the method fetch_production and returns the production mix at current time, in this format:

def fetch_production(zone_key='FR', session=None, target_datetime=None, logger=None):
    return {
      'zoneKey': 'FR',
      'datetime': '2017-01-01T00:00:00Z',
      'production': {
          'biomass': 0.0,
          'coal': 0.0,
          'gas': 0.0,
          'hydro': 0.0,
          'nuclear': None,
          'oil': 0.0,
          'solar': 0.0,
          'wind': 0.0,
          'geothermal': 0.0,
          'unknown': 0.0
      },
      'storage': {
          'hydro': -10.0,
      },
      'source': 'mysource.com'
    }

It contains the following objects:

  • session: a python request session that you can re-use to make HTTP requests.
  • target_datetime: used to fetch historical data when available.
  • logger: a logging.Logger whose output is publicly available for anyone to monitor correct functioning of the parsers.

The production values should never be negative. Use None, or omit the key if a specific production mode is not known. Storage values can be both positive (when storing energy) or negative (when the storage is emptied).

The parser can also return an array of objects if multiple time values can be fetched. The backend will automatically update past values properly.

Once you're done, add your parser to the zones.json and exchanges.json configuration files. Finally update the real-time sources.

Run all of the parser tests with the following command from the root directory:

python -m unittest discover parsers/test/

For more info, check out the example parser or browse existing parsers.

Generate a new map

If your changes involve altering the way countries are displayed on the map, a new world.json will need to be generated. Make sure you're in the root directory then run the following command:

docker-compose run --rm web ./topogen.sh

Learn more about how the map is generated.

Test parsers locally

  1. Make sure you have installed the required modules as described in parsers/requirements.txt, and are using python 3.6 (consider using a virtual environment for python). To confirm this, run:

    pip install -r parsers/requirements.txt

  2. From the root folder, use the test_parser.py command line utility:

    python test_parser.py FR price  # get latest price parser for France
python test_parser.py FR  # defaults to production if no data type is given
# test a specific datetime (parser needs to be able to fetch past datetimes)
python test_parser.py DE --target_datetime 2018-01-01T08:00

Many of the tests require API keys of the data or web service providers, and therefore fail with an error message like

Exception: No ENTSOE_TOKEN found! Please add it into secrets.env!

In such cases, please browse the website related to the provider and ask for an API key. Once you get hold of the API key, make it an environment variable. This fixes the error.

Update the map

We've added a testing server locally.

To add a new country to the map, run:

PYTHONPATH=. python mockserver/update_state.py <zone_name>

from the root directory, replacing <zone_name> by the zone identifier of the parser you want to test. This fetches production, and assigns it a random carbon intensity value. It should appear on the map as you refresh your local browser.

Troubleshooting

  • ERROR: for X Cannot create container for service X: Invalid bind mount spec "<path>": Invalid volume specification: '<volume spec>'. If you get this error after running docker-compose up on Windows, you should tell docker-compose to properly understand Windows paths by setting the environment variable COMPOSE_CONVERT_WINDOWS_PATHS to 0 by running setx COMPOSE_CONVERT_WINDOWS_PATHS 0. You will also need a recent version of docker-compose. We have successfully seen this fix work with v1.13.0-rc4. More info here: docker/compose#4274.

  • No website found at http://localhost:8000: This can happen if you're running Docker in a virtual machine. Find out docker's IP using docker-machine ip default, and replace localhost by your Docker IP when connecting.

Windows Specific

  • FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory. You can configure the memory allocation using NODE_OPTIONS=--max-old-space-size=4096 in the "build-release" script in package.json, example: export NODE_OPTIONS=--max-old-space-size=4096 NODE_ENV=production && webpack --bail --mode production. Also update the node version (last working version was v12.16.3).

  • Expected linebreaks to be 'LF' but found 'CRLF': Configure your IDE to use LF instead of CRLF. If that doesn't fix it, add "linebreak-style": 0 in "rules" in .estlintrc

  • ERROR: for electricitymap-contrib_mockserver_1 Cannot start service mockserver: OCI runtime create failed: container_linux.go:345: starting container process caused "process_linux.go:424: container init caused \"rootfs_linux.go:58: mounting \\\".../server.py\\\" to rootfs \\\"/mnt/sda1/var/lib/docker/overlay2/.../merged\\\" at \\\".../server.py\\\" caused \\\"not a directory\\\"\"": unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type.

    • Check that the project is cloned under C:/Users/

  • nodemon not restarting on file changes: try adding the -L parameter to use the legacy watch: "server-dev": "nodemon server.js -L". See https://www.npmjs.com/package/nodemon#application-isnt-restarting

electricitymap-contrib's People

Contributors

alixunderplatz avatar blackleg avatar bohne13 avatar brunolajoie avatar corradio avatar denplis avatar dependabot[bot] avatar fbarl avatar felixdq avatar gazer75 avatar igorarduin avatar jarek avatar jkopb avatar liang avatar lorrieq avatar magol avatar martindaniel4 avatar maxbellec avatar nessie2013 avatar ovbm avatar paulcornelissen avatar pierresegonne avatar q-- avatar raffox97 avatar robertahunt avatar scriptator avatar systemcatch avatar thierryollivero avatar tmslaine avatar wojciej avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.