irieo/space-time-optimization

Code for the paper "Spatio-temporal load shifting for truly clean computing"

Code for the paper "Spatio-temporal load shifting for truly clean computing"

This repository contains the code to reproduce the complete workflow behind the manuscript.

Abstract

Companies with datacenters are procuring significant amounts of renewable energy to reduce their carbon footprint. There is increasing interest in achieving 24/7 Carbon-Free Energy (CFE) matching in electricity usage, aiming to eliminate all carbon footprints associated with electricity consumption on an hourly basis. However, the variability of renewable energy resources poses significant challenges for achieving this goal. We explore the impact of shifting computing jobs and associated power loads both in time and between datacenter locations. We develop an optimization model to simulate a network of geographically distributed datacenters managed by a company leveraging spatio-temporal load flexibility to achieve 24/7 CFE matching. We isolate three signals relevant for informed use of load flexiblity: varying average quality of renewable energy resources, low correlation between wind power generation over long distances due to different weather conditions, and lags in solar radiation peak due to Earth’s rotation. We illustrate that the location of datacenters and the time of year affect which signal drives an effective load-shaping strategy. The energy procurement and load-shifting decisions based on informed use of these signals facilitate the resource-efficiency and cost-effectiveness of clean computing—the costs of 24/7 CFE are reduced by 1.29±0.07 EUR/MWh for every additional percentage of flexible load. We provide practical guidelines on how companies with datacenters can leverage spatio-temporal load flexibility for truly clean computing. Our results and the open-source optimization model can also be useful for a broader variety of companies with flexible loads and an interest in eliminating their carbon footprint.

How to reproduce results from the paper?

1. Clone the repository:

git clone [email protected]:Irieo/space-time-optimization.git

2. Install the necessary dependencies using environment.yaml file. The following commands will do the job:

conda env create -f envs/environment.yaml
conda activate 247-env

3. The results of the paper can be conveniently reproduced by running the snakemake workflows. The following terminal commands will run the workflows for sections 1-4 of the paper:

snakemake -call --configfile custom_config_s1.yaml
snakemake -call --configfile custom_config_s2.yaml
snakemake -call --configfile custom_config_s3.yaml
snakemake -call --configfile custom_config_s4.yaml

NB Size of a mathematical problem for this paper is optimized so that it is possible to reproduce the results on a private laptop with 8GB RAM.

Model results will be stored in the results directory. For each workflow, the directory will contain:

• solved networks (.nc) for individual optimization problems
• summary (.yaml) for individual optimization problems
• summary (.csv) for aggregated results
• log files (memory, python, solver)
• detailed plots (.pdf) of the results

4. At this point, a curious reader can even reproduce the dashboards from the paper by running the jupyter notebooks in the scripts/ directory and compile the LaTeX project /manuscript/manuscript.tex to reproduce the paper .pdf file.

Data requirements

The workflow is based on PyPSA networks exported from PyPSA-Eur built with myopic setting to get brownfield networks for 2025/2030. For convenience, the workflow uses already networks provided in the input folder by default.

Technology data assumptions are automatically retrieved from technology-data repository for <year> and <version>, as specified in config.yaml.

Several plots from the paper require high-resolution geographical data that is not included in this repository. To reproduce those plots, three files (elec_s_256_ec.nc, profile_solar.nc, regions_onshore_elec_s_256.geojson) from PyPSA-Eur Zenodo repository have to be retrieved and placed in the input/ directory. The following command automates this task:

snakemake -call retrieve_data

Software requirements

The code is known to work with PyPSA 0.26.0, pandas 2.0.3, numpy 1.26.2, and gurobi 10.0.1. The complete list of dependencies is in the envs/environment.yaml file.

License

There are different open licenses for different types of files in the repository. See specifications here.

space-time-optimization's Issues

Issue reproducing the results on linux

Hi I've attempted to run the commands provided in the readme to reproduce the results of the paper locally. This issue is specifically covering x64/linux/ubuntu. I get this same error on x64/WSLv1/ubuntu as well. The error doesn't occur on x64/Windows

repro steps:

1. conda env create -f envs/environment.yaml
2. conda activate 247-env
3. snakemake -call --configfile custom_config_s1.yaml

expected results: outputs in to results folder
actual results: error:

Traceback (most recent call last):
File "/home/scottcha/miniconda3/envs/247-env/bin/snakemake", line 10, in
sys.exit(main())
^^^^^^
File "/home/scottcha/miniconda3/envs/247-env/lib/python3.12/site-packages/snakemake/init.py", line 2736, in main
parser = get_argument_parser()
^^^^^^^^^^^^^^^^^^^^^
File "/home/scottcha/miniconda3/envs/247-env/lib/python3.12/site-packages/snakemake/init.py", line 1607, in get_argument_parser
lp_solvers = pulp.list_solvers(onlyAvailable=True)
^^^^^^^^^^^^^^^^^
AttributeError: module 'pulp' has no attribute 'list_solvers'. Did you mean: 'listSolvers'?

Here is the output of conda list for pulp version 2.8.0
pulp 2.8.0 py312h7900ff3_0 conda-forge

Model too large for size limited gurobi license.

Hi I've attempted to reproduce the results of your paper. It seems that the model provided is too large for the version of gurobi which is being referenced. It might be worth mentioning in the readme or provide an option for a open source solver if one doesn't have the right gurobi license, thoughts?

Output when running: snakemake -call --configfile custom_config_s1.yaml

Set parameter LogFile to value "results\paper-DKDEPT/logs/2025/p1/cfe100/DKDEPT/0_solver.log"
Gurobi Optimizer version 11.0.1 build v11.0.1rc0 (win64 - Windows 11.0 (22631.2))

CPU model: AMD Ryzen 9 3900X 12-Core Processor, instruction set [SSE2|AVX|AVX2]
Thread count: 12 physical cores, 24 logical processors, using up to 4 threads

Traceback (most recent call last):
File "C:\Users\scott\source\repos\space-time-optimization.snakemake\scripts\tmp0_0rtvyg.solve_network.py", line 1492, in
solve_network(
File "C:\Users\scott\source\repos\space-time-optimization.snakemake\scripts\tmp0_0rtvyg.solve_network.py", line 1397, in solve_network
n.optimize.solve_model(
File "C:\Users\scott\miniconda3\envs\247-env\Lib\site-packages\pypsa\optimization\optimize.py", line 649, in solve_model
status, condition = m.solve(solver_name=solver_name, **solver_options, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\scott\miniconda3\envs\247-env\Lib\site-packages\linopy\model.py", line 1060, in solve
result = func(
^^^^^
File "C:\Users\scott\miniconda3\envs\247-env\Lib\site-packages\linopy\solvers.py", line 634, in run_gurobi
m.optimize()
File "src\gurobipy\model.pxi", line 893, in gurobipy.Model.optimize
gurobipy.GurobiError: Model too large for size-limited license; visit https://gurobi.com/unrestricted for more information

Thanks!