dos-group/vessim

A co-simulation testbed for carbon-aware applications and systems 🍃

Vessim

Vessim is a versatile co-simulation testbed for carbon-aware applications and systems which connects domain-specific simulators for renewable power generation and energy storage with real software and hardware.

It simulates energy systems that interact with real or simulated computing systems for:

• Carbon-aware applications: Simulated microgrids offer real-time visibility and control via APIs, enabling the development of novel applications that interact with their energy system.
• Energy system composition: Examine how the integration of solar panels, wind energy, or batteries would affect the energy mix of your datacenters.
• Digital Twins: Predict future system states in carbon-aware datacenters, aid decision-making, and assess risks during extreme events like power outages.
• Quality Assurance: Apply Vessim in continuous integrating testing or use it to validate software roll-outs in a controlled environment.

Vessim can simulate large numbers of microgrids in parallel, comes with ready-to-use datasets, can execute simulated experiments faster than real-time, and is easily extendable with new simulators of any platform through Mosaik's TCP interface.

Check out the official documentation!

Example scenario

The scenario below simulates a microgrid consisting of a simulated computing system (which consistently draws 400W), a single producer (a solar power plant who's production is modelled based on a dataset provided by Solcast), and a battery. The Monitor periodically stores the energy system state.

from vessim.actor import ComputingSystem, Generator
from vessim.controller import Monitor
from vessim.cosim import Environment
from vessim.power_meter import MockPowerMeter
from vessim.signal import HistoricalSignal
from vessim.storage import SimpleBattery

environment = Environment(sim_start="15-06-2022")

monitor = Monitor()
environment.add_microgrid(
    actors=[
        ComputingSystem(power_meters=[MockPowerMeter(p=400)]),
        Generator(signal=HistoricalSignal.from_dataset("solcast2022_global"), column="Berlin"),
    ],
    controllers=[monitor],
    storage=SimpleBattery(capacity=100),
    step_size=60,
)

environment.run(until=24 * 3600)  # 24h
monitor.to_csv("result.csv")

Installation

You can install the latest release of Vessim via pip:

pip install vessim

If you require software-in-the-loop (SiL) capabilities, you should additionally install the sil extension:

pip install vessim[sil]

For complex scenarios that involve custom co-simulation actors we recommend cloning and editing this depository directly.

Work in progress

Our team at the Distributed and Operating Systems group at TU Berlin is actively working to improve Vessim. We are currently working on the following aspects and features:

• Software-in-the-loop API: We will soon release a new API for SiL simulations with new examples and better documentation.
• System Advisor Model (SAM): We are working on integrating NREL's SAM as a subsystem in Vessim, allowing for better simulation of solar arrays, wind farms, and other types of renewable energy generators.
• Flower: We are working on integrating Vessim into the federated learning framework Flower.
• Validation: We are working on validating the accuracy of Vessim compared to real hardware testbeds.

Datasets

Vessim comes with ready-to-user datasets for solar irradiance and average carbon intensity provided by

and

We're working on documentation on how to include custom datasets for your simulations.

Publications

If you use Vessim in your research, please cite our paper on software-in-the-loop simulation for carbon-aware applications:

Philipp Wiesner, Marvin Steinke, Henrik Nickel, Yazan Kitana, and Odej Kao. "Software-in-the-Loop Simulation for Developing and Testing Carbon-Aware Applications" Software: Practice and Experience, 53 (12). 2023.

@article{wiesner2023sil,
    author = {Wiesner, Philipp and Steinke, Marvin and Nickel, Henrik and Kitana, Yazan and Kao, Odej},
    title = {Software-in-the-loop simulation for developing and testing carbon-aware applications},
    journal = {Software: Practice and Experience},
    year = {2023},
    volume = {53},
    number = {12},
    pages = {2362-2376},
    doi = {https://doi.org/10.1002/spe.3275}
}

Also have a look at our overall vision paper:

Philipp Wiesner, Ilja Behnke and Odej Kao. "A Testbed for Carbon-Aware Applications and Systems" arXiv:2302.08681 [cs.DC]. 2023.

@misc{wiesner2023vessim,
    title={A Testbed for Carbon-Aware Applications and Systems}, 
    author={Wiesner, Philipp and Behnke, Ilja and Kao, Odej},
    year={2023},
    eprint={2306.09774},
    archivePrefix={arXiv},
    primaryClass={cs.DC}
}

vessim's Issues

Incompatible readthedocs build with new interface

The new interface has implemented a revised project directory structure. Consequently, readthedocs generates inaccurate documentation, including outdated bullet points.
To fix this issue, the documentation pipeline should be adjusted and updated.

`Environment.run` with `until=None` will raise exception

This will not work:

vessim/vessim/cosim.py

Lines 99 to 100 in 7f68bd8

	if until is None:
	until = int("inf")

That is because there is no such thing as an integer infinity in python.

>>> int("inf")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: invalid literal for int() with base 10: 'inf'
>>> float("inf")
inf
>>> int(float("inf"))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OverflowError: cannot convert float infinity to integer

float("inf") was replaced by int("inf") in ad3adb8 although float("inf") works fine. To satisfy a type checker without any effects on the runtime one could use typing.cast(int, float("inf")).
Alternatively something like 10**100 probably works, too.

RedisDocker keeps running after simulation is terminated

After the simulation ends or is terminated manually, the docker container for Redis does not stop. This is very likely due to the incompatibility of the python multiprocessing and atexit libraries.

Inconsistent Object Access across Mosaik Simulators

In the CACU, the storage.min_soc is set based on the current time. When the time is > 60 * 5, the min_soc is set from .3 to .6. This should be visible in the logs. The logs however display a different time at which the min_soc is changed. However if the threshold is set to 60 * 30, the logs display the correct time, incorrect again if set to 60 * 40. I suspect that Mosaik has a problem with inter simulator object access and therefore causes problems. We may have to connect the output of the cacu sim e.g. to the microgrid to update the min_soc. Needs further investigation.

How to Handle Failed HTTP Requests?

We decided to terminate the simulation or an actor like the CACU if an HTTP request made by them fails. However, pretty much all of our requests are being run asynchronously in a thread. But if a request fails, the unhandled excpetion only terminates the thread and not the application itself.

What's the best way to handle this? A naive approach would involve creating a wrapper class for the request that surveils the state of the thread and terminates the application when the thread is terminated by an exception. We could also look for a library that allows for true asynchronous requests which may offer a more elegant solution.

Running cosim_example requires "requests" dependency

When following the installation process from the README:

$ pip install -e . 

$ python examples/cosim_example.py 

This error occurs: ModuleNotFoundError: No module named 'requests'
In the pyproject.toml 'requests' is not installed as a dependency, only when the sil flag is set.
The issue originates from power_meter.py where HTTPClient is used which needs the 'requests' dependency, but the MockPowerMeter class does not require it.
Therefore, to fix this issue it could be considered to extract MockPowerMeter in its own file apart from power_meter.py

`apply_error()` Function

Give the user the ability to pass a lambda function to apply_error() to define the error to the actual data themselves. Maybe also give a default option.

Integration test pipeline incompatible with new interface

The integration test pipeline will stop working with the new interface and should be updated in the near future.

Make project compatible with older Python versions

As this is a library, we want to support as many Python versions as possible. At least down to Python 3.8, if possible 3.7.

Tasks:

• Make project compatible with Python 3.7 or 3.8
• Adapt CI/CD pipeline to run for 3.7/3.8 and 3.11 (latest): https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-python

AWS Node

In addition to the pi and gcp node, we need an aws node, i.e. a new terraform script.

Example Node README

The Readme in the example node currently does not contain setup instructions for the virtual node.

Collector issue

The collector does not output the data of the very last simulation step. New Data is generated, but not printed to the output file

Simplification of Storage/ StoragePolicy

At the moment, the handling storage of the microgrid is divided into two classes, the Storage itself and the StoragePolicy, managing the charging/discharging of the storage. Further research needs to be made, whether this separation is justified.
Either way, it seems to be unnecessary that the Microgrid contains both the policy and the storage as two separate attributes, and simplifications should be made here (as well as better documentation #117)

Add type checker to CI pipeline

We currently only use ruff, let's also add a type checker such as mypy or pyre to the pipeline and fix any existing violations.

consisent server api for all nodes

Yesterday, we (@birnbaum) discussed the interface between the virtual nodes and the ves simulation and decided for an API server for set and get requests to be the ideal solution for our current setup (instead of a Hypervisor like the paper specifies)

The communication for virtual and physical nodes are basically the same. Therefore, I suggest to replace the current mqqt approach for the physical nodes with the same API server setup that is planned for the virtual nodes. Because each node needs to implement the fastapi set and get requests individually but the API server itself stays the same, I suggest a super classs that maintaines the server and defines abstract function as interface for subclasses that implement the specific node behaviour.

Also, where are the put requests processed? The powermeter handles the get reqests, i.e. power consumptio and cpu util. but where is the e.g. the power-mode set? In the ves?

closes #21. closes #27.

Frequency Behaviour for Forecasts

In Simulator.TraceSimulator.forecast_at() the user should be able to define the behaviour of the frequency the forecasts are returned. i.e. what happens when the user set the frequency to 20 mins but the data only offers data every 30 mins?

Interpolation suggestion:

• https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.reindex.html
• https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.resample.html
• https://pandas.pydata.org/docs/reference/api/pandas.core.resample.Resampler.interpolate.html

Frequencies:

• https://pandas.pydata.org/docs/reference/api/pandas.tseries.frequencies.to_offset.html

Move API Server into separate process

Currently the EnergySystemInterface is hosting the API server directly. This is dangerous, since a large number of requests can make it fall behind real-time simulation schedule. We would need to start the API server in a separate process. Also, the API server must be responsible for starting/stopping the Redis database.

Adjust VES FastAPI to pydantic

Its best practice to use pydantic models for FastAPI data validation. This is already somewhat implemented in example_node but needs to be implemented project wide.

Create purely simulated test scenario

To facilitate testing, we need to create a basic scenario which runs without any SiL integration.

The scenario should comprise of all currently implemented energy system components and a dummy consumer. After this is implemented, we should integrate a small-scale experiment as an integration test for pull requests.

Node class

To send PUT requests to a node in virtual_energy_system, the node address and the desired new power_mode is necessary. For GET requests, the id of the node is necessary. The node address is currently specified in main.py and needs to be transferred to the VES.

Can we have a simple node class, where all these values can be accessed, that is created in main.py and given to VES as a param?

GcpPowerMeter

I have looked into implementation possibilities to retrieve the current cpu utilisation of a gcp compute node.

1. goole-cloud-monitoring API: You can use the google-cloud-monitoring API to retrieve the current CPU utilization of a GCP compute node. This API provides a straightforward way to access monitoring metrics and data. However, the util is only provided for every minute!

2. ssh: connect via ssh and use a command like top. This would allow custom cpu utilisation queries with shorter intervals. This method is however more invasive.

For testing I suggest to set up a simple terraform script to easily create and destroy an instance.

Also for all implementations, info like the gcp project, zone, region and google application credentials (like a service account json) is required. I suggest to create a config.ini in the dir and set these values and then retrieve them with configparser

Better error message when running if docker is not started

Currently, there is a huge error log coming up with redis errors, when docker is not started while executing main.py. A simpler error message would be more helpful.

Where to negate the power consumption?

Currently, the power for the http power meters is still a positive value and so, the power consumed is treated as produced power. Should we negate it every time we poll it from the consumers via the node_api, negate it only when we return the measurement, or negate it inside the computing system when we aggregate the values?

API request log

Create a unviersal request log for api calls that enables the user to decide how to handle multiple requests within a single simulation step

Examples can not be run after installing the distribution package

After the big refactoring of examples the examples can not be run, because path issues occur.
Specifically, when building vessim with Poetry, installing the resulting .tar.gz file with pip and running

$ python examples/cosim_example/cosim_example.py

this error occurs:

from examples._data import load_carbon_data, load_solar_data
ModuleNotFoundError: No module named 'examples'

This is because currently the example directory is treated like a package, as seen in related import statements and in examples/cosim_example/cosim_example.py

 "Cacu": {
        "python": "examples.cosim_example.cacu:CacuSim",
    }

Improve documentation of storage module

The documentation of specific functions in the classes related to the batteries is currently lacking. Especially the apply-method of the DefaultStoragePolicy is missing a docstring explaining what the return means and how it is computed.

`min_soc` field in `vessim.core.storage`

In SilInterfaceModel we are currently calling self.storage.min_soc with storage of vessim.core.storage. The field does not exist in the base class and only works because the storage object is of type SimpleBattery(Storage). Do we rather want to add the min_soc field to the base class or change the expected type of the storage parameter to SimpleBattery?

Heartbeat Functionality for Node API Server

The external node servers take a certain time to initialize until they are accessible via http. To check their initial and ongoing accessibility this issue could be resolved with a heartbeat functionality between simulation and node server.

Times Zones for TraceSimulator

Add to the documentation that users are responsible for managing time zones in data files. We recommend using UTC. We should maybe check for the use of time zones in the data files and throw an error. -> maybe use pydantic?

Adjust API endpoints to paper

The current API endpoints are not consistent with the endpoints defined in Philipps paper and need to be adjusted according to:

Request Method	Endpoint	Request Body	Description
GET	/solar	-	Gets the current solar power production in W.
GET	/ci	-	Gets the current grid carbon intensity in gCO2/kWh.
GET	/battery-soc	-	Gets the current battery state of charge in %.
GET	/forecasts/solar	-	Gets the solar forecast for the next 24 hours.
GET	/forecasts/ci	-	Gets the grid carbon intensity for the next 24 hours.
PUT	/ves/battery	{"min_soc": 0.6, "grid_charge": 10}	Configures the battery: min_soc describes the minimum state of charge in %, and grid_charge the power at which the battery is charged from the public grid in W.
PUT	/cs/nodes/{id}	{"power_mode": "normal"}	Configures the power mode of node {id} (see Table 2).

Open Questions @birnbaum :

• What is the output format of the forecasts?

Consistently integrate real time factor across SiL-related modules

Currently we are defining the real time factor which is passed to world.run() separately from the intervals passed to the HttpPowerMeter and SilInterface, although they depend on each other.

Ideally HttpPowerMeter and SilInterface would automatically derive their interval from the step size and real time factor, so that we do not have to pass this parameter explicitly.

Research python concurrency

How do we run the fastapi server to be truely concurrent to the simulation. Is a thread sufficient, do we need to start it via Popen of multiprocessing?

Supress Mosaik Warnings

I figured out how to supress mosaik's warnings. Mosaik uses loguru for logging. Since most of the warnings we are getting are labeled as bugs in their git, I propose we disable them until a later release of mosaik.

from loguru import logger
import logging

logger.disable(logging.WARNING)  # Disable warning level and above

# Example usage
logger.debug("This is a debug message")  # This will not be displayed
logger.warning("This is a warning message")  # This will not be displayed
logger.error("This is an error message")  # This will be displayed

Adjust Battery

change step to update; step_size -> duration. Adjust in sim

Simulation Does Not Call Mosaik's `finalize()` When `HttpPowerMeter` is Initialized with Non Reachable Server

Adjust docstrings to google style guide.

At multiple occasions the first line of the docstring spans over two lines, exceeding 80 chars in total.

Docker Container for Virtual Node API Server

Its best practice to encapsulate the virtual (cloud) node api server in a docker container instead of installing and initiallizing the server via the gcp startup script metadata.

merge conflict in docstrings

vessim/vessim/cosim/_util.py

Lines 104 to 119 in 147d5dc

	def _convert_inputs(
	attrs: dict[str, dict[str, Any]]
	) -> dict[str, Union[Any, list[Any]]]:
	"""Converts Mosaik step inputs into a simpler format suitable for Vessim.
	If there is a single input, only the input is being returned.
	If there are multiple inputs, they are returned as a list.
	Examples:
	>>> _convert_inputs({'p': {'ComputingSystem-0.ComputingSystem_0': -50})
	-50
	>>> _convert_inputs({'p': {'ComputingSystem-0.ComputingSystem_0': -50,
	>>> 'Generator-0.Generator_0': 12})
	[-50, 12]
	"""

not sure which is the correct example

Wrong Simulation inputs for Solar and Carbon Agent

The input attribute for the Carbon Agent in carbon_controller.py and main.py has to be changed from ci to intensity_input as that is the right input used for computation.
The same change has to be made to the solar controller (solar -> production)

Consistent Logging throughtout Project

Currently were just using simple prints for debugging and its getting a bit hard to manage. Let's use a logger. Since Mosaik is central to our simulation environment and uses Loguru, I propose to use this logger.

`Controller` will cause failed assertion if `Microgrid` did not initialize `p_delta` yet

If a controller is initialized with a step_size lower than its microgrid the controller will be stepped first.
This results in an assertion error here:

vessim/vessim/controller.py

Line 167 in 7f68bd8

assert p_delta is not None

This is despite the KeyError explicitly being ignored earlier:

vessim/vessim/controller.py

Lines 158 to 161 in 7f68bd8

	try:
	p_delta = _get_val(inputs, "p_delta")
	except KeyError:
	p_delta = None # in case there has not yet been any power reported by actors

Maybe setting p_delta to 0.0 in case it is not available yet (and removing the then unnecessary assertion) would be a suitable solution?
Or alternatively letting the Controller deal with a None value however it wants to.

Display Sim Name for INFO Level Logging

Currently, when starting an Actor or Controller

INFO     | mosaik.scenario:start:280 - Starting "Actor" as "Actor-0" ...

is logged by mosaik. However, instead of "Actor-0" we want to display the actual name of the vessim.actor.Actor instance.

With the current mosaik release (3.2) on pypi, this is not possible, as this functionality will first be available starting with mosaik version 3.3. Then, we can implement the intended behaviour with the sim_id parameter of mosaik.World.start()

cacu http requests

Should the http request the cacu sends to the vesssim scenario as part of its scenario be executed as a thread to not fall behind the simulation time?

Bracketing Convention

Let's use a uniform bracketing style. I have used and seen any of these in this project and would like some unity:

Style 1:
my_dictionary = {'uno': 'something',
                 'number two': 'some other thing',
}

Style 2:
my_dictionary = {'uno': 'something',
                 'number two': 'some other thing',
                 }

Style 3:
my_dictionary = {'uno': 'something',
                 'number two': 'some other thing',}

Support for Simulated Nodes in SIL Scenario

Currently we can either have fully simulated scenarios in which MockPowerMeterss are used to execute static power models with a simulated CACU to control their power mode and poll their consumption OR a full SIL scenario with a real (external) CACU to control the power modes of real virtual or physical nodes. While it makes sense to have a seperate fully simulated scenario, the possibility to run a simulated node in a SIL scenario and also control its power mode (running a static power consumption w/out power modes is possible) is yet to be implemented.

Is this something we want to implement? This would include registering the simulated nodes just like the physical/virtual nodes as a Node object and adjusting the SilInterfaceSim to behave differently in setting power modes for each node type.

Reimplement Redis into Api Server

Integrate redis back into the vessim api server.

API Consumer

ich denke für unsere Evaluation macht es Sinn dass dieses Script grob die Funktion und Logik der power_usage Funktion im evaluation.ipynb übernimmt und 1. regelmäßig GET requests für entscheidungsrelevante Metriken wie battery soc und carbon intensity abschickt, und wenn die irgendwelche thresholds über/unterschreiten dann POST requests macht, die das VirtualEnergySystem oder eben die compute nodes irgendwie verändert. Können wir aber alles in nem neuen PR machen dann

CACU Scenario Base Class

Currently, the scenario logic for the simulated and sil cacu is duplicated. Let's build a base class for such a scenario that can be utilized by both cacus.

Vessim API call -> node API call

If the virtual_energy_ststem receives an API call for a power_mode change for one of the nodes, does the VES immediately send a power_mode PUT request to the respective node API server or does this happen at the next step, i.e. when the VES step() function is called?

NodeApiMeters should issue HTTP requests independently from the simulation

Problem

Currently, the NodeApiMeters send an HTTP request to their assigned node directly within their step method. Any networking issues can cause significant delays in the simulation that make it fall behind real-time schedule.

Solution

NodeApiMeters should periodically send HTTP requests to their node and simply cache the result within the class. Simulation steps always return the last cached result.

Side note: Let's maybe rename NodeApiMeter to HttpPowerMeter