powsybl/powsybl-core

A framework to build power system oriented software

PowSyBl Core

PowSyBl (Power System Blocks) is an open source framework written in Java, that makes it easy to write complex software for power systems’ simulations and analysis. Its modular approach allows developers to extend or customize its features.

PowSyBl is part of the LF Energy Foundation, a project of The Linux Foundation that supports open source innovation projects within the energy and electricity sectors.

Read more at https://www.powsybl.org !

This project and everyone participating in it is governed by the PowSyBl Code of Conduct. By participating, you are expected to uphold this code. Please report unacceptable behavior to [email protected].

PowSyBl vs PowSyBl Core

This document describes how to build the code of PowSyBl Core. If you just want to run PowSyBl demos, please visit https://www.powsybl.org/ where downloads will be available soon. If you want guidance on how to start building your own application based on PowSyBl, please visit the tutorials page.

The PowSyBl Core project is not a standalone project. Read on to learn how to modify the core code, be it for fun, for diagnosing bugs, for improving your understanding of the framework, or for preparing pull requests to suggest improvements! PowSyBl Core provides library code to build all kinds of applications for power systems: a complete and extendable grid model, support for common exchange formats, APIs for power simulations an analysis, and support for local or distributed computations. For deployment, powsybl-core also provides iTools, a tool to build cross-platform integrated command-line applications. To build cross-platform graphical applications, please visit the PowSyBl GSE repository https://github.com/powsybl/powsybl-gse page.

Environment requirements

Powsybl-core project is fully written in Java, so you only need few requirements:

• JDK (17 or greater)
• Maven (3.8.1 or greater) - you could use the embedded maven wrapper instead if you prefer (see Using Maven Wrapper)

To run all the tests, simply launch the following command from the root of the repository:

$> mvn package

Modify some existing tests or create your own new tests to experiment with the framework! If it suits you better, import the project in an IDE and use the IDE to launch your own main classes. If you know java and maven and want to do things manually, you can also use maven directly to compute the classpath of all the project jars and run anything you want with it.

Read Contributing.md for more in-depth explanations on how to run code.

Read Install to generate an installed iTools distribution, a standalone external folder that contains all the built objects required to run powsybl programs.

Install

An iTools distribution can be generated and installed. The installation is a standalone external folder that contains all the built objects required to run powsybl programs through the itools command-line interface. This repository contains the install.sh script to do so easily. By default, the install.sh will compile code and copy the resulting iTools distribution to the install folder.

$> ./install.sh

A more detailled description of the install.sh script options follows:

Targets

Target	Description
clean	Clean modules
compile	Compile modules
package	Compile modules and create a distributable package
install	Compile modules and install it
docs	Generate the documentation (Javadoc)
help	Display this help

Options

The install.sh script options are saved in the install.cfg configuration file. This configuration file is loaded and updated each time you use the install.sh script.

Global options

Option	Description	Default value
--help	Display this help
--prefix	Set the installation directory	$HOME/powsybl
--mvn	Set the maven command to use	mvn

Default configuration file

#  -- Global options --
powsybl_prefix=$HOME/powsybl
powsybl_mvn=mvn

Using Maven Wrapper

If you don't have a proper Maven installed, you could use the Apache Maven Wrapper scripts provided. They will download a compatible maven distribution and use it automatically.

Configuration

Configure the access to the maven distributions

In order to work properly, Maven Wrapper needs to download 2 artifacts: the maven distribution and the maven wrapper distribution. By default, these are downloaded from the online Maven repository, but you could use an internal repository instead.

Using a Maven Repository Manager

If you prefer to use an internal Maven Repository Manager instead of retrieving the artefacts from the internet, you should define the following variable in your environment:

• MVNW_REPOURL: the URL to your repository manager (for instance https://my_server/repository/maven-public)

Note that if you need to use this variable, it must be set for each maven command. Else, the Maven Wrapper will try to retrieve the maven distribution from the online Maven repository (even if one was already downloaded from another location).

Using a proxy to access the Internet

If you don't use an internal Maven Repository, and need to use a proxy to access the Internet, you should:

1. configure the proxy in your terminal (on Linux/MacOS, you can do it via the http_proxy and https_proxy environment variables). This is needed to download the Maven Wrapper distribution ;

2. execute at least once the following command:

./mvnw -DproxyHost=XXX -DproxyPort=XXX -Dhttp.proxyUser=XXX -Dhttp.proxyPassword=XXX -Djdk.http.auth.tunneling.disabledSchemes= clean

Notes:

• The 4 XXX occurrences should be replaced with your configuration;
• The -Djdk.http.auth.tunneling.disabledSchemes= option should be left empty;
• Windows users should use mvnw.cmd instead of ./mwn.

This second step is required to download the Maven distribution.

Once both distributions are retrieved, the proxy configuration isn't needed anymore to use ./mvnw or mvnw.cmd commands.

Checking your access configuration

You could check your configuration with the following command:

./mvnw -version

If you encounter any problem, you could specify MVNW_VERBOSE=true and relaunch the command to have further information.

Configuring install.sh to use maven wrapper

To indicate install.sh to use Maven Wrapper, you need to configure it with the --mvn option:

./install.sh clean --mvn ./mvnw

You can revert this configuration with the following command:

./install.sh clean --mvn mvn

Usage

Once the configuration is done, you just need to use ./mvnw instead of mvn in your commands.

powsybl-core's Issues

Rename step.alpha to step.phaseShift

The alpha attribute of the step element will be renamed to phaseShift to be more consistent to the phaseTapChanger element:
IIDM v1.0

<iidm:phaseTapChanger lowTapPosition="0" tapPosition="1" regulationMode="FIXED_TAP" regulationValue="200.0">
      <iidm:terminalRef id="PS1" side="TWO"/>
      <iidm:step r="0.0" x="0.0" g="0.0" b="0.0" rho="1.0" alpha="-20.0"/>

IIDM v1.1

<iidm:phaseTapChanger lowTapPosition="0" tapPosition="1" regulationMode="FIXED_TAP" regulationValue="200.0">
      <iidm:terminalRef id="PS1" side="TWO"/>
      <iidm:step r="0.0" x="0.0" g="0.0" b="0.0" rho="1.0" phaseShift="-20.0"/>

Intro / Documentation for people new to powsybl

Hello and sry for submitting an issue here :)
maybe there is something I have overlooked so far, but googling and searching through some older RTE and iTesla Presentations has not helped me in this regard. Therefore this was my only means to get more information on powsybl.

Is there a form of documentation on how to use powsybl classes or powsybl-gse? I.E. what does the class hierarchy look like. How are interfaces defined and which classes already exist.
I managed to get everything to install and compile but now i am stuck on the next steps. I would like to try out a simple uct import, and do some basic loadflow calculations - contingency analysis, either in my own java project (preferably not) or in powsybl-gse or a command line interface.

Best regards,
Felix

Identifiable getName

The AbstractIdentifiable::getName implementation does not fit with the description of the API:

Get an the (optional) name  of the object.

The implementation returns the ID if the name is null:

return name != null ? name : id;

Is it really needed?

IIDM export in bus/branch mode

If a voltage level contains only disconnected equipments, the BusBreakerTopology is exported with no buses and the injection have neither bus nor connectableBus attributes. If we try to import the exported file, an exception is thrown.

Add a gPerSection parameter in the ShuntCompensator class

Add an optional gPerSection attribute to the ShuntCompensator class.

IIDM v1.0

<iidm:shunt id="SC" name="SC" bPerSection="1.58025E-4" maximumSectionCount="10" currentSectionCount="0" connectableBus="SC_bus"/>

IIDM v1.1

<iidm:shunt id="SC" name="SC" bPerSection="1.58025E-4" gPerSection="1e-4" maximumSectionCount="10" currentSectionCount="0" connectableBus="SC_bus"/>

In CIM1 Importer section count could be greater than maximum sections

In some network samples we have observed that the state variable SvShuntCompensator.continuousSections is greater than the EQ attribute ShuntCompensator.maximumSections.

The IIDM shunt compensator adder throws an exception when finds this problem.

The proposal is to change the call to shunt compensator adder in CIM1Converter::createShunt by:

`.setMaximumSections(Math.max(sc.getMaximumSections(), sectionCount)`

Fix sonar issues in loadflow-validation module

Sonar reports two different kinds of (major) issues in loadflow-validation module:

• the number of paramater of a method should be lesser than 7: we have to find a way to reduce the number of parameters of the write methods
• the member names should be different of the class name to avoid confusion: the validationWriters parameter of the ValidationWriters class must be renamed.

In CIM1 converter warnings should go after trying to add synchronous machines to the Network

Warnings emitted for non-empty collections synchronousMachinesWithoutRegulatingControl, synchronousMachinesRegulatingVoltageWithZeroTargetVoltage, synchronousMachinesWithReactiveRangeForMinus9999MW are made BEFORE code that tries to add to the Network the synchronous machines found in CIM, where these collections could be eventually filled in.

The warnings should go AFTER calls to createGenerator are made.

Enhance the readability of the conditions in loadflow-validation

As suggested in #241, some conditions are difficult to read and we should try to enhance their readability

forEachState

Here the code of the StateManagerImpl:

    void forEachState(Runnable r) {
        stateLock.lock();
        try {
            for (int index : id2index.values()) {
                stateContext.setStateIndex(index);
                r.run();
            }
        } finally {
            stateLock.unlock();
        }
    }

I think the initial state should be kept in a local variable and be restored in the finally block. It's not clear enough that the currentState is changed by this method.

What do you think?

Handle activation of simultaneous rules during the same round

The rules are evaluated sequentially. If two rules are evaluated to "true" during the same round, the second one will potentially be evaluated on an inconsistant network state (because the first action will be taken). In the majority of the cases, this behaviour will cause no harm because there is no impact of action 1 on rule 2.

To avoid potential bugs, the evaluation of the rules and application of the actions should be done differently :

• First, evaluate all the rules but apply none of the actions
• Then, apply all the actions associated with the rules evaluated to "true"

Remove Substation.geographicalTag attribute

The geographicalTag attribute of the substation element will be removed and replaced by a property if needed.

Rename RatioTapChanger.loadTapChangingCapabilities

Rename the RatioTapChanger.loadTapChangingCapabilities to RatioTapChanger.onLoadTapChanger
IIDM v1.0

<iidm:ratioTapChanger lowTapPosition="0" tapPosition="1" regulating="true" loadTapChangingCapabilities="true" targetV="158.0">

IIDM v1.1

<iidm:ratioTapChanger lowTapPosition="0" tapPosition="1" regulating="true" onLoadTapChanger="true" targetV="158.0">

Refactor SecurityAnalysisExporter

Refactor SecurityAnalysisExporter to remove the dependency to the network: it should be fully independant.

The id attribute of a terminalRef is optional

The id attribute of a terminalRef will be optional. If not set, the transformer id will be used.
IIDM v1.0

<iidm:ratioTapChanger lowTapPosition="0" tapPosition="1" regulating="true" loadTapChangingCapabilities="true" targetV="158.0">
     <iidm:terminalRef id="NHV2_NLOAD" side="TWO"/>

IIDM v1.1

<iidm:ratioTapChanger lowTapPosition="0" tapPosition="1" regulating="true" loadTapChangingCapabilities="true" targetV="158.0">
     <iidm:terminalRef side="TWO"/>

Writer should not be closed by the TableFormatter

The TableFormatter::close method closes the writer instance used to write the table. The following code currently throws an IOException:

    Security.printPreContingencyViolations(result, writer, tableFormatterFactory, tableFormatterConfig, limitViolationFilter);
    Security.printPostContingencyViolations(result, writer, tableFormatterFactory, tableFormatterConfig, limitViolationFilter, true);

It's even worst if writer is an OutputStreamWriter(System.out), because the close method closes the System.out writer

In the SecurityAnalysisTool, a hack creates an anonymous Writer to override the close method.

Replacing the content of the TableFormatter::close method to do a flush instead of a close is not compatible with the IPST (at least the AMPL Writer test failed) ; so we have to fix IPST before changing the behaviour of the close method.

Rename step.rho to step.ratio

The rho attribute of the step element will be renamed to ratio to be more consistent to the ratioTapChanger element:
IIDM v1.0

<iidm:ratioTapChanger lowTapPosition="0" tapPosition="1" regulating="true" loadTapChangingCapabilities="true" targetV="158.0">
                <iidm:terminalRef id="NHV2_NLOAD" side="TWO"/>
                <iidm:step r="0.0" x="0.0" g="0.0" b="0.0" rho="0.8505666"/>

IIDM v1.1

<iidm:ratioTapChanger lowTapPosition="0" tapPosition="1" regulating="true" loadTapChangingCapabilities="true" targetV="158.0">
                <iidm:terminalRef id="NHV2_NLOAD" side="TWO"/>
                <iidm:step r="0.0" x="0.0" g="0.0" b="0.0" ratio="0.8505666"/>

Convert the switch.retained to a state variable

Convert the retained attribute of switches to a state variable.

camel case vs snake case

The configuration file mix both snake case and camel case. We have to decide what to use and provide an itools command to help people to convert their configuration files

XNodes not correctly taken into account while importing merged UCTE file

While importing UCTE file, all XNodes are modelled as IIDM DanglingLine objects.
This is a good model for IGMs, but is not correct for merged CGMs.
XNode interconnection should then be modelled as lines.

Attached is an example file that does not import correctly.
mergedXnodeIssue.txt

Rename SetPoint to Setpoint (single word)

Rename all the occurences of SetPoint.

itools config command

Add a new itools command for configuration management. This itools command could provide these features:

• migrate an old configuration file
• dump the configuration in several format (yaml, xml...) of the specified module
• display a documentation (based on annotations?) for the specified module

Support yaml configuration files

Add the support of configuration flies written in yaml in addition of xml and properties format.

Add coherency in API

To access to shunt compensator, all methods are named getShunt() or getShuntCount() whereas the class of shunt is named shuntCompensator
These methods should be named getShuntCompensator(), getShuntCompensatorCount(), etc...

Rename the step attributes

To avoid misunderstanding, the step attributes r, x, g and b will be renamed to rdr, rdx, rdg and rdb.

Rename topological information in BusBreaker view

The bus/connectableBus attributes are redundant if the equipment if connected. There is no check on the consistency of these two attributes. To be more robust, the connectableBus will be replaced by a connectionStatus attribute.

IIDM v1.0

<iidm:load id="LOAD" loadType="UNDEFINED" p0="600.0" q0="200.0" bus="NLOAD" connectableBus="NLOAD"/>

IIDM v1.1

<iidm:load id="LOAD" loadType="UNDEFINED" p0="600.0" q0="200.0" busId="NLOAD" connectionStatus=“CONNECTED | CONNECTABLE” />

powsyblsh.bat

Create a powsyblsh equivalent for Windows.

Prepare to Java module

Here is issues to solve before switching to Java 9 modules:

• package of cim1-model is still the default one from CIM Gateway 'cim1' and is not consistent with the other modules and not compliant with module name com.powsybl.cim1.model.

• cim-anonymiser code is in com.powsybl.cim, so the root package of all cim related code. This is a mistake, module should be renamed cim-util and code should be in com.powsybl.cim.util among all other cim utilities.

• [MINOR] iidm-xml-converter should be renamed iidm-xml as package used for code is com.powsybl.iidm.xml and same is used for module name.

• iidm-impl Maven module will be named com.powsybl.iidm.impl but code is in com.powsybl.iidm.network.impl which is not included in com.powsybl.iidm.impl. Code has to be moved to com.powsybl.iidm.impl.network (or simply com.powsybl.iidm.impl ?).

• iidm-converter-api Maven module is spread over 4 packages import_ export_ parameters tools with com.powsybl.iidm as common root package which is already used by iidm.api module. Everything has to be moved to com.powsybl.iidm.converter.

• loadflow-results-completion code is in com.powsybl.loadflow package so the same package as loadflow-api. Shared package between Java 9 module is strictly forbidden and it has to be fixed.

• security-analysis-afs-local code is in com.powsybl.security.afs package, same package as security-analysis-afs module. Shared package between Java 9 module is strictly forbidden and it has to be fixed to com.powsybl.security.afs.local.

• Maven module powsybl-compulation has to be renamed powsybl-computation-api to be consistent with others.

Use TableFormatter instead sirocco-text-table-formatter

Table generation should be done through the TableFormatter API (commons module).

Concurrent use of Network instances (thread-safety)

The concurrent use of a NetworkImpl object, even with the multi-thread state manager, is partly broken:

underlying containers such as BitSet or TFloatArrayList are not thread-safe, meaning that for exemple extending it in 1 thread while writing to it in another thread may result in unexpected behaviour (exceptions, or corrupt data).

Several solutions may be foreseen:

• make the network fully thread-safe by switching to thread-safe implementations of those collections. The performance cost needs to be evaluated
• declare the API as not thread-safe and have client code handle it
• guarantee thread-safety except for state cloning operations : require to remove use of BitSet, and to provide a kind of lock to ensure that no cloning operation is run while using the Network

ArithmeticBinaryOperatorNode unexpected exception

In an Action DSL script, it is impossible to create a complex expression based on the network

var = (load('LOAD').p0 - load('LOAD').terminal.p) / 2

Evaluating this expression throws an exception:

Left operand of arithmetic operation should return a number (class com.powsybl.action.dsl.ast.ArithmeticBinaryOperatorNode)

It can easily be reproduced with a unit test in ConditionDslLoaderTest class

public void testExpressionEvaluator() throws IOException {
    ...
    evalAndAssert(100, "(load('LOAD').p0 - load('LOAD').terminal.p) / 2");
}

DataSourceUtil extension handling

The handling of file extensions in DataSources is not satisfying today for several reasons.
The current mechanism, based on the creation of a DataSource based on a file name:

1/ cannot handle successive compression extensions (.gz.zip ...), although it makes not much sense...

2/ does not take into account the extension defined at the time of creation, but only at the time of writing the output

3/ deletes everything after the first dot in the name when creating the DataSource, even though it may not be part of the actual extension. It makes it impossible, for instance to read from a file "toto.tata.xiidm.gz" passed to the command line

The DataSourceUtil and DataSource implementations need to be refactored to cope with those problems.

Remove all deprecated code

Remove all deprecated code introduced in powsybl-1.x.x

Remove itools-packager from distribution-core

Remove the packaging of powsybl from distribution-core

Creation of a user friendly message when an ID is not found in the network

When such a logical expression is defined :
Logical_expression = (loadingRank('ID1', ['ID1', 'ID2', 'ID3', 'ID4']) == 1) && Other_expression
and ID2 does not exist in the network on which the DSL script is applied, there is no error (related with the use of loading rank) but when it is applied (in the "when" part of a rule), the error is a NullPointerException.

This kind of problem should be detected more easily since IDs change regularly on a network. If an ID which is not found is used in the script, a dedicated error associated with the use of loadingRank should be thrown.

Multi apply in one rule

Something like that:
rule('rule') {
description 'rule description'
when generator('GEN').terminal.connected
apply 'action'
apply 'action2'
life 2
}

Currently the second action2 will replace but not added after the first apply. Maybe a bug? fix or add a warning message

Add primary regulation attributes on generators

Add primary regulation attributes on generators:

• add a droop attribute
• add a boolean attribute to indicate if a generator is participating or not to the regulation
  These informations must be in a sub-element of the generator.

java.lang.NoClassDefFoundError: javax/xml/bind/JAXBException

Hi,

I am trying to compile with ./install-sh

and I get a compilation error saying java.lang.NoClassDefFoundError: javax/xml/bind/JAXBException
The stack overflow related thread is here

Apparently java-xml-bind (and the java commons in general) are no longer included so the extra dependencies should be added to the download script:

<!-- Java 6 = JAX-B Version 2.0   -->
<!-- Java 7 = JAX-B Version 2.2.3 -->
<!-- Java 8 = JAX-B Version 2.2.8 -->
<dependency>
    <groupId>javax.xml.bind</groupId>
    <artifactId>jaxb-api</artifactId>
    <version>2.2.11</version>
</dependency>
<dependency>
    <groupId>com.sun.xml.bind</groupId>
    <artifactId>jaxb-core</artifactId>
    <version>2.2.11</version>
</dependency>
<dependency>
    <groupId>com.sun.xml.bind</groupId>
    <artifactId>jaxb-impl</artifactId>
    <version>2.2.11</version>
</dependency>
<dependency>
    <groupId>javax.activation</groupId>
    <artifactId>activation</artifactId>
    <version>1.1.1</version>
</dependency>

This is from Stack overflow...

I'm on ubuntu and java -version throws:

openjdk version "10.0.2" 2018-07-17
OpenJDK Runtime Environment (build 10.0.2+13-Ubuntu-1ubuntu0.18.04.2)
OpenJDK 64-Bit Server VM (build 10.0.2+13-Ubuntu-1ubuntu0.18.04.2, mixed mode)

Add a version attribute to each module

To be able to load old configuration files, we should add a version attribute to each module. If this information is not provided, we could consider that the version of the module is the same as the configuration file.

AFS does not handle accent properly

java.nio.file.InvalidPathException: Malformed input or input contains unmappable characters:

/home/baguemat/Mod��les
	at sun.nio.fs.UnixPath.encode(UnixPath.java:147)
	at sun.nio.fs.UnixPath.<init>(UnixPath.java:71)
	at sun.nio.fs.UnixFileSystem.getPath(UnixFileSystem.java:281)
	at com.powsybl.afs.local.storage.LocalAppStorage.nodeIdToPath(LocalAppStorage.java:113)

Throw an exception is two rules / actions have the same name

In a Action DSL file, one can define several actions / rules with the same name. An exception should be thrown when the file is loaded to avoid unexpected behavior during the simulation.

Replace the TieLine class by an extension

The TieLine class will be removed and replaced by an extension

Phase shifter optimizer does not work in a particular case

Case: the init tap of phaseChanger is already set to max, and the less tapPosition cause the bigger current in loadflow.

itools-packager can create other package type

Improve the itools-packager to support tar.gz and tar.bz2

Action-simulator: add possibility to parallelize the computation

Add possibility to distribute computation on sub-lists through the computation manager.

Rename DanglingLine to BoundaryLine

The DanglingLine class will be renamed to BoundaryLine.

Step attributes are optional

The step attributes are not required anymore. If not set, the default values are:

• 0 for dr, dx, dg and db
• 1 for ratio
• 0 for phaseShift
  The default values must be defined in the XSD.

Action-simulator: add possibility to formulate conditions on N-1 states for preventive actions

Add possibility to formulate conditions on N-1 (post-contingency state), for preventive actions (actions implemented on N state).

Syntax exemple:
rule("rule1") {
when contingencyOccured("contingency1") and isOverloaded("line1")
apply "action1" onNState()
}

Replace Step by Tap in all TapChanger classes and usage

To be more consistent, the word "step" will be replaced by "tap" in all relevant classes and usages.

Handle alternative actions

For some use cases, there is a need to test alternative actions. The alternatives should be prioritized and their test should be stopped when some criterias are fullfilled.

Several implementations are possible (to be discussed).

Current implementation (see examples below) : undo manually the actions precedently tested to set the network in the same state than it was before. It implies to use a mechanism to disable the alternatives already tested (life, actionTaken…)

Example :

// Save initial tap of PST
rule('Initial_Tap') {
    when !contingencyOccurred()
      life 1
    apply 'saveInitialTapOfPST'
}

action('saveInitialTapOfPST') {
      tasks {
            script{
                  transformer('PST_ID').phaseTapChanger.tapInit = transformer('PST_ID').phaseTapChanger.tapPosition
            }
      }
}

// Open transformer TR_ID
rule('Open_TR_ID') {
    when !contingencyOccurred() && constraint1
    apply 'openTrId'
}

action('openTrId') {
	tasks {  
		openSwitch 'TR_ID_BREAKER'
	}
}

// Close transformer TR_ID and open line LINE_ID
rule('Open_Line_Breaker') {
    when !contingencyOccurred() && actionTaken('openTrId') && !constraint1
    apply 'openLineBreaker'
}

action('openLineBreaker') {
	tasks {
		openSwitch 'LINE_ID_BREAKER'
                closeSwitch 'TR_ID_BREAKER'
		script {
			transformer('PST_ID').phaseTapChanger.tapPosition = transformer('PST_ID').phaseTapChanger.tapInit
		}
	}
}

Other ideas : keep all the network states for a calculation ?