regnosys/rosetta-dsl

The project containing the Rune DSL grammar and default code generators

rosetta-dsl's Issues

Navigate from Dispatch Functions to the parent

When reading a dispatch function, I want to know what are the inputs and outputs attributes that are being referenced. Can we create content assist to to display such information or provide an ability to click and navigate to the parent function?

Add support for List literal

post-condition: <"Ensuring that the execution attributes are correctly populated on the output.">
	if execution -> after -> execution -> party exists then
		execution -> after -> execution -> party count = 2 and
		execution -> after -> execution -> party contains partyA and
		execution -> after -> execution -> party contains partyB
	else True;

In the above example, we are testing whether execution -> after -> execution -> party contains both partyA and partyB. It would be nicer to specify the below:

	if execution -> after -> execution -> party exists then
		execution -> after -> execution -> party count = 2 and
		execution -> after -> execution -> party contains [ partyA, partyB ]

Scoping: Add Enumerations to Attribute Path scope

The below is valid Rosetta, however the content assist does not suggest QuantityNotationEnum.

assign-output updatedContract -> contractualQuantity -> quantityNotation -> notationTag :
		<"Set the notional amount on the Updated Contract.">  
		QuantityNotationEnum.Notional

Type checker should detect when datetime passed to parameter of type zonedDateTime

func Foo:
  inputs:
    timestamp zonedDateTime (1..1)
  output: result boolean (1..1)

Foo( date )  // should yield validation error for type mismatch

Add Scoping and Validation features to 'func'

• Validate parameter types on the caller side
• Validate alias path do not use a Type as entry point (only inputs or output?)
• Validate same enum type is used in enum dispatch
• Validate no duplicate enum values are used
• one of can only be defined one per Data object
• Dispatch func may only refer to an enum typed attr
• warn if not all the enum values implemented
• statements within func should only references alias, inputs, output and other funcs
• Additionally, conditions within func should only reference inputs, alias's that reference only inputs and other funcs
• constraints are not allowed inside a func conditions

RosettaFunctionDependencyProvider doesn't cover all expressions

For example:

assign-output price:  
   	if (AddOne(arg1, date) -> dat) then 4

AddOne is not recognized as Function dependency. Looks like Parenthesis expression is missing. Maybe it beter to throw an exception if an expression is not handled

[validation] Allow list element access only for multi attributes

quantityNotation is 1..*

assign-output out -> quantityNotation[1] -> notationTag // OK

assign-output out -> quantityNotation -> notationTag[1] // errror

Expand 'func' features to fully support 'calculation'

• func should add support for alias and assign-output

• add support for 'dispatch' functions where the exact implementation of the function depends on an Enum Value

• 'func' code generators to generate concrete implementations

Merge JavaQualifiedTypeProvider and JavaNames

Only one class should survive
com.regnosys.rosetta.generator.java.function.JavaQualifiedTypeProvider
com.regnosys.rosetta.generator.java.util.JavaNames

Same for:
RosettaExpressionJavaGenerator.xtend
RosettaExpressionJavaGeneratorForFunctions.xtend

Add support for defining Data

Clean up grammar

We can remove spec function and calculation

Generated code for functions to all follow same structure

func RateOfReturn utilises code generators that were used for (the now deprecated) calculations. We should aim to have a consistent structure and api for all generated code that relate to Rosetta Functions.

Additionally, This causes incompatibility in the java code when new-style functions reference old-style functions. In the below example, func EquityReset was modified to call func RateOfReturn, which yielded the below error in the generated code for EquityReset.

Support 'empty' type

func NewEquitySwapProduct makes use of EmptyEquitySwapMasterConfirmation2018(), which returns a null or empty object. We need to be able to support passing in a generic empty object into functions.

func NewEquitySwapProduct: <"Function specification to create an Equity Swap according to the 2018 ISDA CDM Equity Confirmation Template, based on a minimum set of inputs which can (optionally) include a Master Confirmation Agreement. The inputs represent the minimum set of inputs required to create an Equity Swap, either based on an existing Master Confirmation Agreement or as a stand-alone Equity Swap">
	inputs: 
		underlier Equity (1..1) <"The underlying Equity asset for the swap.">
		masterConfirmation EquitySwapMasterConfirmation2018 (0..1) <"An (optional) pointer to the Master Confirmation Agreement, if any, that holds further inputs to the Equity Swap">
		
	output:
		product Product (1..1)
		
	post-condition: <"Equity and interest rate payouts must be set-up according to their corresponding payout specifications, and other payout types must be absent.">
		if masterConfirmation exists then
			product -> contractualProduct -> economicTerms -> payout -> equityPayout = NewSingleNameEquityPayout( underlier, masterConfirmation ) and
			product -> contractualProduct -> economicTerms -> payout -> interestRatePayout = NewFloatingPayout( masterConfirmation )
		else
			product -> contractualProduct -> economicTerms -> payout -> equityPayout = NewSingleNameEquityPayout( underlier, EmptyEquitySwapMasterConfirmation2018() );
		product -> contractualProduct -> economicTerms -> payout -> cashflow is absent;
		product -> contractualProduct -> economicTerms -> payout -> creditDefaultPayout is absent;
		product -> contractualProduct -> economicTerms -> payout -> forwardPayout is absent;
		product -> contractualProduct -> economicTerms -> payout -> optionPayout is absent;	
		
	post-condition: <"Non-contractual product types must be absent.">
		product -> foreignExchange is absent;
		product -> index is absent;
		product -> loan is absent;
		product -> security is absent;

The resulting syntax will be easier to read and will be more usable as it avoid the need to create function to produce empty objects.

product -> contractualProduct -> economicTerms -> payout -> equityPayout = NewSingleNameEquityPayout( underlier, empty );

Generated code to support adding values to lists

The below model code will generate java code that does not compile. This is because in order to assign a value to a list (using the current api), we must know which index of that list to assign.

assign-output reset -> after -> updatedContract -> contractualProduct -> economicTerms -> payout -> equityPayout -> rateOfReturn : <"...">
		RateOfReturn( equityPayout )

The default behaviour should be to assign to index 0, however we should also support in the syntax a way to assign to an arbitrary index i, i.e. in the below.

assign-output reset -> after -> updatedContract -> contractualProduct -> economicTerms -> payout -> equityPayout[1] -> rateOfReturn : <"...">
		RateOfReturn( equityPayout )

Add cardinality checks

1. Bar( strings ) should error as strings has multiple cardinality when single cardinality is expected.
2. assign-output fooOut : Bar( strings ) should error as the return type of Bar has multiple cardinality when single cardinality is expected.

func Foo:
	inputs: strings string (1..*)
	output: fooOut number (1..1)
	assign-output fooOut : Bar( strings )
		
func Bar:
	inputs: string string (1..1)
	output: numbers number (1..*)

Generated java code should handle values of multiple cardinality

For . model :

	assign-output execution -> after -> execution -> party:
		[ partyA, partyB ]

The below java code is generated

executionHolder
	.getOrCreateAfter().getOrCreateExecution()
	.addPartyRef(MapperC.of(MapperS.of(partyA), MapperS.of(partyB)).get());
;

But should use MapperC::getMulti:

executionHolder
	.getOrCreateAfter().getOrCreateExecution()
	.addPartyRef(MapperC.of(MapperS.of(partyA), MapperS.of(partyB)).getMulti());
;

Validate name conflicts inside a function

Inputs output and aliases may not share the same name.

When users types <", the corresponding "> should be inserted automatically

Add support for assigning 'key's to 'reference'

In the below extract from the FormContract Function:

func FormContract : <"...">
	inputs:
		executionEvent Event (1..1)
		legalAgreement LegalAgreement (0..1)
		
	output:
		contractFormationEvent Event (1..1)
	
	...
	
	assign-output contractFormationEvent -> lineage -> eventReference : <"Construct the lineage making reference to the input event">
		executionEvent
		
	...

Add support so that the key on executionEvent can be assigned to the reference on eventReference. This can be done via a language level keyword that then generates the correct java code underneath.

	assign-output contractFormationEvent -> lineage -> eventReference : <"Construct the lineage making reference to the input event">
		executionEvent as-key

Validation for choice rules

Validation:

• Validate at least two attributes passed to choice
• condition : oneOf // name mandatory? <-- NO

Originally posted by @dhuebner in #40 (comment)

Add support for defining Annotations

Handle cardinality transformation in function invocation

In the below function, we must explicitly extract out elements from a Rosetta Path that yields a list. We should support an implicit "get only element" when such invocations are made.

The Rosetta Path: contract -> contractualProduct -> economicTerms -> payout -> equityPayout yields a list of values as there can be multiple equityPayouts. Function Equity Reset requires only a single Equity Payout.

func EquityReset: 
	inputs:
		equityPayout EquityPayout (1..)
		observation number (1..1)
		date date (1..1)
        ...

func EquityResetEvent: 
	inputs:
		contract Contract (1..1)
		observation Event (1..1)
		
	output:
		reset Event (1..1)

	...

 	post-condition: <"Event must contain a reset primitive that is calculated based on the observation, and no other primitive.">
		reset -> primitive -> reset = EquityReset( 
				ExtractSingleEquityPayout( contract -> contractualProduct -> economicTerms -> payout -> equityPayout ), 
				ExtractSingleNumber( observation -> primitive -> observation -> observation ), 
				ExtractSingleDate( observation -> primitive -> observation -> date ) );
		reset -> primitive -> reset only exists;

The desired syntax:

 	post-condition: <"Event must contain a reset primitive that is calculated based on the observation, and no other primitive.">
		reset -> primitive -> reset = EquityReset( 
			contract -> contractualProduct -> economicTerms -> payout -> equityPayout, 
			observation -> primitive -> observation -> observation, 
			observation -> primitive -> observation -> date );
		reset -> primitive -> reset only exists;

Validation should still check Type compatibility and the number of arguments.

Reduce warnings in generated Java

Even after switching to new FuncGenerator and Using new DataGenerator we still have around 6k compile warnings in generated Java. Most of them are wrong imports. As we now use the new ExpressionGenerator that supports autoimport we can switch existing generators to use that feature.

Consolidate grammar for Synonyms on Attributes and Enums

enum DayCountFractionEnum <"The enumerated values to specify the day count fraction.">
	[synonym FpML_5_10, CME_SubmissionIRS_1_0, DTCC_11_0, DTCC_9_0, CME_ClearedConfirm_1_17 value dayCountFractionScheme_2_2]
{
	ACT_360 displayName "ACT/360" <"Per 2006 ISDA Definitions...">
		[synonym FpML_5_10, CME_SubmissionIRS_1_0, DTCC_11_0, DTCC_9_0, CME_ClearedConfirm_1_17 value "ACT/360"],

class Contract key <"A class ...">
		[synonym FpML_5_10, CME_SubmissionIRS_1_0, CME_ClearedConfirm_1_17 meta id path "trade"]
		[synonym FpML_5_10, CME_SubmissionIRS_1_0, CME_ClearedConfirm_1_17, Rosetta_Workbench meta id]
{
	contractIdentifier Identifier (1..*) <"The identifier...">;
		[synonym FpML_5_10, CME_SubmissionIRS_1_0 value partyTradeIdentifier path "trade.tradeHeader"]
		[synonym FpML_5_10, CME_SubmissionIRS_1_0 value partyTradeIdentifier path "tradeHeader"]
		[synonym CME_ClearedConfirm_1_17 value partyTradeIdentifier path "trade.tradeHeader"]

1. the value on enums are in double-quotes whilst on attributes are not in quotes. Should make consistent and should prefer no quotes where possible.
2. for attribute synonyms, the path is in quotes whilst the value is not. Should make consistent and should prefer no quotes where possible.
3. the dot . delimiter on the path is inconsistent with the arrow -> on rosetta attribute paths, I think we should use -> for all paths if possible

Update versions of Xtend and Xtext

The versions of Xtext and Xtend currently being used (2.15.0) only supports up to Java 8.
We need to be able to move to a fully supported version of Java.

Validate direct assignment of an alias

In assign-output when using an alias there should be at least one path segment for an assigment setter/adder:

alias EquityPayout_before: <"">
		reset -> before -> updatedContract -> contractualProduct -> economicTerms -> payout -> equityPayout only-element
	
assign-output EquityPayout_before:   // <--- error here 
		<"">
		equityPayout
	

Solution refactoring:

alias Payout_before: <"">
		reset -> before -> updatedContract -> contractualProduct -> economicTerms -> payout only-element
	
assign-output  Payout_before -> equityPayout :   
		<"">
		equityPayout
	

Generate default impl for abstract function

Guice needs a default implementation to inject a class. But we can't use generate once approach or wait until user implements a class.

@ImplementedBy(AddOne.AddOneDummy.class)
public abstract class AddOne implements RosettaFunction {

	/**
	 * @param arg
	 * @param d
	 * @return out
	 */
	public FooDat evaluate(FooDat arg, Date d) {

		FooDat out = doEvaluate(arg, d).build();

		return out;
	}

	protected abstract FooDat.FooDatBuilder doEvaluate(FooDat arg, Date d);


	private static final class AddOneDummy extends AddOne {
		@Override
		protected FooDatBuilder doEvaluate(FooDat arg, Date d) {
			throw new UnsupportedOperationException();
		}
	}
}

Link to CDM repo not working

The README states "The CDM is hosted separately from the Rosetta DSL in its own repository." but the link to https://github.com/REGnosys/rosetta-cdm does not work.

Is the link incorrect? Or is the CDM only privately available?

Nick

Support type checking in 'assign-output' blocks

It should be invalid to assign the value of 1.0 to a string type

class Test {
	string string (1..1)
}

func TestFunc:
	output: test Test (1..1)
	assign-output test -> string : 1.0

Remove support for basic type 'datetime' as it has been replaced by 'zonedDateTime'

Create grammar rules to support 'func'

Based on the content of the Rosetta Syntax Update Wiki, the first step of the plan is to add Grammar Rules to support the new Function concept. Code completion should follow what already exists for specs.

Add generic ListBuilder to allow implementation of function with list return type

A list builder should allow creation and mutation of a list of Rosetta model objects.

Should also support common Builder API patterns.

Issue with count in condition

The function

	func Allocate: <"Function specification to create the fully-formed business event of allocating an 	execution based on allocation instructions.">
	

	inputs:
		execution Execution (1..1) <"Block trade to be allocated.">
		allocationInstructions AllocationInstructions (1..1) <"Allocation instructions to be applied.">
		previousEvent Event (1..1) <"Previous event for lineage purposes.">

	output:
		allocationEvent Event (1..1) <"Allocation event containing both the original block trade (with the status set to Allocated) and the set of allocated trades.">

	condition: 
		allocationInstructions -> breakdowns count > 0

fails to generate code with the server side error
INFO [2019-10-16 10:43:21,771] [] com.regnosys.rosetta.generator.RosettaGenerator: Unexpected calling standard generate for rosetta
! java.lang.IllegalArgumentException: RosettaCountOperation is not covered yet.
! at com.regnosys.rosetta.generator.java.function.RosettaFunctionDependencyProvider.functionDependencies(RosettaFunctionDependencyProvider.java:141)
! at com.regnosys.rosetta.generator.java.function.FuncGenerator.lambda$3(FuncGenerator.java:156)

Allow only a single expression in 'condition' and remove semi-colon delimiter

All conditions that have multiple Expressions should have expressions anded together.

Find a better name for toOne

We have a possibility to convert a Multi type (List) to a single value.
In fact the MapperC.get() function is used where if the list is of size 1 the first element will be retuned null otherwise.
Language feature is called toOne we are looking for a better name.

Use the same starting point in all conditions

For all conditions the first expression should be either an attribute or a function call.
Currently for datarules we use:

data Foo: 
    attr1:  int
    attr2: string
    condition Foo_rule_1:
		if Foo -> attr1 exists
		then Foo -> attr2 exists;

Bug in ValidatorHelper

com.rosetta.model.lib.validation.ValidatorHelper

public static <T> ComparisonResult contains(Mapper<T> o1, Mapper<T> o2) {
    boolean result =  o2.getMulti().containsAll(o2.getMulti());
    if (result) { ...

should be:
o1.getMulti().containsAll(o2.getMulti())

ValidatorHelper::doIf should support missing else statement

    public static <T> Mapper<T> doIf(ComparisonResult test, Mapper<T> ifthen) {
        return test.get() ? ifthen : null;
    }

The doIf is called as follows: doIf( xyz, trueValue ).get()

If the test result is false, null is returned and the calling code will throw a null pointer exception

Support * and / for expressions

MapperMaths needs to be updated to support multiplication and division

Make RosettaConditionalExpression else condition optional

To make Rosetta more concise (especially for functions) can we make the "else" condition of RosettaConditionalExpression optional, and in the code generation default it to true.

func FooFunc:
inputs: a A (1..1)
output: b B (1..1)
post-condition:
if a.name = "Foo"
then b.foo exists
else True; // else condition should be optional and generated code will default to true

Change data keyword to type

All functions to generated abstract java classes

A follow on from #43.

funcs annotated with calculation should generate concrete implementations whilst all other funcs should generate abstract classes. The abstract classes should represent the logic as defined in the func and should delegate to a concrete implementation that throws an UnsupportedOperationException that signals to the users that they should provide their own implementations.

Support assign-output for rosetta paths

Support the following use case in func

data Parent:
  child number (1..1)

func Example:
  output:
    result Parent (1..1)
  assign-output result -> child:
    1.0

setIssuerReferenceRef in org.isda.cdm.Identifier ignores issuerReference

The code below ignores the input parameter issuersReference is ignored and a empty builder is created
public IdentifierBuilder setIssuerReferenceRef(Party issuerReference) {
return setIssuerReferenceRef(Party.builder())
}

Enum consistency

1. So that enums can be syntactically more similar to types, remove the comma separating each enum value.
2. So that referencing enum values is more consistent with referencing types: use the arrow syntax i.e. DayCountFractionEnum -> _30_360 not DayCountFractionEnum._30_360

Choice rule syntax

• support one of and Choices only in condition blocks.
• condition blocks can have either be a single one of, multiple Choices or multiple Expressions

Condition:
    'condition' RosettaNamed? ':' Definable?
        Annotations?
        (constraints += Constraint)+ | (expressions+=RosettaCalcExpression ';')+
;

Constraint:
    Choice | OneOf
;

Choice:
    ('optioinal' | required ?= 'required') 'choice' 'between' attributes += [Attribute | ValidID] ',' attributes += [Attribute | ValidID] (',' attributes += [Attribute | ValidID])*
;

OneOf:
    'one' 'of'
;

Support 'this' or 'it'

There are sometime situations where we need to pass a special kind of parameter. Example:

func ForwardFX:
	inputs: forwardPayout ForwardPayout(1..1)
	output: result ForeignExchange (1..1)
	assign-output result: forwardPayout -> doSmartThings...

data ForwardPayout: 
    condition FxForward_settlementDate: <"...">
		if ForwardFX(<NEED ForwardPayout HERE>) exists

Would be nice instead of an implicit first parameter handling to have this or it as type inside a data element. Another option would be to re-write forwardFX to have underlier as input

see also #38

Code generation fails when using 'count' feature

See https://github.com/REGnosys/rosetta-cdm/pull/524/files for how to reproduce. The symptom is that the Allocate java function no longer generates.

Output object on the Function should be validated

The generate java code should run object validation before returning the output object. Validation invocation example here

Generated code fails to compile

If the line EquityPerformance( equityPayout, observation -> observation, periodEndDate ) is updated to EquityPerformance( updatedEquityPayout, observation -> observation, periodEndDate ), then the generated code does not compile

func ResolveEquityContract: <"Specifies how the updated contract should be constructed in a Equity Reset event.">
	inputs:
		contractState ContractState (1..1)
		observation ObservationPrimitive (1..1)
		date date (1..1)
		
	output:
		updatedContract Contract (1..1)
		
        alias equityPayout:
		contractState -> contract -> contractualProduct -> economicTerms -> payout -> equityPayout only-element
	
	alias updatedEquityPayout :
		updatedContract -> contractualProduct -> economicTerms -> payout -> equityPayout only-element

        ...
		
	alias equityPerformance :
		EquityPerformance( equityPayout, observation -> observation, periodEndDate )