digitaltwinconsortium/dtdlparser

Digital Twin Definition Language parser for .NET

dtdlparser's Issues

Publish Preview with JSInterop APIs

Feature proposal: provide inverse of DTInterfaceInfo/Extends property

This is a proposal for a reverse property companion to the Extends property, enabling top-to-bottom traversal of an Interface hierarchy. So, in addition to the current property:

public IReadOnlyList<DTInterfaceInfo> Extends { get; }

There would be a new property:

public IReadOnlyList<DTInterfaceInfo> ExtendedBy { get; }

Update README before Release

Include links in README to:

• API Docs
• Preview Feed
• Samples
• Remove -prerelease flag

Add index.html to root of parser API docs page

There is no default document on https://digitaltwinconsortium.github.io/DTDLParser/, which is the root of the parser API docs site https://digitaltwinconsortium.github.io/DTDLParser/api/DTDLParser.html. Navigating to this root yields a 404 error, which is potentially confusing. There should be a default document that redirects to the actual docs top-level page.

Quirks for tolerating solecisms should not apply to DTDL v3 models

The specific set of 3 solecisms that are tolerated by the ModelParsingQuirk were issues with an old version of the parser that accepted only DTDL v2 models. Therefore, these solecisms should not be tolerated in DTDL v3 models irrespective of whether this quirk is specified.

ModelParser support for externally defined language extensions

Currently the ModelParser is not able to dynamically load language extensions defined on the @context via the DtmiResolver callback.
Is there a future plan to support this, so that Model Parser could also validate models based on externally/custom defined extensions?

PRs from forked repos fail with error, "Resource not accessible by integration".

PRs from forked repos are failing due to a problem with the im-open/process-dotnet-test-results task reporting an error: "Resource not accessible by integration".

I did a tiny bit of investigation/research on this, and it might be related to the use of GITHUB_TOKEN in the workflow. In https://docs.github.com/en/actions/security-guides/automatic-token-authentication#permissions-for-the-github_token, the "Maximum access for pull requests from public forked repositories" shows "read" for all scopes, including "checks". This would be a bit odd, though, because although this is failing on fork PRs in the digitaltwinconsortium/DTDLParser repo, it works on fork PRs in the Azure/DTDL-parser-proto repo. So perhaps it has an unrelated cause.

Need CLA or other appropriate condition for accepting external contributions

This might need input/action from the DTC.

Tutorials solution does not build on fresh clone

The Tutorials.sln file does not include the Parser.csproj file.

After cloning the repo, the tutorials do not build and fail with:

CSC : error CS0006: Metadata file 'C:\code\DTDLParser\dotnet\src\Parser\bin\Debug\netstandard2.0\DTDLParser.dll' could not be found

This issue should be fixed by including the Parser.csproj when generating the sln file

Fetch Units Information

Could you please advise on how to fetch Units" for all defined Unit Semantic types (Acceleration , Angle, AngularAcceleration....)?
I'm trying to access metadata defined in supplemental types, but I can't find any method or property to retrieve the "Unit" types.
For instance, defined unit types for 'AccelerationUnit' are

• centimetre per second squared
• gForce
• metre per second squared

` {
ModelParser modelParser = new();
var supp = modelParser.GetSupplementalTypes();
foreach (var sup in supp)
{
Console.WriteLine($"{sup.Key} --> {sup.Value.Type}");
foreach (var prop in sup.Value.Properties)
{
Console.WriteLine($"{prop.Key} --> {prop.Value?.Type}");
}
Console.WriteLine();

    }
}`

Reorganize Solution, rename Parser.csproj, unify unit tests projects

Suggestion

• The DTDParser.sln could include solution folders to match the source tree by organizing projects in gen, src and tests.
• To follow best practices, project names should match the AssemblyName, hence Parser.csproj -> DTDLParser.csproj
• Refactor UnitTests in a single project and avoid Linking files

Feature proposal: break out DTInterfaceInfo/Contents property by subtype

This is a proposal for an additional set of properties on the DTInterfaceInfo class that provide a breakout of the Contents property by subtype. So, in addition to the current property:

public IReadOnlyDictionary<string, DTContentInfo> Contents { get; }

There would be 5 new properties:

public IReadOnlyDictionary<string, DTComponentInfo> Components { get; }
public IReadOnlyDictionary<string, DTPropertyInfo> Properties { get; }
public IReadOnlyDictionary<string, DTTelemetryInfo> Telemetries { get; }
public IReadOnlyDictionary<string, DTCommandInfo> Commands { get; }
public IReadOnlyDictionary<string, DTRelationshipInfo> Relationships { get; }

IndexOutOfRangeException when parsing models in multithreads

version: 1.0.4-preview

When multiple threads are parsing models sometimes this exception is thrown:

Message: 
System.IndexOutOfRangeException : Index was outside the bounds of the array.

Stack Trace: 

HashSet`1.AddIfNotPresent(T value, Int32& location)
HashSet`1.Add(T item)
Model.TrySetObjectProperty(Dtmi elementId, String layer, String propertyName, JsonLdProperty propProp, Dtmi referencedElementId, String keyProp, String keyValue, ParsingErrorCollection parsingErrorCollection) line 71
Model.SetObjectProperties(List`1 objectPropertyInfoList, ParsingErrorCollection parsingErrorCollection, StandardElementCollection standardElementCollection, Boolean preserveElementAliases) line 141
ModelParser.ParseInternal(IEnumerable`1 jsonTexts, DtdlParseLocator dtdlLocator) line 168
ModelParser.Parse(IEnumerable`1 jsonTexts, DtdlParseLocator dtdlLocator) line 71

Add Support for Overriding Extension

ValidateInstance. Improve validation message for Enums

When validating a Enum instance, if the value is not defined in the Enum, the validation message says:

.. does not match any name in schema

I think it should be

.. does not match any value in the Enum

repro:

[TestMethod]
public void ValidateEnumStringInstance ()
{
    const string dtdl = @"
        {
        ""@context"": ""dtmi:dtdl:context;2"",
        ""@id"": ""dtmi:tests:enumString;1"",
        ""@type"": ""Interface"",
        ""displayName"": ""enumString"",
        ""contents"": [
        {
            ""@type"": ""Property"",
            ""name"": ""aPropWithEnum"",
            ""schema"": {
            ""@type"": ""Object"",
            ""fields"": [
                {
                ""name"": ""anEnumString"",
                ""schema"" : {
                    ""@type"": ""Enum"",
                    ""valueSchema"": ""string"",
                    ""enumValues"": [
                    {
                        ""name"": ""optionOne"",
                        ""enumValue"": ""one""
                    },
                    {
                        ""name"": ""optionTwo"",
                        ""enumValue"": ""two""
                    }
                    ]
                }
                }
            ]
            }
        }
        ]
    }
    ";
    var model = new ModelParser().Parse(dtdl)[new Dtmi("dtmi:tests:enumString;1")] as DTInterfaceInfo;
    var anEnumStringSchema = model.Properties["aPropWithEnum"].Schema;
            
    string instanceOk = @"
    {
        ""anEnumString"" : ""two""
    }";
    var validations = anEnumStringSchema.ValidateInstance(instanceOk);
    foreach (var validation in validations) Assert.AreEqual("", validation);
    Assert.AreEqual(0, validations.Count);

    string instanceFails = @"
    {
        ""anEnumString"" : ""four""
    }";
    validations = anEnumStringSchema.ValidateInstance(instanceFails);
    foreach (var validation in validations) Assert.AreEqual("\"anEnumString\" does not match any name in schema", validation);
    Assert.AreEqual(1, validations.Count);
}

Feature request: top-down navigation of inheritance hierarchy

Ttraversing the interface inheritance hierarchy top-to-bottom is a fairly common use case in, e.g., building GUI, so a convenience method for such navigation wouldn’t hurt. It’s not critical and it’s easy for devs using the parser to implement themselves, but it would go a little way towards better devex if it was provided out-of-the-box.

For an example of a use case and implementation, see this app, specifically starting here (somewhat ugly but works).

Best,

Karl

Enable existing DTDL V3 models with quantitative type extension to continue working

Willow is using DTDL V3 models in their ADT instances and refer to several quantitative types like EnergyRateUnit which are not present in latest version of parser. Can we provide a strategy that will prevent their models from breaking while supporting new/renaming of quantitative types as well?

MultiTarget to avoid BCL.AsyncInterfaces

With #51 now all consumers using ParseAsync need to add a reference to Microsoft.Bcl.AsyncInterfaces

I believe that multitargeting to 2.1, or even net6, we can improve the consumer experience in newer dotnet platforms.

Review break out properties in typings

These properties are not available in the output of ParseToJson

DTDLParser/javascript/generated/DtdlOm.d.ts

Lines 145 to 152 in f7d1284

	contents: { [name: string]: string };
	commands: { [name: string]: string };
	components: { [name: string]: string };
	properties: { [name: string]: string };
	relationships: { [name: string]: string };
	telemetries: { [name: string]: string };
	extends: string[];
	extendedBy: string[];

Need to figure out how to consume those types in the JS Sample

Expand Max Depth of Model Inheritance

Currently, the Parser limits the maximum depth of inheritance to 10 levels. Would it be possible to expand this limit to 12 or more?

DTDLParser does not support .NET Framework projects

Targeting netstandard2.1 removes support for .NET Framework applications.

A common use case for DTDL is to creating tooling to enable code generation, unfortunately the extensibility model of VS 2022 is based on .NET Framework.

Targeting netsandard2.0 will enable targeting 4.6.2, 4.7, 4.7.1, 4.7.2, 4.8, 4.8.1 as described in
https://learn.microsoft.com/en-us/dotnet/standard/net-standard?tabs=net-standard-2-0

This might be a problem to support the IAsyncEnumerable as it will force consumer applications to install Microsoft.Bcl.AsyncInterfaces

Another option will be to multi-target NS2.0 and NS2.1 and apply conditional logic to disable AsyncParsing in NS2.0

Parsing on multiple threads can lead to InvalidOperationException exceptions

In addition to production code, this can be reproduced when running unit-tests that use the model parser from the Visual Studio unit-test runner, which will execute the tests on multiple threads by default.

Related to #86

Feature proposal: GetImplicitElements method

Proposal to add a new method to ModelParser:

public IReadOnlyDictionary<Dtmi, DTEntityInfo> GetImplicitElements()

This would be analogous to the extant method:

public IReadOnlyDictionary<Dtmi, DTSupplementalTypeInfo> GetSupplementalTypes()

The new method would provide an object model that includes all elements implicitly defined by DTDL, by the standard extensions, and by any partner or feature extension that is directly supported by the parser. Eventually, whenever dynamic extensions are supported, the returned object model will also include elements defined by any extension that has been loaded into the ModelParser instance.

DTDL v2 models should allow undefined extensions by default

The previous GA parser, which understood only DTDL v2, allowed undefined extensions in models. The current parser supports configurable support for undefined extensions via ParsingOptions, but the presence or absence of support applies to models in all versions of DTDL, and the default behavior (with no option specified) is to disallow undefined extensions. For better backward compatibility, the parser should default to allowing undefined extensions for DTDL v2 models but continuing to disallow them for DTDL v3 models.

ModelParser does not sanity-check MaxDtdlVersion value

MaxDtdlVersion should be limited to the range [2, ParsingOptions.MaxKnownDtdlVersion]

Flatten Model

We're currently working on a project where we need the "effective" or "flattened" model definition. In other words, given the following models:

vehicle.json

{
  "@id": "dtmi:example:Vehicle;1",
  "@type": "Interface",
  "displayName": "Vehicle",
  "contents": [
    {
      "@type": "Property",
      "name": "make",
      "schema": "string"
    },
    {
      "@type": "Property",
      "name": "model",
      "schema": "string"
    }
  ],
  "@context": "dtmi:dtdl:context;2"
}

car.json

{
  "@id": "dtmi:example:Car;1",
  "@type": "Interface",
  "displayName": "Car",
  "contents": [
    {
      "@type": "Property",
      "name": "tireSize",
      "schema": "string"
    },
    {
      "@type": "Property",
      "name": "numberOfDoors",
      "schema": "int"
    }
  ],
  "@context": "dtmi:dtdl:context;2",
  "extends": "dtmi:example:Vehicle;1"
}

We want the effective or flatted schema for dtmi:example:Car;1 - which would be:

{
  "@id": "dtmi:example:Car;1",
  "@type": "Interface",
  "displayName": "Car",
  "contents": [
    {
      "@type": "Property",
      "name": "make",
      "schema": "string"
    },
    {
      "@type": "Property",
      "name": "model",
      "schema": "string"
    },
    {
      "@type": "Property",
      "name": "tireSize",
      "schema": "string"
    },
    {
      "@type": "Property",
      "name": "numberOfDoors",
      "schema": "int"
    }
  ],
  "@context": "dtmi:dtdl:context;2",
  "extends": "dtmi:example:Vehicle;1"
}

We're planning on doing this with json manipulation, but this might be useful to other folks as well.

Proposed Solution

Perhaps add something like a FlattenInterface extension method:

public static class Extensions
{
   public static DTInterfaceInfo FlattenInterface(this IReadOnlyDictionary<Dtmi, DtmiEntityInfo> parsed, Dtmi dtmi)
   {
      //TODO: Magic
   }
}

var jsonTexts = new string[] 
{
  File.ReadAllText("vehicle.json"),
  File.ReadAllText("car.json")
}

var parser = new ModelParser();
var result = parser.Parse(jsonTexts);

DTInterfaceInfo flattened = result.FlattenInterface(new Dtmi("dtmi:example:Car;1"));

var json = model.GetJsonDLText();

Add Samples

Create samples to show how to enumerate the interface contents

• JSInterop sample
• .NET sample