equinor / data-modelling-tool Goto Github PK
View Code? Open in Web Editor NEWA tool for modeling and presenting json-based data models
Home Page: https://proxy-data-modelling-tool-dev.radix.equinor.com/
License: MIT License
A tool for modeling and presenting json-based data models
Home Page: https://proxy-data-modelling-tool-dev.radix.equinor.com/
License: MIT License
Store templates and blueprints in a central database. e.g maf
Select blueprint
Select or create db from gui in entities page. (needed to populate treeview with rootnodes, other entities from same blueprint)
Type title of new entity, (needed to add it as root node in the treeView)
Treeview:
one root element, click on to edit title name etc of entity. (root-blueprint)
render blueprint children in treeview. Click on a children to fill in entity values.
show other entities of selected blueprint as rootnodes.
Submit: use server and db name in endpoint?
Only keep Radix as CD
Should handle getting package from client and insert into DB at selected data source
Output should be same as output of index endpoint. input is same as input_import_collection('blueprint' || 'entities')
the input should be stored to database, just create an index which will be used to display the uploaded files in the treeview. A separate action must be taken by the user to actually store the uploaded files. Not sure about naming here but the file-upload is upload to client side, not to persisting to the database.
A blueprint should be able to reference another blueprint.
ref meeting 2.9
One standard type of index nodes and tree nodes: file | folder
How metadata is stored to mongo is abstracted by api, then other datasources, (file, postgres) can store this meta data however it wants. For instance, file storage can store this metadata in paths/filename uri.
Tasks Api:
Tasks Web:
Clean code + will be easier to test API without needing to connect to database.
Required for production use.
To avoid duplicate code we should move CRUD operations to the DmtAPI.
Not only use this to get URL, but it should include get/put/post/del operations also.
This will make the rest of the code more readable and clean.
When creating attributes for a blueprint, we should be able to specify dimensions.
The workspace would use local storage to store blueprints and entities.
This "database" should always be available to the user, without any authentication.
We need to change "create package" to "add data source". And then we need to move "create package" to underneath data sources.
Later: We need to be able to create default (maybe several) UI schema for blueprint.
Show not edit mode when clicking a blueprint, show view mode.
What's the desired behavior here?
Suggest using an alert to notify the user. Overwriting a existing package will most likely surprise the user.
To be able to use our json-schema templates as validators for form-data, they should contain stricter validation keywords.
Should be able to limit what blueprints to be shown when adding a data source. This is to avoid showing to many blueprints in tree view.
Use UI schema?
show notification on submit form in both modal and BlueprintForm
We need to translate our JSON files to react-jsonschema-form structure.
For file system, we need to get files local in client before sending it to the API for translation.
The endpoint?
/api/transformer/json-schema?
Kind of library where several jsonschemas are given upfront, in this case before build or start steps in scripts. The same code is used in all apps generated. The fields in the jsonschemas decide which path the app will go. Solved by lots of conditionals and switches.
Code is generated by templates where output is exactly one path.
Extending create react app can be used as a starter. The example does not support branches of "templates".
Suggest building a small poc of both.
TODO: Still need to address a codegenerator for api and storage.
Create new API endpoint to store data-sources.
How to get/post/put data sources:
/api/data-sources
How to get blueprint:
/api/blueprints?dataSourceId=,blueprintId=
/api/data-sources/:id/blueprints/:id
Tables that needs to be created:
The web application creates basic json-objects from a template/blueprint. These needs to be translated into the json-schema standard for storage and future use.
This could be done on a new endpoint in the API etc. /api/generate-schema
Usecase 1: create blueprint (Domain Developer) støtter også create templates (admin)
Krav:
Ha tilstand for state i page rot komponent. Følgende states trengs:
Søk: lokal state i Treeview.
Usecase 2: create entities (Domain Expert).
api/services/database.py
create db runtime from a list of uri, username, password.
expose a db instance by invoking a method on database.py instead of exposing the actual db client variable.
Look into azure services of password manager. (fetch these compile time or runtime)
Not needed until a user need to use a database from a new resource group.
Api
postBlueprint / postDocument data = formData.
ny id blir parent path som angitt i og <title> .json
response:
{ <parent-id>: {
children: [....children, ....[...<id på ny blueprint>]]
},
{
<new-id>: {
nodeType: "file",
_id: <new-id>,
....
}
},
}
dermed kan vi merge nodes med disse 2 og få ny index som er riktig.
New REST endpoint to GET a schema/form from a specific data-source.
Optional transform to json-schema
Default should be an attribute. This will be the default value for an attribute.
This is not probably not needed in schema for attributes, just use dimensions for single type also.
Maybe something like this would work:
{
"title": "DatabaseDataSourceTemplate",
"description": "",
"type": "object",
"required": [
"firstName",
"lastName"
],
"properties": {
"name": {
"type": "string",
"title": "Name",
"default": ""
},
"host": {
"type": "string",
"title": "Hostname",
"default": ""
},
"username": {
"type": "string",
"title": "Username"
},
"password": {
"type": "string",
"title": "Password",
"minLength": 10
}
}
}
Put it under /api/schemas/templates (maybe own folder called data-sources)
For the time being, this will be done in entrypoint.sh
.
The team decided to change the core structure of treenodes (with more metadata) and move the treeview generation to api. The api has also had mayor changes due to supporting different datasources which are now implemented.
Some issues are not finished:
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.