arkisto-platform/describo-online

describo-online's Issues

Refresh should not make the user lose their place

If you hit refresh (eg cos there's an annoying alert) then it goes back to the root of your data. I would expect it to refresh the screen where I am currently working).

Allow ARCPs as IDs and do not add a #

If you change an item's ID to something that uses the arcp:// URI scheme it gets prepended with a # -- maybe this should be coded to comply with the JSON-LD spec so there are no surprises later.

If the node object contains the @id key, its value MUST be an absolute IRI, a relative IRI, or a compact IRI (including blank node identifiers). See § 3.3 Node Identifiers, § 4.4 Compact IRIs, and § 4.16 Identifying Blank Nodes for further discussion on @id values.
Source: https://json-ld.org/spec/latest/json-ld/#node-objects

Saving an entity template when one already exists

Saving an entity template when one already exists doesn't seem to update the data. That is, add a prop to an entity and save the template and the template data isn't updated.

[Bug] Filebrowser selects parent folders

I presume this is not intended behavior.

Steps to reproduce:

1. Have a ressource with at least one folder containing at least one file.
2. Go to Manage Collection Data Files.
3. Expand folder in tree, select only the file contained in folder.
4. A dataset corresponding to the name of the folder, containing the selected file, will be added to the collection.

Expected behavior: Only the selected file will be added to the collection, as it happens with file that are located in the root folder.

This might be related to this Element UI bugreport.

SelectObject Component

Leave SelectURI as is (an array of strings) but extend that to SelectObject where the profile author defines complex objects that are then spat out as is to be joined into the graph. Display on the select "name (@id)" or jus @id if name undefined.

Need a URL type

Need a special URL component to produce an object with an @id and a url value

{ @id: 'http://.....' }

Data not being saved when adding files to crate

When adding files to the crate via the file selector a write back to the remote is not happening so the info from those files doesn't get written in unless a property is added to those entries.

Handling @id references that point to URL

Add a link back from admin section to the root

[Feature] Ways to provide profiles in Describo (embedded mode)

@Heiss and I thought of a mechanism to allow 3rd party apps that embed Describo (like RDS) to manage profiles used in Describo on their own. This would allow us to modify and use profiles on the fly without having to worry about Describo knowing about them beforehand or Describo having to know about e.g. the current profile version for a specific repository service.

Proposal:

• In embedded iframe mode, Describo takes a profile as a json string (preferably base64 encoded), which will be used as the active metadata profile (in the session creation POST request to /api/session/application and in update session PUT requests to /api/session/application/{sessionId}). This could work in addition (XOR / defaulting to either one when both are given) or instead of (replacing) the current "file" key inside the "profile" object within the request body. [1]

Sciebo RDS can use this field to manage the used profiles withouth changes in Describo. Sciebo RDS will take care of everything profile management related (like choosing the right version according to the repository service used or injected customization and adding new profiles), while Describo will simply use the profile provided in the request.

[1]

describo-online/test-iframe-mode.html

Line 36 in 109b232

file: "schema.org",

Improvement: Make property help searchable in AddPropertyDialog.component

Right now, the attribute set will only be filtered by the name of an attribute. As people often won't be super familiar with the names of ro-crate attributes, filtering for whether a search term applies to the name OR help/description text of an attribute would give users a better chance to find what they are looking for.

Need a SelectURL type

It takes text values as options and produces

{@id: 'http:// .....' }

UI Bug - select box needs to be full width

Some entities not getting written out to crate file

When working with the text commons profile Maria Weaver has added some RepositoryCollection entities as hasMember to another collection - these were showing in Describo but not getting written into the ro-crate-metadata.json - we went in and out of describo a few times and the entities persisted but were not written to disk.

Will have to get Maria to demo - as I have been unable to reproduce

Check modification time when saving crate back to storage

When saving a crate file back to the storage layer check modification time of crate file on disk and if newer than updatedAt of the collection in the db, rename disk instance to ro-crate-metadata.${ISO timestamp}.conflict.json before sending the serialisation from the db back to the storage layer.

Properties with sub properties

Can we define in the profile some configuration that tells the UI to render a contributor field but when going to add one you get a dropdown of the sub contributor properties (e.g. annotator, depositor etc).

Potential UX issues: a user adds a contributor as annotator but then when the crate loads up next time we see the annotator field with the value. Is this an issue? To be determined.

Integrating reva based authentication

Describo has two methods for authenticating users:

• okta: https://github.com/Arkisto-Platform/describo-online/blob/master/documentation/okta-authentication.md
• external service: https://github.com/Arkisto-Platform/describo-online/blob/master/documentation/sid-authentication.md

Describo is driven by a configuration file as documented at https://github.com/Arkisto-Platform/describo-online/blob/master/documentation/configuration.md:

Reva integration

Reva authentication will be similar to the okta flow. Following is a possible way to do it.

Configuration change

• update UI configuration with a top level property authentication that defines the auth method for this installation 'okta' || 'reva'

In the UI code

• create UI component @components/Login.component.vue and wire up to the /login route in routes.js
  • The login component should use the configuration stored in the store to either load the OktaLoginComponent or the RevaLoginComponent
  • Implement @/components/RevaLogin.component.vue which knows how to perform reva logins (in the Ailleron implementation you post to the API which then talks to Reva. Can this component talk to Reva directly the way the okta component does?`
  • The RevaLoginComponent should then store the token in local | session storage and POST back to the API to create a session for the user (pretty much the same as what happens in @components/OktaLoginCallback.component.
  • update http.service.getHeaders to get the reva token or okta based on the authentication property in the configuration
  • refactor: might be worth extracting OktaLoginComponent and RevaLoginComponent as vue plugins in https://github.com/Arkisto-Platform/describo-ui-plugins

In the API code

• implement and wire up /session/reva in @/routes/index.js. It can be almost exactly like the handler for /session/okta but calling reva to check the passed in token. Instantiate a DB session for the user to use the rest of the time in the same way as happens for the okta session.
  • probably worth refactoring this file and extracting the session handlers
• update @/middleware/demandKnownUser with an option to handle authorization: reva ... in the headers. Again, this will work the same way as okta where the session lookup happens on the db.
• if doing the auth to Reva from the API: add a route that the UI form can post to which then auth's against Reva.

Fully local development setup

While looking for a RO-Crate editor I just found Describo and tried the offline version and it is really impressive, thank you for your efforts!

Now I would like to try experimenting with Describo online and try to set up a local development environment. I have a couple of problems I couldn't find answers to reading through the wiki documentation.

First of all the architecture is not really clear to me: what is a backend and how does it relate to an auth provider and what is the postgresql dependency is used for when there's a backend?

This page states that "Authentication for describo is provided by Okta". But then the opening page also lists "reva configuration" as an authentication provider. And then it turns out that if ownCloud is the backend then ownCloud can be the auth provider as well.

So overall it is not clear what a backend is for, what an auth provider is for and how these can mix and match?

Then when I try to setup a local development environment following

https://github.com/Arkisto-Platform/describo-online/wiki/setting-up-for-development

step 3 is "Register your application with Microsoft" and 4. "Create an Okta organisation and setup your application". From my previous readings these should be optional steps rather than the actaul 3. or 4. steps, right? Moreover I would prefer not to use an external system like OneDrive and Okta if they arenot the actual target of my development.

What would really be helpful is an example with a docker-compose.yaml file, which sets up local dependencies for postgresql/backend/auth (ownCloud?, Minio?) and a configuration for Describo API and UI so that both of these can be run in development mode connecting to the services provided by the docker-compose.yaml.

Property grouping in the UI

Following discussion with @ptsefton

Add a layout property to each class defined in the profile that is an array of objects as follows:

    "Dataset": {
            "definition": "override",
            "subClassOf": [],
            "layout": [
                { name: "group1", description: "", inputs: [ "location", "name", "place"] },
                { name: "group2", description: "", inputs: [ "contributor", "annotator" ] }
                { name: "...", leftover things not grouped }
            ],
           ...

• A name property defining the group name
• A description - optional
• An inputs property with the ordered list of properties to include in this group.

Any properties not defined in the layout must be added to the final tab in alphabetical order.

If a class does not have a layout property then properties must be listed in alphabetical order.

ro-crate-metadata.json synchronization fails

I am trying to use the latest version and both with the local and the onedrive configuration I get an error like this and ro-crate-metadata.json doesn't get synced to neither to the local directory nor to onedrive (the temp ro-crate-metadata.json exists and readable):

api_1   | 2022-09-29T12:35:53.217Z DEBUG: GET: /load/:service
api_1   | 2022-09-29T12:35:53.221Z DEBUG: POST: /profile
api_1   | 2022-09-29T12:35:53.264Z DEBUG: List folder: local /home/xxx/19
api_1   | 2022-09-29T12:35:53.355Z DEBUG: loading crate file @ /srv/tmp/b2303d0d-3ecc-4a96-ab04-977b7ad39109/current/ro-crate-metadata.json
api_1   | 2022-09-29T12:35:53.356Z DEBUG: Minting a new collection for this crate
api_1   | 2022-09-29T12:35:53.363Z DEBUG: Adding collection identifier to crate
api_1   | 2022-09-29T12:35:53.365Z DEBUG: Setting the collection id in the user session
api_1   | 2022-09-29T12:35:53.467Z ERROR: syncLocalFileToRemote: 2022/09/29 12:35:53 ERROR : : error listing: directory not found
api_1   | 2022/09/29 12:35:53 Failed to hashsum with 2 errors: last error was: directory not found
api_1   |
api_1   | Error: 2022/09/29 12:35:53 ERROR : : error listing: directory not found
api_1   | 2022/09/29 12:35:53 Failed to hashsum with 2 errors: last error was: directory not found
api_1   |
api_1   |     at ChildProcess.<anonymous> (file:///srv/api/src/lib/file-browser_rclone.js:234:20)
api_1   |     at ChildProcess.emit (events.js:400:28)
api_1   |     at maybeClose (internal/child_process.js:1088:16)
api_1   |     at Process.ChildProcess._handle.onexit (internal/child_process.js:296:5)
api_1   | 2022-09-29T12:35:53.469Z DEBUG: Loading the crate data into the database
api_1   | 2022-09-29T12:35:53.849Z DEBUG: GET: /entity/:entityId
api_1   | 2022-09-29T12:35:53.881Z DEBUG: POST: /definition
api_1   | 2022-09-29T12:35:53.929Z DEBUG: GET: /entity/:entityId/properties

My development-configuration.json is this:

{
  "ui": {
    "siteName": "Arkisto Platform - Describo",
    "logo": "http://www.researchobject.org/ro-crate/assets/img/ro-crate.svg",
    "login": "localhost",
    "services": {
      "localhost":  true,
      "onedrive": {
        "clientId": "XXXXXXX",
        "tenantId": "XXXXXXX",
        "redirectUri": "http://localhost:9000/onedrive-callback"
      }
    },
    "maxSessionLifetime": "86400",
    "maxEntitiesPerTemplate": "100"
  },
  "api": {
    "port": 8080,
    "periodicProcessInterval": 300,
    "typeDefinitions": "https://raw.githubusercontent.com/Arkisto-Platform/arkisto-type-definitions/master/types/type-definitions.json",
    "typeDefinitionsLookup": "https://raw.githubusercontent.com/Arkisto-Platform/arkisto-type-definitions/master/types/type-definitions-lookup.json",
    "session": {
      "lifetime": {
        "hours": 5
      },
      "secret": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
    },
    "services": {},
    "applications": []
  }
}

No autosave on textarea and text boxes

• remove autosave - add check control for consistency

Let the user type a date in the date picker rather than having to click

Back button capability

Update URL bar to keep state so back button works.

File preview components not loading

check browser console

Don't write \n to the end of a text entry (even if the user hits return)

[Feature] Option to hide resource path and "loaded collection" in embedded mode

Profile authors should be able to disable missing property errors (e.g. not defined in profile)

Socket io communications to UI need to be based on session user

When a user logs into describo and loads a folder, the API will communicate with the UI via socket io to send progress events for the user. If a second user then uses describo to do the same, the socket io comms will be seen by the first user in addition to the second user.

Obviously, the socket io comms need to be tied to a session so all users don't see them.

Form should NOT reorder itself

Currently if you add something to a property then that jumps to the top of the form. This is counter to the way forms normally work on the web and is confusing. (I know this was requested by someone, but IMO it is not a good UI choice - one we have the ability to split a profile into tabs or groups then the number of form fields showing at any one time will get smaller. Properties that are not essential can be shunted to an "Advanced" tab where they are out of sight for most users.

Add ability to load/save profiles in the validator

Save time in development. This feature might need to be turned off in production systems.

Make it more obvious you can create an new item by typing in the text box

Make it say "or create a new entity by typing a name for it" or something

Add auto cleanup process for collections

Add a property to the config to specify how old collections can be before being purged from the DB (max age) and associated code to do the cleanup.

IDs for Contextual Entities should start with #

The random IDs used should be prefixed with # as they are not URIs (paths)

Files should not be selectable when starting, see screenshot

Files should not be selectable when starting describo see screenshot

Profile integration

This ticket is to list the requirements for describo profiles.

This is a revised development of the ideas @ https://github.com/Arkisto-Platform/describo/wiki/dsp-index. If a bullet point needs to be fleshed out further add a comment with the bullet point quoted and then follow on with the discussion.

Mandatory requirements - profile handling

• validate a profile: check structure, target link types, data sources are sensible
• allow a user to upload their own profile - UPDATE: not allowing
• allow a user to select from a set of administrator defined profiles
• allow an administrator to disable profile uploads for users; leaving only admin defined profiles - UPDATE: not required
• allow an external app to define which profile to use for an external session
• attach a profile to a collection - UPDATE: profile can be changed dynamically
• change the profile associated to a collection

Mandatory requirements - profile structure

• add extra properties to classes defined in schema.org
• override schema.org class definitions - define their own inputs
• relabel properties in the UI - override schema.org name for a given community in the UI
• define result sets for a property - e.g. name can be ['x', 'y', or 'z' ]
• define singular text values for a property - e.g. additionalType = 'item'
• define which classes can be created - e.g. [ 'Dataset', 'Person' ]. The validator must ensure that the type definitions for each class do not define a property which links to some other type.
• be able to hide properties from the UI - e.g. don't show @id, @type

Desirable requirements - profile dependencies

• be able to define a lookup endpoint for a type and prevent users from creating their own information
  • is it possible to define how to do a lookup within the profile? e.g. url to call, how to call it, template for transforming the returned response into a json-ld blob...

Application should not give up on loading profiles if it hits a bug in one of them

Need a geo type

Needs to allow the user to select a point on a map or draw a region and return a bbox.

If a point then create a GeoCoordinates entry.
If a bbox then create a GeoShape entry.

@ptsefton What do you think?

Bug loading data with linked entities - some links are not picked up

Problem: When I load this RO-Crate which contains some Classes, Properties and DefinedTerms for a vocabulary, and use this profile some of the links between items are are not picked up by Describo.

Things linked from the root dataset work OK:

But inter-item references get turned into Text fields:

Even when there are links to in the original crate:

{
"@id": "https://purl.archive.org/textcommons/terms#OrthographyTypeTerms",
"@type": "DefinedTermSet",
"hasDefinedTerm": [
{
"@id": "https://purl.archive.org/textcommons/terms#Handwritten"
},
{
"@id": "https://purl.archive.org/textcommons/terms#Typewritten"
},
{
"@id": "https://purl.archive.org/textcommons/terms#Typeset"
}
],
"name": "OrthographyTypeTerms"
},

I have tried re-linking using Describo, and that works, but if I remove the Describo Identifier and re-load the crate it reverts to the behaviour where it does not link.

Add configuration to profile to disable changing resource, folder and profile

Configuration.json inconsistency

If you do not provide okta in ui.services in configuration.json, then api container will not be started with error message: configuration.json not found. For me, it has to be triggered in this line.

describo-online/api/src/common/index.js

Line 18 in 8ff05be

configuration.ui.services.okta.issuer = `${configuration.ui.services.okta.domain}/oauth2/default`;

The error message is not very helpful, if other problems can trigger it, too.

Thank you.

If I change a the ID of a person to a URL it does not get updated in the RO-Crate on disk

See this example where the ID did not change but the @reverse link is actually correct.

{
      "name": "alice",
      "@id": "f0d9ca45-e662-4774-90f8-8c988ca64d96",
      "@type": "Person",
      "@reverse": {
        "author": [
          {
            "@id": "http://orcid.org/test"
          }
        ]
      }
    }

Make it clearer where the user has to click when making a new entity

When linking to a new entity you type a name to create it - can we make it clearer where to click - instead of just clicking on the name below where you are typing, add a [Create] button beside to so you can click on either

Bug adding URL property to Dataset

• identify what types it should support (looks like a text box is presented as an option)
• adding something and the box disappears - what's happening?

[Documentation] Wiki unclear about session creation in embedded mode

Regarding Integrating Describo Online into owncloud: In the initial definition of the POST session creation object, the key in L:4 is given as service, whilst the example below shows the key as session.

Looking at 23e2fca, I'm assuming that service is the currently correct key name.

Type part of what you're looking for to select values from the SelectObject list

In the text commons profile there's a looooong list of Types (yes, we'll talk about making it shorter) - it would be good to be able to type to select from the list. Eg if it type "orth" show me all the things with Orthographic in their name.

Text boxes should auto-save when they lose focus

When the user clicks away from (say) the description field to another field it should auto save if possible.

[Embedded mode] "Loaded collection: my Research Object Crate" although the crate is not called `my Research Object Crate`

When loading the following ro-crate in embedded mode

{
  "@context": "https://w3id.org/ro/crate/1.1/context",
  "@graph": [
    {
      "@type": "CreativeWork",
      "@id": "ro-crate-metadata.json",
      "conformsTo": {
        "@id": "https://w3id.org/ro/crate/1.1"
      },
      "about": {
        "@id": "./"
      },
      "identifier": [
        {
          "@id": "#:localid:describo:c20b02de-6832-4aa5-b8ba-a3f569b79809"
        }
      ]
    },
    {
      "@id": "./",
      "@type": "Dataset",
      "name": "RO Crate Name Test",
      "@reverse": {}
    }
  ]
}

I get the following screen that shows the default crate name instead of the one that is contained in the crate:

Is this expected behaviour?

[Feature] Dark mode option in embedded mode

Element plus offers a native dark mode, which could be triggered by e.g. window.postMessage: https://element-plus.org/en-US/guide/dark-mode.html

Show the name above the map for Geo locations

This has a name but it is not showing alongside the prop (also the map or the yellow box do not seem to be click-to-edit-able - I was not able to get back to the data entry screen to edit it.