geosmartcity-cip/gsc-datacatalogue

GeoSmartCity Data Catalogue

gsc-datacatalogue's Issues

Parameters for a valid datasource on the gsc-catalogue host server

@AndreaDiNora, @giannibarrotta When issuing the command listDataOrigin I suppose it is the intention that the web service shall try to connect to the remote resource and retrieve potential datasets? In this case, do you have a valid path to a directory of ESRI Shapefiles - or to a PostgreSQL/PostGIS database table?

Update/Assign layers/group to application layers parameter

According to "11.2 Update/Assign layers/group to application", for each layer there are some parameters as:

queryable
basic,
overview

these parameters are useful to set info on client maps

Assign permission don’t show all functions

Q: There are admin functions as map functions that will be show if you use parameter includeadminrecords

A: I have made it so that the system functions are included by default now.

Make description mandatory on application

@AndreaDiNora : description parameter is mandatory for application form

Register user does not accept anything but sinergis.it addresses

@AndreaDiNora, @giannibarrotta When trying to register a user with an e-mail address other than [email protected] a message is returned that the 'specificed email address does not exist'.

Client needing update

Current client functionalities have some problems because service have been updated.

Add schema param to Postgis datasource

We need to add also schema params in postgis datasource.

Verify the updated wiki.

Validation of SLD, Metadata fields

Q: We should probably find some way of validating the content of the sld and metadata files? Presently I am loading text into the fields using the HTML5 FileAPI

A: Yes can be useful, but I think that at this moment there are many other problems

Q: Agreed, 2nd priority.

User access control and restrictions in user interface

Q: According to wiki:
Three possible functions categories: application, data catalogue, layers.

• Application functions will be use by client applications.
• Data catalogue functions will be use to access data catalogue sections and activate specific functionality
  . Layers functions will be use to access specific action on layer, for example (read, write…)

A/Q: I have added type to the functions section but am unable to use type for anything since it is not returned by the list request.

A: Ok

In particular:

Data catalogue functions will be use to access data catalogue sections and activate specific functionality
I don’t see any control connected to this functions.
Admin can see admin sections as user controls panel. Any other user without specific function cannot see this panel.

A: I will use role to determine this but I didn’t know whether they were fixed with their final names as I need to use these as a fixed enumeration to make the checks.

A: These are current static functions common to all organization.

They cannot be edit.

A correction to what was said earlier, there are only two types of functions, Data Catalogue (code datacatalogue) and Client application (code map) the third type are in fact permits to give to the layer for single application.

Check “11.2 Update/Assign layers/group to application” of the wiki for more info, for example queryable is a parameter of layer that show the possibility to perform a wfs query on this layer on client.

A: OK – I receive these roles when I authenticate the user – but my users are not members of any roles and I am not able to get the function to add him to a role to work.

Simple layer switcher control for gsc.js that can parse config file

Response of Permissions web service, action: listperm

Q: The function that lists permissions for roles returns an array without wrapping it in a response object. This is handled in the application but is different from all other Web Services.

A: Ok

Cannot list layers by organization, Layers web service, action: listlyr

Q: The function to list layers does not accept organization alone as a parameter but requires specifying datasetId or layerId in addition. It would be convenient to be able to see all layers for an organization without these filters.

A: Ok

Add value "auto_update" to "Data origin" combo

We need another value on combo in data origin in edit and create dataset page to allow ingestion
this value is "auto_update". The description of this value could be "Ingest from url".

Form to assign permissions

Q: Add permission to role force the user to choose also layer and application.
There are permission not bounded by application and layer, for example show a specific control on client, or other functions type as say before.

A: I can only do this once the function returns the type so that I can differentiate between functions that relate to layers, applications.

Q: Remove layer in this screen, because the permissions for Layer will be indicated by assigntoapp service (see 11.2 Update/Assign layers/group to application on wiki page)

A: Done

Level of detail for response from list operation on applications

Q: The function that lists applications does not list all details about the application - in order to populate the editing form this would be convenient

A: There is the search for id for details. If we return all details in application list we return too much data because the application config can be complex

Q: I now load the application using the idapplication parameter but I still do not get back any info

A: Ok

Current organizations not shown when editing user: Web Service: user, action: listuser

Q: bug on edit user page

• the page don't show the organizations saved for the user selected
• can you try on your local and verify if is a real bug or no?

What are the valid values for DatasourceType

@AndreaDiNora, @giannibarrotta I have temporarily used

• Oracle
• ESRI Shapefile
• PostGIS

Add link to PDF user guide in application

@monstruil: Suggest to create and link to PDF user guide in user interface.

Also to be assessed, inline captions, lead-texts, placeholder text

Structure of repository: need room for both client (HTML5/JavaScript) and server (Java)

With the present repository layout the server code is placed directly on the root. I have made a sub-folder called "client" where I will commit the client code.

Return type as part of Function web service, action: listfunc

Q: The function that lists 'functions' does not return organization id, type

A: List function service require an organization as parameter, why do you need an organization id? Only function without an organization id are preconfigured admin functions showed with includeadminrecords to true. These functions are not linked to any organization.

Q: merely because it would be consistent with the behavior of other services. However, I can hide organization from the function form altogether and just use the current organization for all CRUD operations.

Type is missing from the response. Required to distinguish which other fields to show.

A: Ok

Clear data entry forms when navigating between tabs

Q: Clean a previous edit

If I enter in edit mode, datasource, dataset etc, when change tab e reenter without select the data, I found previous settings, so user to create a new data, must click on clear. Can be useful clear section when enter on it from tab.

A: Yes – I understand. I initially ran a function to clear the form when the tabs were clicked but it turned out that function also was invoked when I changed the focus programmatrically. When I had populated the model of the form it was suddenly empty. I will find a solution to this. I need some more thinking on this issue.

Add loading icon whenever loading data from server

Need to be able to select 'current organization' for logged in user

Q: Configuration of data is strictly connected to organization. One user of an organization cannot see data of another organization.

Use an admin account to create data for different organization is a bit an error, but in this case we need a combo where we can set the organization so the admin can see data for this specific organization and no others.

A: I will add a drop-down that permits users to select the current organization id of theirs if they have several.

SLD support on GeoServer

Missing datasources parameters

Q: Where are the parameter of db configuration?

There are some examples of full parameters request on wiki

A: They had fallen out due to a typo in the code

Error on Role web service, action: assignrole

Q: The list role function does not return associated users – I need this list to be able to edit the number of users of an existing role

A: The list of users can be found on response of service. Try:
{ "organization": "872" }
Do you have a role with user assigned?

A: Aparently, that was the problem. The listing works now – but an error persists whenever I try to add a new user to the SystemAdmin role:

{"status":"error","description":"One of the specified id parameters is not a number.","request":{"idrole":1,"users":[{"iduser":1,"username":"Admin"},{"iduser":405,"username":"runar"}]}}
The same function works for other roles.

Manage group layers error on description parameter

When create a group, instead of description, the request contain organization with the value of description.

Description missing on response, Group Layer web service, action: listgrp

Q: The function to list group layers does not return the group layer description, also (layers appears to be missing if the layer was created with layers that do not belong to the same organization)

A: Is not a problem return also description.

Layers of another organization must not showed. Organization are important to aggregate data of same pilot.

Q: There does not appear to be any method to edit group layer name or description

A: Yes, this is true, if you think that is useful we can create this service.

Make screen capture demo with voice over - demonstrating user case

@monstruil: Suggest to make screen capture demo that illustrates the end-to-end workflow using the data catalogue.

What is the parameter name for verifying a user email?

@AndreaDiNora, @giannibarrotta No parameter name is specified in the test set specification. The error message suggets that it should be 'id'.

Functions section lack type

Q: Always according to wiki, there are three types of functions. Function section, don’t give possibility to choose it.

Persistent sort order for list requests

Probably best if they are sorted by name?

Publish on Geoserver

We added to wiki some examples of datasource and dataset creation for publish to Geoserver.

Datasource:
https://github.com/GeoSmartCity-CIP/gsc-datacatalogue/wiki/GSC-Data-Catalogue-Services#71--create

Application:
https://github.com/GeoSmartCity-CIP/gsc-datacatalogue/wiki/GSC-Data-Catalogue-Services#111-create

It is clear that you need also datasets and layers but their creation is standard.

Placement and operation of getConfiguration method in data catalogue

Q: Where should we implement the get configuration method - mainly in client applications I assume?

A: Correct

Handle mandatory empty arrays for groups and layers in application form

List of preliminary issues after integration

@AndreaDiNora, @giannibarrotta

The following are questions that needs to be resolved. The issue may be on the server or it may be that the client is passing the wrong parameters. We should verify this in a screen-by-screen walk-through of the service at your earliest convenience.

The issues that I have identified are as follows:

• What are the conditions when the publish (to CKAN) button should be displayed for data sources. Presently it is shown in the action column for all data sources - but obviously this is not the intention?- - [ ] A single request to create dataset creates two duplicate datasets... This may be wrong on the client end - but I am unable to see what I am doing wrong?
• The function to return column names for some layers/datasets reports that no columns could be found?
• It works as exepected e.g. for 'Luxembourg today' and 'Test data set (4)'
• It does not work for 'gianni3'
• The function to list group layers does not return the group layer description, also (layers appears to be missing if the layer was created with layers that do not belong to the same organization)
• The function to list layers does not accept organization alone as a parameter but requires specifying datasetId or layerId in addition. It would be convenient to be able to see all layers for an organization without these filters.
• There does not appear to be any method to edit group layer name ordescription?
• The function that lists 'functions' does not return organization id
• As with the 'publish to CKAN' button, When to display the publish (to GeoServer) button for applications? Presently it is shown for all. Also, this service does not return valid JSON
• This is not an error - but we should probably find some way of validating the content of the sld and metadata files? Presently I am loading text into the fields using the HTML5 FileAPI
• General question: is it an error when there are no elements for list? Some of the response objects return error when there are no data to be returned for a list request. It could be ok with 'done' and an empty array.
• The function that lists applications does not list all details about the application - in order to populate the editing form this would be convenient
• The function that lists permissions for roles returns an array without wrapping it in a response object. This is handled in the application but is different from all other Web Services.
• Where should we implement the get configuration method - mainly in client applications I assume?

Verify if organization returns users as part of response

Otherwise it might be useful to have a mechanism to list users directly

Application section

Hi,
we need to add to client also the Application section.
This section is important to publish on Geoserver for two reasons:

1. Application is the container for all layers that we want publish. It is equivalent to Geoserver's workspace. For this reason application has three optional parameters for geoserver (url,username and password).
2. Application section is the right place where we can put the button to publish on geoserver, because usually we dont want publish a single layer, but some layers with a common logic (same application for example)

When to display publish to GeoServer button

Q: As with the 'publish to CKAN' button, When to display the publish (to GeoServer) button for applications? Presently it is shown for all. Also, this service does not return valid JSON

A: It’s ok on applications list

Publish to Geonetwork API from front-end

Q: What are the conditions when the publish (to CKAN) button should be displayed for data sources. Presently it is shown in the action column for all data sources - but obviously this is not the intention?

A: Publish on Ckan must be showed when datasource (or dataset can be used as ckan data). For example a single shapefile is a dataset that can be be published on Ckan.

Q: Ok –are we able to determine that from some analysis of the other properties so that I can hide show the button? Should probably discuss.

A: Ok

List user function does not return associated roles?

Q: The list user function does not return associated roles – that means I cannot edit other users than the currently logged in one.

A: User can be assigned to Roles only by admin.

The flow is this:
Admin use listrole (4.3 Search/List) to list all roles with all user associated, with assignrole (4.4 Assign/remove users to role) admin, can edit list users associated to a rule.

A: This flow is not implicit. Users can create roles that are specific to organizations – but need to have the aid from an admin to assign users to it? We need to discuss this flow.

Update user requires username to remain unchanged

@AndreaDiNora, @giannibarrotta The update method in the user web service requires the username to remain unchanged. This may be intentional behavior - but in such case, what is the rationale for keeping the id parameter? As additional security?

List organizations - query parameters

Now accepts only organization name - not id.

This is ok - but if there are two organizations with partially the same name - they will both be returned.

Column names some times not found?

Q: The function to return column names for some layers/datasets reports that no columns could be found

Steps to reproduce:

• It works as exepected e.g. for 'Luxembourg today' and 'Test data set (4)'
• It does not work for 'gianni3'

A: There is a bug on our service that delete columns when update the dataset. So you can see the columns only on dataset not updated.

Force users to keep at least one organization

Q: Login service accept username, no email. I cannot use my username to access to application.
Is [email protected] cabled?

A: [email protected] and its password is hard-coded for development purposes

The user.login service returns user data directly and not a status message. Therefore it failed. I have now altered it so that it checks for a numeric iduser on the response – if it is present it interprets it as success.

Q: Problem persists after checking out most recent version

A: Looks like gsc.js library is out of date on server.

Remove organization drop-down from application form

@AndreaDiNora : drop-down is redundant as organization already selected in left-bar.

Should an empty list response return status="error"?

Q: General question: is it an error when there are no elements for list? Some of the response objects return error when there are no data to be returned for a list request.

Could it be ok with 'done' and an empty array. It has some consequence for how responses are parsed.

A: Can you give me an example?

Q: An error is returned if listperm is executed with role as input parameter?

A: Ok

Organization field in Datasource page

In Datasource create/update page, organization is a normal field. This must be removed because, the user has already an organization (choosed from combo).

List data source method requires iddatasource / datasourceId as input parameter

@AndreaDiNora, @giannibarrotta This means a list request will only return exactly ONE record. This function needs to return all datasources for the logged in user and/or his organizations?

Duplicate data sets created

Q: A single request to create dataset creates two duplicate datasets... This may be wrong on the client end - but I am unable to see what I am doing wrong?