tudelft3d/3dcitydb-tools-for-qgis

This repository contains code of a QGIS plugin that facilitates data handling of the CityGML 3D City Database.

3dcitydb-tools-for-qgis's Issues

Bug when adding but not storing a new connection

Example:

Open the Layer Loader, choose to add a new connection.
Fill out the parameter, test it.
Press OK, but without saving it.

Then connect. This will cause an error.

Add addresses to layers and/or to children tables in attribute forms

As the title says, addresses (with point geometries) could be added to the list of layers to be loaded - if there are data for them.

Add support for ADEs

Might not be that easy to implement for any ADE, but it would be great to be able to deal with ADE-related data...

Can't delete waterbody thematic surfaces

There are some issues in the trigger functions.
This is due to the wrong string that has to be used to extract the name of the cdb_schema from the layer name.

Import selected layers button should be disabled when selecting a new LoD

In the Layer Loader, in Layers tab, you select for example the Feature type "Building". Then you select a LoD, and then you can pick from the list of available layers which to import.

If you change the LoD, the import button should be disabled till you select again a layer.

Can't update WaterSurface layer

This is due to:

• Wrong strings to be used to extract the name of the cdb_schema in the trigger function
• Wrong function names to update attributes (_atts function)
• Syntax error in table update function (:= instead of =)

QGIS PostgreSQL connections are not updating in "Layer loader" dialog

When adding a new db connection (or removing one), it is not updated at the "PostgreSQL existing connection" comboBox at the "Layer loader". At the other two ("Bulk deleter" "QGIS Package Administrator") the connection is updated as expected.

To recreate it:

(Adding new connection)

1. Make a clean installation of the plugin.
2. Open "QGIS Package Administrator"
3. Press "New Connection"
4. Add new connection credentials and click OK.
5. Check comboBox of existing connections at "Layer loader" (you should not be able to see the new connection)
   5.1 Check comboBox of existing connections at "Bulk deleter" or "QGIS Package Administrator" (you should be able to see the new connection)

(Removing connection)

1. Check the existing connections comboBox "Layer loader"
2. Remove one of those connections (from QGIS)
3. Check again the existing connections comboBox "Layer loader". The connection that you removed is still there (and you can also connect to it - connection info are still stored in object inside the comboBox)

Ad hoc solution: Reload the plugin

Uninstall error and unintended consequences

ERROR: local variable 'usr_name' referenced before assignment

How did i notice it?
I had an old server side version which I wanted (as prompted) to uninstall in order to install the newer supported one.

How to recreate it:

1. Go to QGIS Package Administrator

2. Connect to a database that DOES NOT have the main installation but has some data in it.

3. In Main installation click the install button. 2023-11-18T11:38:36 SUCCESS Schema 'qgis_pkg' installed

4. Add user to the group as normal. 2023-11-18T11:38:38 INFO User 'postgres' has been succesfully added to the database group 'qgis_pkg_usrgroup_citydb_vienna'

5. Create schema for said user as normal. 2023-11-18T11:38:40 SUCCESS Schema 'qgis_postgres' installed

6. Grand all privileges to user. 2023-11-18T11:38:44 INFO For user 'postgres', read & write privileges granted to ALL citydb schemas.

7. Close the dialogue and open the "Layer Loader"

8. Follow the normal process to import any layer form the database. 2023-11-18T11:39:17 SUCCESS Layer(s) successfully imported

9. With the layer(s) loaded close the dialogue and go back to the QGIS Package Administrator.

10. Connect to the database where the loaded layers are coming from

11. In Main installation, click the "Uninstall from ..." button.
    2023-11-18T11:39:31 CRITICAL Establishing temporary connection ERROR at 3DCityDB-Tools-for-QGIS/cdb4/gui_admin/functions/threads.py>uninstall_thread_qgis_pkg_current ERROR: local variable 'usr_name' referenced before assignment

12. If you click it again, NO ERROR! 2023-11-18T11:40:07 SUCCESS Schema 'qgis_pkg' uninstalled

However, now the connection to the loaded layer(s) is broken so an avalanche of errors from QGIS's PostGIS service is falling to the user. Even if we were to re-install the server-side with the exact same settings as before, the leftover layer(s) are going to remain broken and throwing errors.

So my proposal is that when users opt-out we clean their project from any connected layers.
Although in cases where the user is decoupled from administrator (not the same person/computer) this might be impossible.

I think it is quite important to address it as updating from older server-side version by uninstalling them and installing the new ones is going to be a semi-frequent process for all users.

So first lets fix the ERROR because it persist even if you delete the loaded layers when attempting to uninstall.

• Fix ERROR (bug)
• [Clean project from connected layers in the "Uninstall from '{db}" event. (enhancement)]

PS. I had a deja vu about this error

User should not be able to import selecter layers if there are no selected layers.

How to recreate:

1. Connect to a database with generated layers

2. In "Layers" tab -- the "Import selected layers" is disabled (expected correct behavior)

3. Select one of the available features -- the "Import selected Layers" is enabled (expected correct behavior)
   

4. Set a different extent (e.g. click the "Set to layer extents" button) --
   the previously selected available feature is unselected (expected correct behavior) HOWEVER the "Import selected Layers" is still enabled (unexpected error-prone behavior)
   

Load multiple times the same layer

Currently, it is possible to load several times the same layer in the ToC.

Dropping layers (server-side) while layers are loaded in QGIS must be prevented

Already loaded layers will lose the respective server-side updatable view if the layers are dropped by the user via the Layer Loader dialog.

If this is the case, the loaded layers (actually the whole QgsLayerTreeGroup) must be dropped

Wring names in qml files for Building/Tunnel/Bridge Furniture

*_furniture_form.qml must be renamed to *_frn_form.qml

Add external references to attribute forms of features

As mentioned in the title, it should be possible to view and edit (if allowed) external references associated to cityobjects.

Issues when appending data to database

What is the bug or the crash?

First of all, thank you for your efforts and initial contributions + leadership in prodiving this plugin for handling data from 3DCityDB-databases.

As I I've been trying the plugin recently, it seems to be that, there might be an issue when you append more data to your database and you want to import all the new data to QGIS with 3DCityDB-QGIS-loader. This could potentially be a conceptual issue, where I haven't fully grasped the underlying database features, resulting in my inability to use the plugin appropriately.

Issue description

Let me describe the use case and my flow of thought: my use case is related to appending data to the same database and importing the data to QGIS. In other words, I need to update the materialized views when I append data with Importer/Exporter(3dcitydb/impexp). It seems to me that spatial extent of the data within the citydb-schema is not updated as wanted and maybe that's why the materialized views are not updated accordingly.

Workflow:

1. First I loaded some initial .gml data with the Importer/Exporter from the southern Finland.
2. Then I appended the data with another dataset from the South of Finland (see below in blue) and
3. finally, I appended the data with a third .gml file corresponding to the West of Finland.

To clarify, let me share some screenshots.

With blue we can see all the data in the citydb.surface_geometry-table and with red the data that was imported in first instance with 3DCityDB-QGIS-loader

So the problem is that I can't refresh the red-coloured LOD2-layer to the full extent of the data. After the first import somehow the 3DCityDB-QGIS-loader does not update the map extent to the full spatial extent of the data:

The Set to schema does not result to extend the area to cover the whole spatial extent of citydb.surface_geometry.

Versions

I'm using QGIS version 3.26.3 in Linux (Pop!_OS 22.04). I'm using 3DCityDB with Docker (3dcitydb/3dcitydb-pg:latest (CITYDBVERSION=v4.3.0) and 3dcitydb/impexp:latest (v5.2.1)) and v0.5 of the plugin).

Thank you for any help, hope this is useful, and please, carry on the great work.

Catch error when citydb schema(s) are all empty

If the citydb has been correctly installed but it is empty, or there are multiple citydb schemas and they are all empty, the user client will raise an error.

Rework attribute form of generic attributes

At the moment, it is rather user UNfriendly as generic attributes can be explored and - most importantly - edited. The user must know about the way the cityobject_genericattrib table is structured, which is far from being intuitive if you have no previous experience.
Such complexity should be hidden "somehow"

OPEN List of ideas and suggestions for future releases :-)

Dear 3DCityDB Tools plug-in users,

If you are using our plug-in and you like it or find it useful, let us know!
If you want, you are heartily invited to add here ideas and suggestions for future improvements.

Remember: The sky is the limit!! ;-)

Add more GUI blocks to avoid user panic

In the Layer Loader, during layer creation or refreshing, more dialog widgets should be disabled while the thread is on-going.

E.g.: PostgreSQL connection groupbox, CityDB groupbox, Basemap groupbox, Close connection button.

Of course, to be re-enabled as soon as the thread finishes.

Issues with compound EPSG

With some compound EPSG, determining wether the planar coordinates are project or not may lead to wrong results.

Example: EPSG: 6697

Planar coordinates are in degrees, but QgsCoordinateReferenceSystem(crs).isGeographic() will return False.

This affects the way the bbox of the 3DCityDB is computed.

The underlying function qgis_pkg.compute_cdb_schema_extents() must be changed.

Possibility of deleting layers with empty citydb schemas

It should be a possibility to delete existing layers that were created before dropping a the 3dcitydb schema. If the user decided to not drop the existing layers before cleaning the schema it work be allowed to do so. For example:

1. User imports the data to an specific schema inside the DB
2. User creates and refreshes the layers of the corresponding schema
3. User clean the schema, like using the cleanup_schema() function
4. when trying to connect from the plug-in to the schema it won't work since it is empty but the mat views are still created and populated.

reading Generic attributes from a different schema

Although the plug-in supports multiple 3dCityDB schemas from the same database, the generic attributes are linked and loaded only from the default schema (citydb), it should be dynamic.

Add support for line and point based layers

Example: MassPoint Relief, Breakline Relief

Or: Terrain Intersection Curve, or the MultiCurve geometries of Buildings, Bridges, etc.

Changing user from rw to ro doesn't apply directly

Tested Scenario (imagine Admin and user are different persons/computers):
Admin grants RW permissions to user X
User X load layer(s)
Admin wants to revoke all user X's permissions
User X still has its RW layer(s) on his project and he can STILL make changes and edits

We need to come with a plan to block user X's RW permissions
From the top of my head, in the server-side should be quite easy but for the client side I am not really sure what to do.
I thinks this is a quite severe security risk

The same scenario could be applies for the reverse situation:
User X starts with RO permissions until admin grants him RW.
User X still can't edit the layer(s) without deleting and loading from scratch.

When cleaning up schema, sequences are not restarted correctly

This is a bug as of version 0.7.0.

Unless you are using very big databases, it won't cause any harm when reloading data.

This bug will be solved in the next minor release.

Add/extend Admin GUI to allow for user rights management

At the moment, it can be done only via a SQL console (e.g. via PgAdmin), but it could be made easier via - for example - a GUI Tab in the Admin dialog

Can't delete solitary vegetation objets from layers

It's a bug due to an error in the delete trigger functions that are invoked when trying to delete features from a layer.

Add check for PostgreSQL version < 10.0

Upon connection to install the QGIS Package, add a check that blocks installation if PostgreSQL version is < 10.0

Refine selection at FeatureType level before creating layers

This could reduce the time needed to generate the materialized views for geometries, if someone does not need to work with all existing layers.
E.g.: I only need buildings, I only generate building layers. No need to wait and generate TINs, Bridges, etc.

Ideally, this could be further narrowed down to root classes.

PostgreSQL users without password cannot login via plugin gui

The Add connection dialog requires a password which is not null.
On some installations (e.g. on MacOS), the default user is the same as the OS user, and it may have no password.

Catch error when no cdb_schemas can be accessed due to insufficient privileges

Depending on the privileges of a user, it may not be possible to access any existing cdb_schema, even though they exist in the database.

Currently, the plugin crashes in such a case.

Add possibility to select layers also based on top-level features

At the moment, a user can refine layer generation up to Feature type level (i.e. CityGML modules).
It'd be nice to further refine this possibility, similarly to what is already available in the Bulk Deleter.