geonodeusergroup-de/geonodectl

a commandline interface tool for Geonode Rest API v2

geonodectl

geonodectl is a commandline interface tool for geonode. It uses the geonode apiv2 to interact with a geonode installation.

How to use

geonodectl has currently the following capabilities:

usage: geonodectl [-h] [--not-verify-ssl] [--raw] [--page-size PAGE_SIZE]
                  {resources,resource,dataset,ds,documents,doc,document,maps,geoapps,apps,people,users,user,uploads,executionrequest}
                  ...

geonodectl is a cmd client for the geonodev4 rest-apiv2.
To use this tool you have to set the following environment variables before starting:
  
GEONODECTL_URL: https://geonode.example.com/api/v2/ -- path to the v2 endpoint of your target geonode instance
GEONODECTL_BASIC: YWRtaW46YWRtaW4= -- you can generate this string like: echo -n user:password | base64

positional arguments:
  {resources,resource,dataset,ds,documents,doc,document,maps,geoapps,apps,people,users,user,uploads,executionrequest}
                        geonodectl commands
    resources (resource)
                        resource commands
    dataset (ds)        dataset commands
    documents (doc,document)
                        document commands
    maps                maps commands
    geoapps (apps)      geoapps commands
    people (users,user)
                        people|users commands
    uploads             uploads commands
    executionrequest    executionrequest commands

options:
  -h, --help            show this help message and exit
  --not-verify-ssl      allow to request domains with unsecure ssl certificates ...
  --raw, --json         return output as raw response json as it comes from the rest API
  --page-size PAGE_SIZE
                        return output as raw response json as it comes from the rest API

Currently not all features of the API are implemented. Here is a list of what you can do with geonodectl:

geonode resource	capabilities
resource	list, delete, download metadata
dataset	list, delete, patch, describe, upload
documents	list, delete, patch, describe, upload
maps	list, delete, patch, describe
geoapps	list, delete, patch, describe
people	list, delete, patch, describe
uploads	list
executionrequest	list

This project is WIP, so feel free to add more functions to this project.

Usage

first install the project with:

pip install  -e 'git+https://github.com/GeoNodeUserGroup-DE/geonodectl.git@main#egg=geonodectl'

Additionally to package install, geonodectl requires to set two environment variables to connect to a geonode instance like:

GEONODECTL_URL: https://master.demo.geonode.org/api/v2/ # make sure to supply full api url
GEONODECTL_BASIC: dXNlcjpwYXNzd29yZA== # you can generate this string like: echo -n user:password | base64

Now you are ready to go. upload shape file:

❯ ./geonodectl ds upload -f ~/data/geolocation.shp -t example-shape
| key     | value                                       |
|---------|---------------------------------------------|
| title   | example-shape                               |
| success | True                                        |
| status  | finished                                    |
| bbox    | 13.1832819,52.4059715,13.5891838,52.5867805 |
| crs     | {'type': 'name', 'properties': 'EPSG:4326'} |
| url     | /catalogue/#/dataset/36                     |

show all datasets:

❯ ./geonodectl dataset list
|   pk | title                       | owner.username   | date                        | is_approved   | is_published   | state     | detail_url                                              |
|------|-----------------------------|------------------|-----------------------------|---------------|----------------|-----------|---------------------------------------------------------|
|   36 | example-shape               | admin            | 2023-02-06T14:52:31.991113Z | True          | True           | PROCESSED | https://geonode.corki.bonares.de/catalogue/#/dataset/36 |
|   35 | gps_mastertable_prieros2015 | thomas           | 2023-02-06T14:16:45.375526Z | True          | True           | PROCESSED | https://geonode.corki.bonares.de/catalogue/#/dataset/35 |
|   34 | layer                       | thomas           | 2023-02-06T10:08:12.182176Z | True          | True           | PROCESSED | https://geonode.corki.bonares.de/catalogue/#/dataset/34 |
|   33 | a__30                       | admin            | 2023-02-03T13:18:27.715898Z | True          | True           | PROCESSED | https://geonode.corki.bonares.de/catalogue/#/dataset/33 |
|   28 | test                        | admin            | 2023-02-03T11:30:00.609472Z | True          | True           | PROCESSED | https://geonode.corki.bonares.de/catalogue/#/dataset/28 |
|   13 | geolocation3                | mwall            | 2023-02-02T12:15:25.477127Z | True          | True           | PROCESSED | https://geonode.corki.bonares.de/catalogue/#/dataset/13 |
|   12 | geolocation2                | mwall            | 2023-02-02T11:53:19.231994Z | True          | True           | PROCESSED | https://geonode.corki.bonares.de/catalogue/#/dataset/12 |
|   11 | geolocation1                | mwall            | 2023-02-02T11:51:28.975906Z | True          | True           | PROCESSED | https://geonode.corki.bonares.de/catalogue/#/dataset/11 |
|    6 | geolocation0                | mwall            | 2023-02-02T11:03:06.859857Z | True          | True           | PROCESSED | https://geonode.corki.bonares.de/catalogue/#/dataset/6  |
|    5 | arh                         | admin            | 2023-01-31T09:11:00Z        | True          | True           | PROCESSED | https://geonode.corki.bonares.de/catalogue/#/dataset/5  |
|    4 | data_00                     | admin            | 2023-01-25T15:56:05.026049Z | True          | True           | PROCESSED | https://geonode.corki.bonares.de/catalogue/#/dataset/4  |
|    3 | geolocation                 | admin            | 2023-01-25T15:23:44.439151Z | True          | True           | PROCESSED | https://geonode.corki.bonares.de/catalogue/#/dataset/3  |
|    2 | data_0                      | admin            | 2023-01-25T15:01:51.042680Z | True          | True           | PROCESSED | https://geonode.corki.bonares.de/catalogue/#/dataset/2  |
|    1 | wheaterdata 2004            | admin            | 2023-01-23T10:19:00Z        | True          | True           | PROCESSED | https://geonode.corki.bonares.de/catalogue/#/dataset/1  |

patch dataset:

geonodectl ds patch 36  --set 'category={"identifier": "farming"}'
...

patch dataset from jsonb:

geonodectl ds patch 36  --json_path 'path_to/your_json_with_attributes_to_patch.json'

delete dataset:

❯ ./geonodectl ds delete 36
deleted ...

check if deleted:

./geonodectl ds list
|   pk | title                       | owner.username   | date                        | is_approved   | is_published   | state     | detail_url                                              |
|------|-----------------------------|------------------|-----------------------------|---------------|----------------|-----------|---------------------------------------------------------|
|   35 | gps_mastertable_prieros2015 | thomas           | 2023-02-06T14:16:45.375526Z | True          | True           | PROCESSED | https://geonode.corki.bonares.de/catalogue/#/dataset/35 |
|   34 | layer                       | thomas           | 2023-02-06T10:08:12.182176Z | True          | True           | PROCESSED | https://geonode.corki.bonares.de/catalogue/#/dataset/34 |
|   33 | a__30                       | admin            | 2023-02-03T13:18:27.715898Z | True          | True           | PROCESSED | https://geonode.corki.bonares.de/catalogue/#/dataset/33 |
|   28 | test                        | admin            | 2023-02-03T11:30:00.609472Z | True          | True           | PROCESSED | https://geonode.corki.bonares.de/catalogue/#/dataset/28 |
|   13 | geolocation3                | mwall            | 2023-02-02T12:15:25.477127Z | True          | True           | PROCESSED | https://geonode.corki.bonares.de/catalogue/#/dataset/13 |
|   12 | geolocation2                | mwall            | 2023-02-02T11:53:19.231994Z | True          | True           | PROCESSED | https://geonode.corki.bonares.de/catalogue/#/dataset/12 |
|   11 | geolocation1                | mwall            | 2023-02-02T11:51:28.975906Z | True          | True           | PROCESSED | https://geonode.corki.bonares.de/catalogue/#/dataset/11 |
|    6 | geolocation0                | mwall            | 2023-02-02T11:03:06.859857Z | True          | True           | PROCESSED | https://geonode.corki.bonares.de/catalogue/#/dataset/6  |
|    5 | arh                         | admin            | 2023-01-31T09:11:00Z        | True          | True           | PROCESSED | https://geonode.corki.bonares.de/catalogue/#/dataset/5  |
|    4 | data_00                     | admin            | 2023-01-25T15:56:05.026049Z | True          | True           | PROCESSED | https://geonode.corki.bonares.de/catalogue/#/dataset/4  |
|    3 | geolocation                 | admin            | 2023-01-25T15:23:44.439151Z | True          | True           | PROCESSED | https://geonode.corki.bonares.de/catalogue/#/dataset/3  |
|    2 | data_0                      | admin            | 2023-01-25T15:01:51.042680Z | True          | True           | PROCESSED | https://geonode.corki.bonares.de/catalogue/#/dataset/2  |
|    1 | wheaterdata 2004            | admin            | 2023-01-23T10:19:00Z        | True          | True           | PROCESSED | https://geonode.corki.bonares.de/catalogue/#/dataset/1  |

geonodectl's Issues

Add official pip package

make this package available via pip install geonodectl

allow patch json from file

Adding an argument to the patch function that allows to patch changes from a given json file.

add examples and docs for library usage

As this package can also be used as a library for interacting with the geonode api, more documentation and examples would be helpful to make usage more easy for new developers.

listing exectionrequests returns error

when running: ./geonodectl executionrequest list the following error occures.

Traceback (most recent call last):
  File "/home/mwall/Sourcecode/geonodectl/./geonodectl", line 517, in <module>
    sys.exit(main())
             ^^^^^^
  File "/home/mwall/Sourcecode/geonodectl/./geonodectl", line 513, in main
    g_obj_func(**args.__dict__)
  File "/home/mwall/Sourcecode/geonodectl/src/executionrequest.py", line 49, in cmd_list
    GeonodeObjectHandler.print_list_on_cmd(obj)
  File "/home/mwall/Sourcecode/geonodectl/src/geonodeobject.py", line 115, in print_list_on_cmd
    values = [
             ^
  File "/home/mwall/Sourcecode/geonodectl/src/geonodeobject.py", line 116, in <listcomp>
    generate_line(i, obj, cls.LIST_CMDOUT_HEADER) for i in range(len(obj))
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mwall/Sourcecode/geonodectl/src/geonodeobject.py", line 113, in generate_line
    return [cmdoutkey.get_key(obj[i]) for cmdoutkey in headers]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mwall/Sourcecode/geonodectl/src/geonodeobject.py", line 113, in <listcomp>
    return [cmdoutkey.get_key(obj[i]) for cmdoutkey in headers]
            ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mwall/Sourcecode/geonodectl/src/geonodetypes.py", line 46, in get_key
    return ds[self.key]
           ~~^^^^^^^^^^
KeyError: 'pk'

Improve error handlling

when:

• geonode host is not available
• geonode host doesnt answer in time
• configuration missing

patching data to geonode rest API

When patching complex metadata to geonode rest API like:

geonodectl ds patch 1  --set 'related_identifier=[{"related_identifier": "test4","related_identifier_type": {"label": "ARK"}, "relation_type": {"label": "Continues"}}]'

(example from future zalf metadata schema) The REST-API receiving and <class 'django.http.request.QueryDict> object that looks like:

{'related_identifier': ['[{"related_identifier": "test4"}]']}

The value for the related_identifier in this case is a string. BUt it should be a dict. This is related to the way geonodectl uses requests data parameter in the rest interact. This has to be changed to use the json parameter.

Furhter it might be helpful to let the users dump a raw json string not the key=value implemention.

doc upload fails with error 400

Hello,

I'm using geonodectl v0.1 installed via pip from Github as described in the readme of this repo.

When attempting to upload a document to a Geonode server, I get the following error message:

ERROR:root:{'success': False, 'errors': ['Dieses Feld ist erforderlich.'], 'code': 'invalid'}
400 Client Error: Bad Request for url: https://tisdar.dev.thuenen.de/api/v2/documents

The command was geonodectl doc upload -f archive.tar. Attempts with other file types (PNG) failed too.

However, via the web interface of the Geonode instance I could upload the same archive.tar document

I was able to run subcommands doc list, users list, ds list, ds upload successfully. With ds upload I uploaded a test KML file without errors, using the same credentials.

Could you please help? It's not clear to me which metadata field is missing and how this can be supplied as an argument to geonodectl.

add people create function

add function to people (users) to geonode

Issue describing users

❯ geonodectl users describe 1000`

/geonodectl users describe 1000
Traceback (most recent call last):
  File "/home/mwall/Sourcecode/geonodectl/./geonodectl", line 507, in <module>
    sys.exit(main())
             ^^^^^^
  File "/home/mwall/Sourcecode/geonodectl/./geonodectl", line 503, in main
    g_obj_func(**args.__dict__)
  File "/home/mwall/Sourcecode/geonodectl/src/people.py", line 43, in cmd_describe
    if kwargs["user_resources"] is True:
       ~~~~~~^^^^^^^^^^^^^^^^^^
KeyError: 'user_resources'

patch --set argument not required anymore

With #7 having the --set argument is not required anymore for patch argument

described install process not working properly

setup.py requires changes so that geonodectl is going to be installed into PATH with installing via pip install -e git+https://github.com/GeoNodeUserGroup-DE/geonodectl.git

bug when deleting an object

❯ ./geonodectl ds delete 2
Traceback (most recent call last):
  File "/home/mwall/Sourcecode/geonodectl/./geonodectl", line 507, in <module>
    sys.exit(main())
  File "/home/mwall/Sourcecode/geonodectl/./geonodectl", line 503, in main
    g_obj_func(**args.__dict__)
  File "/home/mwall/Sourcecode/geonodectl/src/geonodeobject.py", line 44, in cmd_delete
    self.delete(pk=pk, **kwargs)
  File "/home/mwall/Sourcecode/geonodectl/src/geonodeobject.py", line 49, in delete
    pk = kwargs["pk"]
KeyError: 'pk'

json output is not parsable on cmdline via jq

I would expect to parse the output of a geonodectl ds describe 1 via jq. Sadly pprint is currently used which uses single quotes and not double quotes as expected in json.

allow possibility to write configfile

make config load from .conf file available so its optional to set enviornment vars

given a title to a dataset won't set the title at geonode

When uploading a file using for example:
./geonodectl dataset upload -f myshape.shp -t test_file --json

this will not set title inside of the metadata of the dataset. Either there is any parameter which can be used to set the title or this parameter can be removed.

install script is not working as intended

When running the install script python setup.py install or via git like: pip install git+https://github.com/GeoNodeUserGroup-DE/geonodectl.git installation ends up in a with project name src and not geonodectl

[UPLOAD] support new importer design in verison 4.1

In geonode v4.1 the new importer is not responding the result of the upload directly. It passes an execution_id. The result of the import can be gathered at a new api endpoint called: /api/v2/resource-service/execution-status/<str: execution ID>

I think its a good idea to add a new parameter to geonoctl called resource-service. This sould implement list and describe functionallity.

When using geonodectl upload [dataset, document] ... it should immediately reply the response from /api/v2/resource-service/execution-status/execution_id