openwisp/django-loci

Reusable Django app for storing geographic and indoor coordinates. Maintained by the OpenWISP Project.

django-loci's Issues

[bug] websocket connect error in location change form page

1. Create a location, set the marker and then check is_mobile.
2. Save the location.
3. Open it again and look at the console. (Firefox can’t establish a connection to the server at ws://127.0.0.1:8000/ws/loci/location/undefined/.)

[admin] Validate type before calling

the issue in #50 shows that a 500 error is gotten when adding a device with location set to new and type set to none. this happens because the django-loci throws a KeyError exception when trying to access the type field. Even though a fix was attempted by overriding the methods in the #86 PR, it will be good if this problem is handle at the root.
I think it can be solved by inserting the line which throws the exception in a try-except block.

[ux] bug introduced in django 2.2 update

Fix frontend bug causing jquery to be not loaded due to Django 2.2 update

Add support for django 3.0

Add support for django 3.0, drop django version older than 2.2.

[model] Allow indoor location without indoor coords

When adding a location using the base class AbstractObjectLocationInline, we may want to flag that the object is indoor, but we may not have the floorplan image ready yet, in that case we are forced to set it to outdoor in order to save, but it's incorrect since it's not an outdoor location.

We could ask for confirmation to the user that they want to save the object even if they have not specified the indoor coordinates and if they confirm we can save the object anyway.

Changes to the JS and admin form will be necessary.

[docs] Outdated ASGI_APPLICATION directive in README

ASGI_APPLICATION = "django_loci.channels.routing.channel_routing"
CHANNEL_LAYERS = {
    "default": {
        "BACKEND": "channels.layers.InMemoryChannelLayer",
    },
}

NO: django_loci.channels.routing.channel_routing
YES: django_loci.channels.asgi.channel_routing

[dependencies] Update django-leaflet

Upgrade django-leaflet to the latest available version, ensuring that the outdoor and indoor map features still work as expected.

Reserved for GCI.

Remove custom leaflet template

Since makinacorpus/django-leaflet#204 is merged, when a new version of django-leaflet will be released we can remove our custom template.

[dependencies] Add support for django 2.2

Add support for django 2.2.

[bug] ignores `is_mobile` False from device

In admin interface:

1. Create a device with a location, set the marker, check is_mobile & save.
2. Open it again, uncheck is_mobile, save
3. Open the device again and notice is_mobile is checked.

[ci] Add CI check for migrations

Add CI check for migrations with this solution for ensuring the migrations created have have been renamed to something more descriptive like this GCI task says.

[qa] Add commit message style check

[bug] NOT NULL constraint failed: django_loci_floorplan.floor

IntegrityError at /admin/testdeviceapp/device/add/
NOT NULL constraint failed: django_loci_floorplan.floor

Steps to reproduce:

1. Create a device with new indoor environment, flag it as mobile
2. In the "Indoor coordinates" section, do not fill a value for floor.
3. Complete the rest of the form correctly.
4. Submit

[ux] When WebUI is too narrow geography map disappears

When WebUI is too narrow geography map in add/change view of location disappears

[ux] Location selection hidden when error occurs while isMobile toggled on

To produce this issue, follow this step:

1. Add device, leave some fields empty. (MAC, Name)
2. Set Location selection to New
3. Set Type to Outdoor environment
4. Toggle isMobile to true.
5. Save device

On the form error page, we see "Location data not received yet" and the isMobile check box is hidden.

Expected output: Location selection & isMobile field should be showed.
Additional note: Leaving isMobile to false will show the field, though. It's also occurs in Indoor environment, existing location selection, and even in the change device form, as long as isMobile is toggled on and there is error when saving.

Geometry shouldn't be allowed to be None if not mobile

Location.geometry can be NULL, but that should be allowed only if_mobile is True, otherwise the model should raise a ValidationError.

[change] Adjust or remove strict test

This strict test feature is backfiring quite a bit.

It's triggered every time django is initialized, also with scripts, management commands, celery, etc.

We should address this, maybe we can turn this into a management command to execute only some times.

[dependencies] Upgrade Pillow

Upgrade pillow to the latest available version, ensuring that the feature to upload floorplans still works as expected.

Reserved for GCI.

[ux] Automatically calculate geographic coordinates or address

When the position on the geographic map is updated:

1. if the address field is empty we should automatically get the best address from the geographic coordinates and update the address field with the result
2. if the address field is not empty we should ask the user if the user wants the map to be updated (for now with a simple javascript confirm dialog); If the user replies affirmatively we can do the same described in point 1

When address field is updated:

1. If the geographic coordinates on the map are not set, we can get the geographic coordinates from the address and update the map accordingly
   2 if the geographic coordinates on the map are already set, we should ask the user if the user wants the map to be updated (for now with a simple javascript confirm dialog); If the user replies affirmatively we can do the same described in point 1

For the geocoding operations, we can use geopy.

We should implement an abstraction that allows to change the provider, trying to support at least Google Maps, Nominatim and ArcGIS in the first version fo this feature.

We should handle failures and retry, retry by default 3 times with a delay of 1 second in case of failures, the geocoding library should have a built-in implementation for this, see RateLimiter.

We will likely need the addition of a few django global settings that allow to set:

• the chosen provider, defaults to ArcGIS
• the API key, defaults to None
• whether the django app may refuse to start if the testing of the geocoding operation fails, defaults to True, eg: DJANGO_LOCI_STRICT_GEOCODE_TEST
• amount of retries after failures, default to 3
• delay in seconds before trying again after a failure, default to 1

On startup, the app shall test an arbitrary geocoding operation (eg: get the coordinates of the address of the Python Foundation) and if it fails it will raise an ImproperlyConfigured exception if DJANGO_LOCI_STRICT_GEOCODE_TEST is True or it will log an error otherwise (using the python logging facility).

Since this app does not use django-rest-framework, the geocoding operations should be performed via a simple JSON view API that it is added to the admin as we do in django-netjsonconfig (see context_view as a reference), although the logic to perform the geocoding can be stored in a function or a class to improve reusability.

This feature needs to be documented in the README and good tests must be written for it.

Upgrade to latest django-channels version

Upgrade to latest django-channels version.

[docs] Add animated gif to README

Add a brief animated gif which quickly shows:

• add an item to an existing indoor location
• automatic conversion of point to address (after #46 is merged)

[bug:ux] You can select outdoor locations when creating floor plans

Whe clicking on "Select item" to select the location of the floor plan, you can select outdoor locations, and when you try to save it, an error "floorplans can only be associated to locations of type "indoor"" is shown.

[ux] Floorplan fields is shown when error occurs in new indoor environment

To produce this, follow these steps:

1. Add device, leave some fields empty (especially the required one: name, organization and mac address) to produce error when saving.
2. In Geographic Information secton, set Location selection to New and set Type to Indoor environment
3. In Indoor Coordinates section, select file to the Image field, and set the position for Indoor (you can just use dummy image and set position anywhere) to bypass the client indoor position validation.
4. Save the device.
5. In Indoor Coordinates section, you can see the Floorplan section and Floorplan fields is showed.

The expected output: Floorplan fields should be hidden (You can change the type to Outdoor and change it again to Indoor, then you see Floorplan fields is hidden again).
Additional note: The Floorplan fields should be showed only in existing location

[CI] Move to github actions

Move to github actions as we did in other repos.

[bug] Geocoding operations should not hit external services during tests but should be mocked

Try to run the tests of django-loci while offline.

Result: some tests fail because they perform real geocoding operations.

We need to fix this, we should use mocking, I have used the responses library for similar simple HTTP request mocking operations successfully in the past and we could reuse it here if applicable.

The tests that have to be fixed are:

• test_geocode
• test_reverse_geocode

Once the response from the geocoding server is mocked, the real web service won't be called anymore during tests, therefore we expect to run tests while being offline with no failures.

This issue is affecting also openwisp-controller, because openwisp-controller imports the test suite from django-loci and executes it.

[change] Rethink LOCI_FLOORPLAN_STORAGE

Maybe it's overkill to have a dedicated storage class for django-loci, we could investigate whether we can default to the global storage class.

I believe the purpose of the custom storage class was to:

• delete the floorplan files when floorplan objects are deleted
• ensure uploading a new floorplan overwrites the current floorplan successfully

Use AllowedHostsOriginValidator

I couldn't get @allowed_hosts_only to work with automated tests.

After investigating it seems to me that it's a bug in django-channels which has been fixed but not released yet, so I'm noting down to add @allowed_hosts_only back when the new version of django-channels is released.

[bug] Delete geometry in change view doesn't work

Steps to reproduce:

1. Create mobile location with geometry set to random location
2. Go to change view of location delete geometry and click save

By doing these steps and going to location change view you can see that geometry haven't been deleted

[ux] errors made while adding a new device gets hidden

When adding a new device if someone makes an error of not entering Geometry for a Outdoor environment, the user will get error Please correct the errors below. at the top of the page but the actual field to make the correction is not shown.
The user needs to change the Type field then select Outdoor environment again to see the errored field.
This may cause confusion to users.

[qa] Cleanup isort and flake8 settings

Any runflake8 or runisort script which uses any command line options are moved to the setup.cfg as it's done in django-freeradius, which is cleaner and easier to review.
This is one of the GCI task available

[ci] Add python 3.8 and python 3.9, remove python 3.6

Add python 3.8 and python 3.9, remove python 3.6 from GitHub actions testing.

Fix any bug that may appear.

[build] Test multiple django and python versions

In the travis build that runs automated tests, we should be testing the different supported django versions and at least python 3.6 and python 3.7.

PS: we can drop support for django 1.11.x.

Reserved for GCI.

[change] Use ReconnectingWebsocket to websocket connection

openwisp/ansible-openwisp2#321 sets the websocket_timeout of daphne to 1 hour (reducing from the 24 hours default value).
This means that the websocket connection will get terminated abruptly for users after the timeout. To prevent that use ReconnectingWebsocket from https://github.com/joewalnes/reconnecting-websocket which re-establishes the connection when the connection is terminated.

This library is already used in openwisp-notifications and openwisp-controller.

[ux] isMobile UI inconsistency

In add/change device, when user toggle on isMobile, then related fields (location, address, and geometry) should be hidden. But when user change the Type while leaving isMobile toggled on, it will show the related fields again, even though isMobile is still toggled on. The expected result is the related fields should be stay hidden.

[bug] Assigning an existing geographic location is broken

How to replicate:

• create a device
• create a geographic location in a separate step
• go to the device created in the first step, then go to map tab, select "Existing"
• select the geographic location just created
• hit save

Expected result:

• when the location is selected, the geo information should appear
• device is saved normally

Actual result:

• when the location is selected, the geo information does not appear
• device is not saved because a validation error

[deps] Update channels and pytest

Channels 3.0 brought some breaking changes.

This upgrade has to be coordinated with other OpenWISP modules (mainly notifications and controller) which also use django-channels.

[bug] ObjectLocation admin doesn't work in readonly mode

When a user has only view permission, the map doesn't work because the JS fails (it was not tested in this scenario).

[release] Release django-loci 0.4.0

Let's release openwisp-utils 0.7.0 first and update the requirement files to support both 0.6 and 0.7 (to avoid conflict with other modules having older versions).

Then we can release django-loci 0.4.0.

[deps] Add support for django 3.2 and django 4.0 alpha

Let's make sure this module can run with Django 3.2 and Django 4.0 alpha.

[qa] Add run-qa-checks, upgrade to openwisp-utils 0.5.1, black formatter

Add run-qa-checks, upgrade to openwisp-utils 0.5.1 and reformat code with black.

[feature] Allow to configure leaflet referrerPolicy

When using tiles which have an access token tied to a particular domain or URL pattern, the leaflet library is not sending the http referer and the tile requests return 403 Forbidden if the web server is configured with strict referer policy (eg: we use "same-site" in OpenWISP deployments by default).

In order to fix this we need to find a way to set https://leafletjs.com/reference.html#tilelayer-referrerpolicy at JS level.
It's possible we may have to patch https://github.com/makinacorpus/django-leaflet to get this working.