ifgi / optimetaportal Goto Github PK
View Code? Open in Web Editor NEWGeospatial discovery of research articles based on open metadata
Home Page: https://projects.tib.eu/optimeta
License: GNU General Public License v3.0
Geospatial discovery of research articles based on open metadata
Home Page: https://projects.tib.eu/optimeta
License: GNU General Public License v3.0
https://docs.djangoproject.com/en/4.0/ref/contrib/gis/feeds/
The first admin user is created as part of the installation (cf. #46). On the page:
Important: This should reuse as much as possible of existing Django features.
Users should be able to be notified about new papers when they are added to the portal. For this, users need to authenticate. We do not want to store passwords.
See https://github.com/betagouv/django-magicauth and evaluate it, including some alternatives.
This is part of #10
See https://docs.djangoproject.com/en/4.1/topics/testing/overview/#the-test-database
Current error:
======================================================================
ERROR: test_api_root (tests.test_publications_api.SimpleTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/daniel/.virtualenvs/optimetaPortal/lib/python3.8/site-packages/django/db/backends/base/base.py", line 244, in ensure_connection
self.connect()
File "/home/daniel/.virtualenvs/optimetaPortal/lib/python3.8/site-packages/django/utils/asyncio.py", line 26, in inner
return func(*args, **kwargs)
File "/home/daniel/.virtualenvs/optimetaPortal/lib/python3.8/site-packages/django/db/backends/base/base.py", line 225, in connect
self.connection = self.get_new_connection(conn_params)
File "/home/daniel/.virtualenvs/optimetaPortal/lib/python3.8/site-packages/django/utils/asyncio.py", line 26, in inner
return func(*args, **kwargs)
File "/home/daniel/.virtualenvs/optimetaPortal/lib/python3.8/site-packages/django/db/backends/postgresql/base.py", line 203, in get_new_connection
connection = Database.connect(**conn_params)
File "/home/daniel/.virtualenvs/optimetaPortal/lib/python3.8/site-packages/psycopg2/__init__.py", line 122, in connect
conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
psycopg2.OperationalError: connection to server at "localhost" (127.0.0.1), port 5432 failed: Connection refused
Is the server running on that host and accepting TCP/IP connections?
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/daniel/git/OPTIMETA/optimetaPortal/tests/test_publications_api.py", line 12, in test_api_root
response = self.client.get('/publications/api/publications/')
File "/home/daniel/.virtualenvs/optimetaPortal/lib/python3.8/site-packages/django/test/client.py", line 836, in get
response = super().get(path, data=data, secure=secure, **extra)
File "/home/daniel/.virtualenvs/optimetaPortal/lib/python3.8/site-packages/django/test/client.py", line 424, in get
return self.generic(
File "/home/daniel/.virtualenvs/optimetaPortal/lib/python3.8/site-packages/django/test/client.py", line 541, in generic
return self.request(**r)
File "/home/daniel/.virtualenvs/optimetaPortal/lib/python3.8/site-packages/django/test/client.py", line 810, in request
self.check_exception(response)
File "/home/daniel/.virtualenvs/optimetaPortal/lib/python3.8/site-packages/django/test/client.py", line 663, in check_exception
raise exc_value
File "/home/daniel/.virtualenvs/optimetaPortal/lib/python3.8/site-packages/django/core/handlers/exception.py", line 55, in inner
response = get_response(request)
File "/home/daniel/.virtualenvs/optimetaPortal/lib/python3.8/site-packages/django/core/handlers/base.py", line 197, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/home/daniel/.virtualenvs/optimetaPortal/lib/python3.8/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view
return view_func(*args, **kwargs)
File "/home/daniel/.virtualenvs/optimetaPortal/lib/python3.8/site-packages/rest_framework/viewsets.py", line 125, in view
return self.dispatch(request, *args, **kwargs)
File "/home/daniel/.virtualenvs/optimetaPortal/lib/python3.8/site-packages/rest_framework/views.py", line 509, in dispatch
response = self.handle_exception(exc)
File "/home/daniel/.virtualenvs/optimetaPortal/lib/python3.8/site-packages/rest_framework/views.py", line 469, in handle_exception
self.raise_uncaught_exception(exc)
File "/home/daniel/.virtualenvs/optimetaPortal/lib/python3.8/site-packages/rest_framework/views.py", line 480, in raise_uncaught_exception
raise exc
File "/home/daniel/.virtualenvs/optimetaPortal/lib/python3.8/site-packages/rest_framework/views.py", line 506, in dispatch
response = handler(request, *args, **kwargs)
File "/home/daniel/.virtualenvs/optimetaPortal/lib/python3.8/site-packages/rest_framework/mixins.py", line 46, in list
return Response(serializer.data)
File "/home/daniel/.virtualenvs/optimetaPortal/lib/python3.8/site-packages/rest_framework_gis/serializers.py", line 23, in data
return super(ListSerializer, self).data
File "/home/daniel/.virtualenvs/optimetaPortal/lib/python3.8/site-packages/rest_framework/serializers.py", line 253, in data
self._data = self.to_representation(self.instance)
File "/home/daniel/.virtualenvs/optimetaPortal/lib/python3.8/site-packages/rest_framework_gis/serializers.py", line 32, in to_representation
("features", super().to_representation(data)),
File "/home/daniel/.virtualenvs/optimetaPortal/lib/python3.8/site-packages/rest_framework/serializers.py", line 686, in to_representation
return [
File "/home/daniel/.virtualenvs/optimetaPortal/lib/python3.8/site-packages/django/db/models/query.py", line 320, in __iter__
self._fetch_all()
File "/home/daniel/.virtualenvs/optimetaPortal/lib/python3.8/site-packages/django/db/models/query.py", line 1507, in _fetch_all
self._result_cache = list(self._iterable_class(self))
File "/home/daniel/.virtualenvs/optimetaPortal/lib/python3.8/site-packages/django/db/models/query.py", line 57, in __iter__
results = compiler.execute_sql(
File "/home/daniel/.virtualenvs/optimetaPortal/lib/python3.8/site-packages/django/db/models/sql/compiler.py", line 1359, in execute_sql
cursor = self.connection.cursor()
File "/home/daniel/.virtualenvs/optimetaPortal/lib/python3.8/site-packages/django/utils/asyncio.py", line 26, in inner
return func(*args, **kwargs)
File "/home/daniel/.virtualenvs/optimetaPortal/lib/python3.8/site-packages/django/db/backends/base/base.py", line 284, in cursor
return self._cursor()
File "/home/daniel/.virtualenvs/optimetaPortal/lib/python3.8/site-packages/django/db/backends/base/base.py", line 260, in _cursor
self.ensure_connection()
File "/home/daniel/.virtualenvs/optimetaPortal/lib/python3.8/site-packages/django/utils/asyncio.py", line 26, in inner
return func(*args, **kwargs)
File "/home/daniel/.virtualenvs/optimetaPortal/lib/python3.8/site-packages/django/db/backends/base/base.py", line 244, in ensure_connection
self.connect()
File "/home/daniel/.virtualenvs/optimetaPortal/lib/python3.8/site-packages/django/db/utils.py", line 91, in __exit__
raise dj_exc_value.with_traceback(traceback) from exc_value
File "/home/daniel/.virtualenvs/optimetaPortal/lib/python3.8/site-packages/django/db/backends/base/base.py", line 244, in ensure_connection
self.connect()
File "/home/daniel/.virtualenvs/optimetaPortal/lib/python3.8/site-packages/django/utils/asyncio.py", line 26, in inner
return func(*args, **kwargs)
File "/home/daniel/.virtualenvs/optimetaPortal/lib/python3.8/site-packages/django/db/backends/base/base.py", line 225, in connect
self.connection = self.get_new_connection(conn_params)
File "/home/daniel/.virtualenvs/optimetaPortal/lib/python3.8/site-packages/django/utils/asyncio.py", line 26, in inner
return func(*args, **kwargs)
File "/home/daniel/.virtualenvs/optimetaPortal/lib/python3.8/site-packages/django/db/backends/postgresql/base.py", line 203, in get_new_connection
connection = Database.connect(**conn_params)
File "/home/daniel/.virtualenvs/optimetaPortal/lib/python3.8/site-packages/psycopg2/__init__.py", line 122, in connect
conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
django.db.utils.OperationalError: connection to server at "localhost" (127.0.0.1), port 5432 failed: Connection refused
Is the server running on that host and accepting TCP/IP connections?
The popup should contain the paper's
Always check if the field exists before adding it to the popup.
https://www.askpython.com/django/django-app-structure-project-structure
https://djangobook.com/django-tutorials/mastering-django-structure/
Django app
Dependency management file (requirements.txt
?)
Add documentation for running the app locally and running the app in Docker
The free option now has 3 GB of free storage, e.g., for the Postgres database, see https://fly.io/blog/free-postgres/
The free CPU and RAM options should be enough to run the app: https://fly.io/blog/free-postgres/
We might be able to deploy with our existing Docker container: https://fly.io/docs/languages-and-frameworks/dockerfile/
Or just run the Django app (though that will be possibly difficult when we need GIS libs): https://fly.io/docs/django/getting-started/
More on PostGIS:
HTTPS:
search for papers with data only from specific months or seasons
Maybe the map should be full screen all the time? Then the above need to be small overlays to the map.
In
we do not need to show "Username" placeholder (because there is no user logged in yet), and "Subscription", "Logout" and "Settings" also do not make sense yet.
Instead, the menu should just show "Login" and the short sentence "If you don't have an account yet, one will be created when you first log in."
e0d1d65
(#3)<automatic unique ID>
, https://service.tib.eu/optimeta/index.php/optimeta/oai/
, 7
job
to connect them with the harvestingReview https://djangostars.com/blog/configuring-django-settings-best-practices/ and decide what probably works well for us (having only one app)
Make sure the approach works well with Docker, for that revisit https://www.digitalocean.com/community/tutorials/how-to-build-a-django-and-gunicorn-application-with-docker#step-3-making-django-configurable-by-environment-variables
When a publication is selected in the timeline or on the map, then the other part is marked.
When a geometry is hovered on the map, then the items in the timeline are highlighted.
Part of #37
When clicking the user icon in the top right of the page (e.g., on OJS)
a pop-up (similar to the one on OJS)
offers options to login, if not logged in, or a menu of options (settings, subscriptions, logout). If logged in, the popup should also show the current username.
There should also be pop up messages for when a login link has been sent, and when a new account has been created (after the first login)
Users can draw their own geometries on a map after logging in.
Use http://draw.io/ and see the issues linked below for details of each page.
Possible starting point: https://dev.to/errietta/documenting-a-django-api-with-openapi-and-dataclasses-1p6h
OpenAPI docs should be automatically generated from the code and served at /docs
.
Better starting point: https://github.com/tfranzel/drf-spectacular and also use https://github.com/tfranzel/drf-spectacular-sidecar so that we don't rely on CDNs.
Interactive user interface for adding, editing, and removing OJS instances that are harvested.
Needs "Admin" user level.
Depends on #8
Try out storing data in S2 (https://github.com/google/s2geometry#python and https://github.com/google/s2geometry/tree/master/src/python, or https://pypi.org/project/s2-py / https://github.com/mira/s2-py or others: https://pypi.org/project/s2sphere/, https://github.com/qedus/sphere)
Classifying this as "small" just to make sure we're not wasting time.
Provide 7 feeds for continents and the 5 named oceans.
See #6
For development purposes, add a docker-compose
configuration that runs two containers:
python:3.x
postgis
nginx
#158F9B
#9B2115
#3C159B
#9B7115
#158F9B
#159B71
#159B8C
#158F9B
#15749B
#15599B
https://seochecker.it/color-palette-generator
For a fixed lists of users in the database (table with emails) send out an email at a fixed interval (end of the month) with all new manuscripts of that month.
See also #11 about scheduling - the same scheduling method should be used
Start with some reading on GitHub actions. Then implement first test cases:
<meta name="DC.Coverage" xml:lang="en" content="Earth, Europe, Republic of France, Pays de la Loire"/>
<meta name="DC.SpatialCoverage" scheme="GeoJSON" content="{"type":"FeatureCollection","features":[{"type":"Feature","properties":{"provenance":{"description":"geometric shape created by user (drawing)","id":11}},"geometry":{"type":"LineString","coordinates":[[0.19995063543319705,47.83528342275264],[-0.6350103020668031,47.80577611936812]]}}],"administrativeUnits":[{"name":"Earth","geonameId":6295630,"bbox":"not available","administrativeUnitSuborder":["Earth"],"provenance":{"description":"administrative unit created by user (acceppting the suggestion of the geonames API , which was created on basis of a geometric shape input)","id":23}},{"name":"Europe","geonameId":6255148,"bbox":{"east":41.73303985595703,"south":27.6377894797159,"north":80.76416015625,"west":-24.532675386662543},"administrativeUnitSuborder":["Earth","Europe"],"provenance":{"description":"administrative unit created by user (acceppting the suggestion of the geonames API , which was created on basis of a geometric shape input)","id":23}},{"name":"Republic of France","geonameId":3017382,"bbox":{"east":9.56009360694225,"south":41.3335556861592,"north":51.0889894407743,"west":-5.14127657354623},"administrativeUnitSuborder":["Earth","Europe","Republic of France"],"provenance":{"description":"administrative unit created by user (acceppting the suggestion of the geonames API , which was created on basis of a geometric shape input)","id":23}},{"name":"Pays de la Loire","geonameId":2988289,"bbox":{"east":0.916650657911376,"south":46.2666616230696,"north":48.5679940644253,"west":-2.62573947290169},"administrativeUnitSuborder":["Earth","Europe","Republic of France","Pays de la Loire"],"provenance":{"description":"administrative unit created by user (acceppting the suggestion of the geonames API , which was created on basis of a geometric shape input)","id":23}}],"temporalProperties":{"unixDateRange":"[1654041600000,1654214399000]","provenance":{"description":"temporal properties created by user","id":31}}}" />
<meta name="geo.placename" content="Pays de la Loire" />
<meta name="DC.box" content="name=Pays de la Loire; northlimit=48.567994064425; southlimit=46.26666162307; westlimit=-2.6257394729017; eastlimit=0.91665065791138; projection=EPSG3857" />
<meta name="ISO 19139" content="<gmd:EX_GeographicBoundingBox><gmd:westBoundLongitude><gco:Decimal>-2.6257394729017</gco:Decimal></gmd:westBoundLongitude><gmd:eastBoundLongitude><gco:Decimal>0.91665065791138</gco:Decimal></gmd:eastBoundLongitude><gmd:southBoundLatitude><gco:Decimal>46.26666162307</gco:Decimal></gmd:southBoundLatitude><gmd:northBoundLatitude><gco:Decimal>48.567994064425</gco:Decimal></gmd:northBoundLatitude></gmd:EX_GeographicBoundingBox>" />
Use pytest
for tests, see if https://pypi.org/project/pytest-docker-compose/ is useful.
We should use the Django testing framework, see https://docs.djangoproject.com/en/4.1/topics/testing/
(we don't want to forget the test cases; ideally we can just check these off when the features are implemented)
api/v1/feed/de.xml
api/v1/feed/aut.xml
The templates confirmation_login.html
and login_response.html
are a direct copy of main.html
with the added.
Either the templates need to import common parts, or the notifications should be shown with JavaScript. It is unacceptable that so much template HTML code is duplicated.
geography
type is used, see https://docs.djangoproject.com/en/4.0/ref/contrib/gis/model-api/#geography-type/api/v1/article/1
returns the geometry in GeoJSON and some fake metadataSee #14 for minimal feature.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.