eoepca/open-science-catalog-frontend

Repository to hold the OSC frontend app

open-science-catalog-frontend

The Open Science Catalog (opensciencedata.esa.int)

The Open Science Catalog is one of the elements contributing to an Open Science framework and infrastructure, with the scope to enhance the discoverability and use of products, data and knowledge resulting from Scientific Earth Observation exploitation studies.

Adhering by design to the “FAIR” (findable, accessible, interoperable, reproducible/reusable) principles, the Open Science Catalogue aims to support better knowledge discovery and innovation, and facilitate data and knowledge integration and reuse by the scientific community.

This repository

This repository holds the static frontend app powered by Nuxt. On every push to the dev branch, a GitHub Action triggers a pre-rendering of the application and publishes it in the gh-pages branch, thus deploying a development version on the domain eoepca.github.io/open-science-catalog-frontend. The production version is built on each push to the main branch as a docker image and is available at opensciencedata.esa.int.

Every PR triggers GitHub Actions for linting and testing, and additionally checks for conventional style commits.

Used EOEPCA endpoints

This static frontend access two endpoints provided by EOEPCA:

• OSC Metadata ("static catalog") - see open-science-catalog-metadata repository
• OSC pycsw access ("dynamic catalog")

These endpoints are configured in the axios plugin file (using nuxt/axios) and then injected into the app, so they can be used like so:

this.$staticCatalog.$get("/metrics");
this.$dynamicCatalog.$get("/collections");

Note that by default, all calls to the static endpoint add the .json file ending and all calls to the dynamic endpoint add the f=json query parameter to the request url for convenience.

These endpoints are primarily used within the application central store: staticCatalog.js and dynamicCatalog.js.

Development Setup

Install dependencies

$ npm install

Build and configure Open Science Catalog STAC Browser locally

Find STAC Browser runtime configuration in static/stac-browser/config.js

$ npm run build:browser

Serve with hot reload at localhost:3000

$ npm run dev

Build for production and launch server

$ npm run build
$ npm run start

Generate static project

$ npm run generate

Lint

$ npm run lint #linting only
$ npm run lint:fix #lint and fix

Test

$ npm run test

For detailed explanation on how things work, check out the documentation.

Special Directories

You can create the following extra directories, some of which have special behaviors. Only pages is required; you can delete them if you don't want to use their functionality.

assets

The assets directory contains your uncompiled assets such as Stylus or Sass files, images, or fonts.

More information about the usage of this directory in the documentation.

components

The components directory contains your Vue.js components. Components make up the different parts of your page and can be reused and imported into your pages, layouts and even other components.

More information about the usage of this directory in the documentation.

layouts

Layouts are a great help when you want to change the look and feel of your Nuxt app, whether you want to include a sidebar or have distinct layouts for mobile and desktop.

More information about the usage of this directory in the documentation.

pages

This directory contains your application views and routes. Nuxt will read all the *.vue files inside this directory and setup Vue Router automatically.

More information about the usage of this directory in the documentation.

plugins

The plugins directory contains JavaScript plugins that you want to run before instantiating the root Vue.js Application. This is the place to add Vue plugins and to inject functions or constants. Every time you need to use Vue.use(), you should create a file in plugins/ and add its path to plugins in nuxt.config.js.

More information about the usage of this directory in the documentation.

static

This directory contains your static files. Each file inside this directory is mapped to /.

Example: /static/robots.txt is mapped as /robots.txt.

More information about the usage of this directory in the documentation.

store

This directory contains your Vuex store files. Creating a file in this directory automatically activates Vuex.

More information about the usage of this directory in the documentation.

open-science-catalog-frontend's Issues

Update Readme with endpoint documentation

The Readme should include information about the endpoints used in data mangement (see #66), especially highlighting the use of EOEPCA services.

Store metrics etc. in store to not fetch unnecessarily

Same could go for individual products

Rename records to products to align with catalog(s) and API

Since in the catalog, a "record" can actually be a project or a product, we decided today to use "products" instead of "records" also in the frontend

Get records from live API

Static endpoint

https://constantinius.github.io/open-science-catalog-builder/ (see "products")

• Records should be displayed in a grid with title and theme(s) each
• Individual Record should display metainfo: Name, theme, variable, start&end date, release date, project, satellite mission, description, links to products and documentation
• Individual records should include a map view showing the spatial extent
• Metrics view: records should be fetched dynamically when a variable is expanded and display the available years and a link to the record
• Metrics view: record coverage should be shown on a map for individual records, and records summarized by variable
• Metrics view: in the record coverage popup, when showing multiple records, there should be a way to highlight the records in the map (currently implemented on list hover)
• Statistics: Number of records should be shown in the chart, chart title, and variable list & chart
• Statistics: Satellite missions and number of records should be shown in radial chart (like variable list & chart) - not yet implemented in backend

Dynamic endpoint

https://resource-catalogue.osc.develop.eoepca.org/collections/metadata:main/items?type=dataset&q=Water+level&f=json

• Record grid should be sortable by name (ascending and descending)
• Record grid should be filterable by satellite mission (not implemented yet in backend)
• Record grid has pagination (max 10 for now, more in future?)
• Search: It should be possible to search for a keyword and get a grid of matched records, including display as described above

Typescript config

So far our Vue components, Vuex store modules, Mixins are written in plain JS, contradicting our attempt to use Typescript.
I will keep track of the progress of the "conversion" in this issue

Goals:

• use Typescript-style Vue components with Options API
• write Vuex store modules in Typescript
• write Mixins in Typescript

Hide "empty" projects by default in ItemGrid

Similar to metrics view, projects with 0 products should be hidden by default in the ItemGrid / Search results.

But, there should be a checkbox or something similar to allow the user to display empty projects, maybe this can be a global checkbox with the state stored and also accessible in the metrics view.

Basic look and feel

• The look and feel
• The names of themes
• The names of links that need to be provided although may do nothing at this stage. E.g. Home, Contribute, Metrics, Search….. etc
• If place holders for the pages “metric” and “search” can be provided with navigation to and from that would be helpful.
• like wise if the contribute options of New Record and New Project can be added that would be helpful.

Introduce configurability for dev/staging/production versions

• endpoint configuration via env
• feature flags via env (e.g. auth/login functionality)
• github action needs to create different images for certain releases that then can be deployed on dev/staging/production

Version mismatch between docker image / footer version

If e.g. the image version eoepca/open-science-catalog-frontend:v2.0.9-dev is deployed, the footer shows version 2.0.8:

Maybe this is due to the order of things (bumping, building, etc.)

See also

open-science-catalog-frontend/nuxt.config.js

Line 5 in 612a8d4

const version = JSON.parse(packageJson).version || 0;

Consider migration to Vue3

View on github URLs are wrong

When trying to click the "view file on github" button on any catalog item, the user is redirected to a 404 page of the github repository. The issue is that there is a data/ prefix in the button URL, which should be removed.

Have a central location that does data fetching/mangement

This could be seen as an extension of #47.
The store should be used to do all data fetching/handling in a central location, instead of inside components. This makes it easier to maintain and document the fetching logic.

Initial setup of user authentication/authorization feature

• different viewing capabilities depending on public (not logged in) and authenticated: flag to show/hide components/pages
• login/signup page (need to wait for decision on which system we are going to use in the end)

Re-evaluate usage of Nuxt

Since the introduction of STAC Browser for navigating through the catalog, it might no longer make sense to use Nuxt as framework for OSC. Lots of things could be simplified by converting to a "simple" Vue app.
This would ideally be done in conjunction with #366.

Missing catalog endpoint data

Some things are still missing from the dynamic endpoint:

• The query function is inconsistent, let's say we initially have the starting point https://resource-catalogue.osc.develop.eoepca.org/collections/metadata:main/items?type=dataset&q=Aerosols&sortby=title&f=json, if one wants to also add the "dust" as a search keyword it works pretty well https://resource-catalogue.osc.develop.eoepca.org/collections/metadata:main/items?type=dataset&q=Aerosols&q=dust&sortby=title&f=json but if you want to search for "climatology" (which is a pretty common word in the initial response titles as well as descriptions) we get an empty response https://resource-catalogue.osc.develop.eoepca.org/collections/metadata:main/items?type=dataset&q=Aerosols&q=climatology&sortby=title&f=json, not only that but the empty response appears as soon as I try to query "cli"
• pagination: get more than 10 items at once

Display record data access more prominently

Right now, we only have a collection of links, but as soon as we know which link is data access, which one is documentation etc. it would be good to render them more prominently on the record details page.

Add an "intro" section to the landing page

This intro should include general information about the OSC, regarding e.g. the quality of products, that they are not offical ESA products etc.

Display WMS products on map

Some items will have a WMS link in the metadata; these items should show the WMS as a layer additionally to the data footprint.

Pre-populate process selection in processing UI after selecting a process in STAC Browser

Prerequesites:

• STAC Browser fork at v3.1.0
• Process definition in the catalog as it currently is on testing: https://github.com/EOEPCA/open-science-catalog-metadata-testing/blob/main/processes/catalog.json

Currently, the processing UI (processing-federation branch) lets the user select a process from a dropdown list in the first step of the form. We need to extend the STAC Browser link action view to include a button which redirects the user to the processing UI, with the process (Step 1) already pre-selected.
This makes use of the linkActions config where we could add a new button to the processing links, something like

import LinkActionPlugin from "../LinkActionPlugin";
​
export default class Processing extends LinkActionPlugin {
​
  get show() {
    return (this.link.rel === 'process' && this.link.type === 'application/cwl');
  }
​
  //get uri() {
  //  return 'http://localhost:300/processing?url=' + encodeURIComponent(this.link.href);
  //}
​
  get onClick() {
     return () => {
         window.parent.postMessage({
           navigate: /new-process?process=<process id>
        }, '*');
     };
  }
​
  get text() {
    return 'Open in Processing UI';
  }
​
}

(see https://github.com/EOEPCA/open-science-catalog-stac-browser/blob/6a6f97525bc91e872031cd9b0deb0dd33d6a86ec/src/StacBrowser.vue#L70-L75 for reference).

Migrate STAC Browser from bundle to external library that gets pulled on image build time

Currently, our forked version of STAC Browser is included in the app as a bundle, which is not ideal (loads of changes each commit etc.).

The ideal solution would be to set the Docker image up in a way to fetch the latest version of the fork and include the bundle only at run time, so when the image is started.

With the new runtime config options available at STAC Browser this would still allow to have different configurations for testing, staging, and live environments.

One challenge will definitely be the local dev setup since currently we are not developing inside a container. We could either create a second, alternative way on how to launch STAC Browser locally, or introduce a container workspace (VS Code dev container) that does the same things a deploy environment would do.

Modularize page layout

Currently, we have similar but slightly different layouts for theme/projects, theme/variables, projects and variables pages. Since they all have a header and an item grid underneath it would be good to always reuse the same layout component.

Also consider re-using the top (header/hero) part in this layout.

Update ESA logo

There has been an update to the ESA logo recently (subtle changes), and not sure if we're already using the latest one - it will probably be best to check https://esa.int again and get the latest logo!

Get variables, projects and themes from live API

The first data we will try to connect from the real API will be:

• variables

• projects

• themes
  Each of these should be navigable in the frontend. Examples:

• On the landing page, see 6 themes loaded from the api

• Click on a theme, see theme meta info (background image, description, link), and all projects and variables for that theme

• For projects and variables pages, it should be possible to sort them alphabetically (ascending & descending) or consortium, start date etc.

• For projects and variables pages, it should be possible to search items (fulltext)

• When clicking on a project, see all metainfo for that project (start date, end date, description, etc.) (no records yet)

• When clicking on a variable, see all metainfo for that project (description, etc.) (no records yet)

• When "drilling down", a navigation should be visible on top (e.g. ESA EO CAtalogue / Atmosphere / Aerosols / )

• In Metrics View, the themes should be taken from API as well

• In Metrics View, the Variables (Above-ground biomass, Aerosols etc.) should link to the corresponding variable page

Improve metrics view performance

STAC Browser: Render OSC properties as clickable links

Custom field rendering of osc:themes, osc:eo-missions, osc:variables and more as clickable link. Probably can follow a schema and use the root catalog config to build the links.

https://github.com/EOEPCA/open-science-catalog-stac-browser

Display contribution status

A "status" to the OSC contributions backend has been added, so each PR has a status of "Accepted" or "Pending" or "Rejected", so the status page needs to be expanded to not only show "pending" contributions, but also the other types:

• Rename "Pending Contributions" to "Contribution status"
• Create three subheaders: "Pending", "Accepted", "Rejected"
• Under each subheader is the list like it is now but with items filtered by status
• Pending items keep the orange "waiting" symbol, Accepted have a green symbol with some sorts of check symbol (for example mdi-progress-check), rejected in red with some sort of rejection symbol (e.g. mdi-progress-close)
• Somewhere we need a small disclaimer (either at the top under the title, or at the bottom), with an info icon (e.g. mdi-information-outline), saying "Note: only your most recent contributions are shown in this list. For a full list of contributions, please refer to the Open Science Catalog Metadata GitHub Repository."

Add "zooming" functionality to metrics view timeline

Currently, we always display each year and the user has to scroll horizontally to view the older entries.

It could be useful to add a functionality to either "zoom" the timeline, or to display only ever 5th, 10th year and so on. It is expected that someway the user can also see all the years at once.

Metadata editing forms

• "new product/project" page with form
• "edit product/project" page with form - ideally using the same components as the "new product/project" page
• loading of existing metadata (from product/project) into editing form
• make this only accessible for authorized users (see #102 )

Replace prototype API documentation with pycsw starting page

Currenty, we still link to the (deprecated) prototype swagger API docs. This should instead link to the various endpoints (STAC, OpenSearch etc.), which are described in the pyscw landing page.

Global search/filter combobox

This will be the main feature of the Search page, but should also be usable in other places, e.g. the header search box.
In the end, it will be probably good to unify the input fields to not have separate filter/search boxes, but rather one search box per page max. Some ideas:

• Provide a prominent search box in the Search page
• Add the same functionality to the header search box
• Hide the header search box on the Search page itself?
• In Projects/Variables/Records views, the Search box could come with a pre-selection, e.g. current variable (but the user could still remove it and search all variables)
• The search combobox should give hints on what the user can search, so display themes, variables, projects with autocomplete
• Ideally, the user can combine multiple search strings, maybe even with operands (not sure if backend supports it)

Search box scenarios

• As a SCIENTIST, I am searching the Catalogue for products to use in my research. I start by typing free text (e.g. Antarctica), then I refine the search by variable (e.g. Sea Ice).
• As a SCIENTIST, I want to retrieve some of my results. I search by product name, or DOI.
• As a PROJECT MANAGER, I am searching the Catalogue to retrieve results of my project. I start the search from the project name, then I refine with variables or free text.
• As an ESA USER, I am using the Catalogue to do research for an ITT that I am preparing. I filter by theme or geographical location, and refine the search with variables.
• As an ESA USER, I want to see what results have been produced from Sentinel-1 data. I start the search with the EO mission, then I refine.

Get PR link from pending contributions

Use dynamic endpoint for metrics search

This is connected to search in general, but the search box in metrics view (and probably all other search boxes as well) should use the capabilities of the dynamic catalog to find e.g. records that have not been fetched yet.

Allow HTML rendering in themes/projects/variables/records descriptions

It would be good to provide a demo description that can show how HTML rendering would look like, e.g. including bullet point lists, links...

Set up Metrics View

Features:

• Horizontal (time) and vertical scrolling (variables)
• Each variable is expandable and shows the individual records
• Each record has one or multiple coverage times, which aggregate on variable level
• The records can be filtered by theme (All, Atmosphere, Cryosphere, Land etc.)
• A filter input box allows free text input (does nothing for now)
• Each record/variable has a spatial coverage button (far right) that opens a dialog (empty for now)
• Option to open/close all variables (top left)
• Stats button (top left) opens a dialog that shows a bar chart and a doughnut chart (random placeholder data for now) using chart.js

additional fixes & improvements

• add tooltip to all buttons
• rethink expand all end contract all functionality...
• handle "no products" case (lower part of table)
• customize scrollbars
• fix overflow of individual records
• fix z-index of sub-cells
• improve positioning of table on page and add a rounded corner
• check mobile view
• expandable row icons are wrong way round

See reference: https://opensciencedata.esa.int/metrics

Using some placeholder data, this is how it should look like:

First Release Checklist

stability improvements

• on the search page, the show/hide empty projects button does nothing
• navigating to themes -> atmosphere and clicking submit on the pre-filled search box, the results are products
• metrics - filtering by static and dynamic catalog returns different results
• bbox search also adds bbox as query to call and breaking the query
• metrics search menu is overlayed by table header
• metrics - expanding certain items breaks the page (e.g. expand "biomass")
• type "projects" actually returns products in search
• page breaks if one item from static matching is 404
• chips in search box overlay box border
• some pages are empty (e.g. https://eoepca.github.io/open-science-catalog-frontend/projects/4d-earth-swarm), link on project breadcrumb leads to empty project
• search: sometimes adding more filters returns more results this is the functionality of the q keyword
• on certain screen widths, the dates in the project itemgrid have a line break
• map sometimes does not fully load
• pagination in itemgrid shows more pages than available
• hover over time range in metrics could show the actual values start/end for confirmation makes table laggy
• individual coverages of a product could be better differentiable on a map if there are more of them (toggle off/on pointer or highlight on click of poi icon?), because if they are "almost similar" the center map move to action is hard to distinguish
• on individual variables LINK button could be disabled if does not lead anywhere

todos before first release

• deployment setup (Dockerfile, env configuration etc.)
• check all projects/products IDs working
• backlink from products to projects
• check on all screen sizes
• check on all major browsers
• host google fonts /material design icons locally
• create flag to disable auth/login system (ideally do not even load auth if flag disables it?)
• terms & legal skipping for v1.0 as no contribution feature yet
• cookie notice? skipping for v1.0 as no cookies are set yet
• create v1.0 tag and release note

Remove empty variables from metrics view

As discussed in today's meeting, the metrics view should only display variables that have at least one record.

The entire row should be hidden!

Metadata editing improvements

• add way to input geographic data for products (map + text input as alternative)
• add markdown info and preview for description fields
• error handling & error message display in forms
• move OSC-backend endpoint url to centralized config
• adjust request body format for projects & products to fit STAC requirements
• review which fields can be edited (e.g. unique ID should not be editable)
• introduce way to DELETE an item
• add field to add WMS link for products
• BACKEND: pass authenticated user parameters (email/github handle) to requests for backend to use
• BACKEND: add line breaks for generated json files
• BACKEND: retrieve PR metadata (type like new item, edit item ect., plus link to PR)
• BACKEND: retrieving pending items - only retrieve items by user who does request
• BACKEND: add special label to PRs from "data owners"

Fix theme image height on narrower screens

This probably happens because the images are set to 100% width - probably it would be better to use background-images with a background-size cover to do this automatically. Is there a reason not to use a Vuetify v-img?