tnris/data.tnris.org

data.tnris.org v2.0

data.tnris.org's Issues

Create data pipeline that generates collection footprint from area_type_geometry for each collection

Order Submission Confirmation

add confirmation message when order has been completed and submitted.
Currently only shows that cart is empty after submission.

Add 'Buildings' category to collections/filtering

Hey @ctrepka . We are close to uploading the 3dbuilding datasets as assets to their corresponding lidar collections. Management would like an easy way to identify these collections within the DataHub. One suggestion made was to add another category within the API in the Lookup/Relate Associations.

We'd like this option as it would make it visible to the downloader via the collections tiles.

Figured this might be the simplest way to make these data easy to find.
Let me know your thoughts and suggestions.
CS

City of Lubbock and City of Austin

The City of Lubbock and City of Austin external entity card are not drawing. These cards were changed from the download to the external entity template before the launch. not sure but that could have something to do with it.

Contact form

Is it possible to add a space between first and last name when the question is submitted?

Acquisition Date included for Historic Imagery Requests

Please include 'Acquisition Date' for all dataset requests submitted through DataHub.
Please include the 'From Date' within LORE as the 'Acquisition Date' for each Historical Request. This was how the orders would come in prior to the switch to V2.

V1 Submitted Ticket

VS. Current V2 Tickets

Logan UI Improvements

DataHub_2.0_Notes.pdf

Lgsmiths suggestions regarding UI changes to betadata.tnris.org (dh_v2)

WMS and External Entity Catergories Disappeared

No ideas on what happened.

Add collection id as metadata field

per user request, add collection id as metadata field. this will make it easier for users to extract the collection id for use with the bulk downloader

Add basemap toggle to map pane

Quick running list of tasks to complete

Created a task list for one-off tasks required for the minimum-viable-product / beta of the app. @ yourself to assign yourself to a task and check it when complete

Running Task List

• Create collection footprints
• Look into area_type tiles directly from postGIS @JasonKleinert
• Geofilter connected to Django admin @ctrepka
• Create WMS Link / Share collection tab
• Create Custom Order Form @ctrepka
• Contact Form @ctrepka
• Cart Form / Nav Icon @ctrepka
• Cart localStorage Recoil Atoms / selectors @ctrepka
• Collection Downloads table pagination options to base_12, 24 as default @ctrepka
• Add feature to filter downloads list by selecting area on map in Collection View
• Add filter by bounds feature to catalog view, collection view
• Add controls to draw on map for catalog query, as well as for Custom Order form
• Recreate Survey Modal functionality
• Turn off coverage layer when in Download Tab of Collection View
• Zoom extent conditionally to collection, catalog, or boundingBox of geosearch
• Popup with dl links in map pane for collection when an area is clicked
• Highlight hovered area in list in collection Download Tab
• Highlight list selection on map in Download Tab

Reduce size of network requests

• Network requests are quite large. After some time spent on the current app, 83.6mb was downloaded. That's about double Airbnb in roughly the same amount of time
• Primary targets for reduction in network cost are image sizes and mvt tiles.
• Right now, the catalog view pulls both quad, qquad, and county. Reduce to just county to examine cost reductions
• quad and qquad should load only when necessary. This can be done in the collection download view when the dl table mounts into the dom
• Since each zoom action pulls the loaded data sources again, each map source may need to be un-mounted and re-mounted on view changes. This should be fairly easy to handle using useEffect in each respective view.

Only show pertinent county / quads / qquad in collection view

Currently, all counties/quads display in the collection view. Only pertinent areas should display, as done in current datahub

Zip files not included in orders

Orders coming in from datahub do not include uploaded zip file, instead they are listed as undefined.

Collection View "Back" button does not go to "home" if first page in browser history stack is from another website

Expected behavior:

scenario 1
navigate to data.tnris.org -> navigate to collection -> press back button -> takes you to home
scenario 2
copy link to collection into browser directly -> press back button -> takes you to home

Actual behavior

scenario 1
as expected
scenario 2
goes to new tab page / previous page in browser history

Suggested Solution

The current implementation of the back button just pops from the browser history stack, so the actual behavior in scenario 2 makes sense. Checking if the previous url in the browser history has an origin of data.tnris.org and conditionally routing may be an appropriate solution to ensure users are directed to the home page.

ZoomTo collection footprint on entering collection view

Use new fileSystem API to allow download queue in Chrome-based browsers

• https://developer.chrome.com/docs/extensions/reference/fileSystem/
• New updates to chrome fileSystem API may permit sufficient access to create a downloads queue to facilitate a downloads checklist interface in collections view

Collection Missing from Front End

@ctrepka @L-Har The collection https://data.tnris.org/collection?c=af8fc6c9-3d3a-4a55-b08d-697d4d4dadb7#9.26/26.4386/-97.375 is missing from the front end of data.tnris.org. It exists when pathed directly to, and is listed as public in the api.

Add GA4

Add WMS preview toggle where appropriate

External Entity Cards

The External Entity Cards have no resources to be downloaded or ordered. Could those Tabs not show on cards?

https://data.tnris.org/?inc=24&map=false&pg=1&availability=External+Link#5.5/31.33/-99.341

BBox into Collection Cards

Could we investigate passing the bbox from the address or drawn search area into the collection card download map. So download map zooms that area.

Online Mapping Services Text

online mapping services text has an incorrect link. on both imagery and feature based collection cards. missing the REST portion of the URL.

Feature

https://feature.tnris.org/arcgis/services instead of https://feature.tnris.org/arcgis/rest/services

Imagery

https://features.tnris.org/arcgis/services instead of https://imagery.tnris.org/arcgis/rest/services

Create map legend displaying labels for different map highlights

Geofilter, catalog hover, collection view layers, etc should have distinct colors / styles outlined in a map legend

Add Bathymetry Catergory

Can we add the Bathymetry category to the front end UI drop down?

Category has been added to API

Geocoder Extent returning non-intersecting collections

When using 'Search by location bbox' several collections were returned that did not intersect with the extent of the AOI

Good example would be Lake Athens. https://betadata.tnris.org/?map=true&pg=1&inc=24&geo=-95.7856702%2C32.1970145%2C-95.7190112%2C32.2335574&category=Lidar#7/31.262/-97.031
Location name search: 'Lake Athens' and filtered by category 'Lidar', returned 7 collections 5 of which do not interest the extent.

ZoomTo collection footprint on hover in catalog

Blacklist Error Message

Current message:
"why you blowin' up our spot? excessive and suspicious request activity has been identified. contact us about your operations and intentions to regain access. [email protected]"

Suggestions:
Maybe adding standard sentence case.
Maybe a high level desc of what and why it is happening.
Maybe telling them the subject to use in the email to [email protected]. i do not look at that mail box too often. was looking to setup a forwarding rule. Mr Wade also happened to sent something into the ticketing system.

Sample:
This IP address has been flagged for excessive and suspicious request activity. This happened because someone from the outgoing IP address has been downloading a lot of data, either from the site, the bulk down loader, or the CLI. It is an automated function pulled from information within the download logs and blocks the "offending" IP address, similar to a SPAM filters. It can be triggered based on the amount of data (1000 of files and|or 100 of TBs) or that someone is repeating downloading the same file over and over again within a period of time.

Contact us about your operations and intentions to regain access. [email protected] Please use the email subject: Unable to access tnris.org

Lidar Blurb

About Lidar Data
Lidar data for Texas is made available online through the use of data compression using LASTools. LASTools is an open-source collection of tools for lidar data viewing and manipulation. Click here to download a complete index of all available lidar data at TNRIS.

Old Location:
src/components/DialogTemplateListItems/LidarBlurb.jsx

Test and ensure accessibility of key aspects of data.tnris.org

Given agency initiatives surrounding accessibility, use of ARIA tags and other key accessibility features will be better developed early than later. Some features of the new application will assist with this task, such as the download list (map objects are not easily made accessible to screen readers), but other aspects of the react application such as components mounting and unmounting will need to be kept in mind

Filter downloads list to downloads for an area type when hovering over that area in the map

Add geometry of collection footprint to collection table

TODO

• Create area_type_geometry table containing fields area_type_id and geom
• Create collection_footprint table containing fields collection_id and geom
• Add GeoDjango filter to collections endpoint that performs ENVELOPE query on collection footprints with bbox

---

Suggested Solutions

• For data pipeline, use combination of ST_SnapToGrid and ST_INTERSECTION to get simplified intersection of all areas related to a given collection.
• Create collection_footprint as separate table in Django.
• Populate existing collections using PgAdmin4.
• Add collection_footprint.geom to collection viewtable sql.
• Add geofilter to collections viewset
• Add Django Signal to trigger lambda to generate collection_footprint.geom when a new collection is created
• Create Lamdba endpoint for above purpose

Consume historical and contemporary datasets at single endpoint

Add area-type-name to /resources endpoint view table

Add endpoint to area-type-name to /resources endpoint view table for downloads list table

Add mapdraw search to map pane and synchronize with geofilter / url

New metadata file needed for Pecos Dallas Hypsography

@csidenblad Please update the metadata file for the Pecos Dallas hypsography dataset. Current PDF is unreadable. Below is PDF to use.
usgs2019-metadata-contours.pdf

Add Google Analytics 4

• #20

Add ids to important dom action elements

Add S3 URI and Site for Download Template.

Im looking to add the S3 location to the data.tnris.org download template.

S3 site ex. - https://tnris-data-warehouse.s3.us-east-1.amazonaws.com/index.html?prefix=LCD/collection/
S3 URI ex. - s3://tnris-data-warehouse/LCD/collection/
S3 Key ex - stratmap-2010-1m-lee-leon-madison-milam/

To do this for download collection i think we will need to field or table to the DB, to hold the key (dir) locations. Unfortunately the key and collection names are the same and you can not build the URL|URI from existing information within the collection. The key location will need to be manually added by @csidenblad or myself.

i was think we could add the input location to the console here:

Maybe adding it to the download template here, under Metadata. Maybe collapsible too.

As also, Im open to improvements on this rough idea.