chubl/openmindat

A Python package for the OpenMindat API.

openmindat's Issues

Data queries involving multiple pages might return an Internal Server Error.

Bug Report

Bug Type: Server

Observation:

Once the quantity of the queried item is less than the page size, the server will return a Server Error (500).

Case Study:

url1 = ‘https://api.mindat.org/geomaterials/?id__in=&lustretype=Adamantine&page_size=200’

The query of url1 involves no more than 600 items; hence, page_size=200 is functional for the first two pages. However, the third page(https://api.mindat.org/geomaterials/?id__in=&lustretype=Adamantine&page=3&page_size=200) is not accessible.

If we increase the page_size to exceed the total item quantity, say 800, the query (https://api.mindat.org/geomaterials/?id__in=&lustretype=Adamantine&page_size=800) will fail directly.

Hazard:

The queries involving multiple pages will fail, e.g., the geomaterials endpoint is somehow experiencing an outage.

Mineral Occurence Endpoint

Hello!

Thanks for the awesome work opening up this great data. The OpenMindat paper shows Mineral Occurrence records linking Mineral Species to Localities. Can end points for accessing that data be added to the API?

Data License

Hello!

The only license I currently see associated with the data under the API is the Apache license for the API access code. The OpenMindat paper suggests, however, that "We are creating a forked copy of the CC-BY-NC-SA 4.0 licence tailored specifically for these needs.". Is a copy of that license* available anywhere.

An AUP with any quota and rate guidelines would also be helpful.

*As a side note, I really do hope this licensing policy is reconsidered. It adds significant friction and limits the openness of the data significantly (see, e.g. https://ben.balter.com/2016/08/01/why-you-shouldnt-write-your-own-open-source-license/ and http://esr.ibiblio.org/?p=4559).

FEATURE REQUEST: Retrieval without writing to file

So far as I can tell, the library requires any requests to be saved to a file in order to retrieve the data. It would be much more helpful if results could be retrieved as a function result or, ideally, an iterable list of model objects or dicts.

Freezing URL Query for locgeoregion2

api.mindat.org/locgeoregion2/

The above URL seems to be causing a heavy data query with too many items at the same time, which causes the webpage to freeze.

GeoJSON Location Information

My use case for this API is extracting Mindat information into a GIS consumable format (probably Spatialite). The OpenMindat paper suggests that locations have "[b]oundary polygons recorded in GeoJSON." Can that be added to the API?

Best,

0 used as missing data placeholder

I've noticed that, for many fields, 0 is being used as a missing data representation. For latitude and longitude, however, this is particularly problematic, since 0N0E is a real place! (https://en.wikipedia.org/wiki/Null_Island). Testing for zero equality is also troublesome, since equality is ill-defined over floating point values (such as latitude and longitude).

Can the API return None or some other marker for these missing values instead of 0, or does the underlying data have the same ambiguity?