The crabpy_pyramid from onroerenderfgoed

Handling unexisting objects

I don't think we're handling certain things correctly. Eg. when I go to https://geo.onroerenderfgoed.be/crab/gemeenten/354556, I get a HTTP 500 error while this should actually be a HTTP 404.

Please check all routes that should be returning 404's and add these where necessary. Make sure to keep the distinct between a 404 and a 500. If the CRAB service is down or something else is wrong, it's still a 500. It's only a 404 when a certain object does not exist.

Release 0.2.0

Remove templates

There's a template dir that's probably not doing anything. Remove if possible.

Add list of postkantons in a gemeente

Please add a route /crab/gemeenten/{gemeente_id}/postkantons that calls crabpy.gateway.crab.list_postkantons_by_gemeente.

Example output:

[
 {"id": "8300"},
 {"id": "8301"}
]

/cc @cahytinne

add http caching

These services are being used a lot, some extra http caching here and there might do wonders.

Create basic sphinx doc

Create the docs folder as an empty Sphinx docs.

Figure out if crabpy.routes is still doing something

I think it's abandoned, just need to make sure.

Add adresposities

See OnroerendErfgoed/crabpy#37.

/crab/huisnummers/{huisnummer_id}/adresposities Maps to list_adresposities_by_huisnummer(self, huisnummer)
/crab/subadressen/{subadres_id}/adresposities Maps to list_adresposities_by_subadres(self, subadres)
/adresposities/{adrespositie_id} Maps to get_adrespositie_by_id(self, id)

Don't forget to update the service docs.

Module header

Provide every python module with a header that contains the following:

# -*- coding: utf-8 -*-                                                         
'''                                                                             
<Description of Module>
.. versionadded:: 0.1.0
'''

add class Provincie views

Add views list_provincies, get_provincie_by_id and List_gemeenten_by_provincie

Verwijderen rq_dashboard

Aangezien we het rq-dashboard kunnen raadplegen in de monitaurus mogen alle verwijzingen naar rq-dashboard uit de toepassingen gehaald worden. Dus indien van toepassing moet de view verwijderd worden en rq-dashboard uit de requirements gehaald worden.

bounding box for huisnummer and straat

Since a bounding_box property has been added to huisnummer and straat objects in crapby, we will be adding it here as well.

Add niscode to Gemeente renderer

Currently the Gemeente renderer does not include the niscode.

Upgrade crabpy==0.8.2, crabpy_pyramid==0.6.2

Aangemaakt in bulk, indien al ok of nvt gewoon sluiten.

Include CAPAKEY by default?

In the past, Capakey was not included by default because this required authentication. Since version 0.6.0 we're using the REST service, which does not require authentication. Might be useful to add Capakey by default from now on.

It does break BC somewhat, but I guess if we're the only ones using it, it does not matter so much.

Create basic skeleton

Create the basic directory structure:

crabpy_pyramid package
setup.py
MANIFEST.IN
CHANGES.rst
README.rst

0.6.0 Create unit test skeleton

Setup basic stuff for unit testing:

crabpy_pyramid/tests module
unit testing configuration: tox.ini, nose files, .travis.yml

Coveralls has wrong coverage stats

Hi @claeyswo and @TalissaJoly,

coveralls reports coverage as being 38%. I don't think this is entirely correct.

According to the README at https://pypi.python.org/pypi/coveralls/0.4.1, there's a problem with collecting coverage from nose. Some packages need to be explicitly excluded.

Would running nosetests --config nose_cover.cfg solve this problem?

Otherwise, the README suggest modifying the .coveragerc file.

Correct handling of start and count

These need to be passable through the Range headers used by dojo:

Range: items=0-19

Meaning, give me 20 items, starting with item 0. If this header is not present, a X-Range header needs to be checked (used because Opera blocks the normal Range header).

X-Range: items=0-19

Responses need to incorporate the Content-Range and X-Content-Range to indicate what slice of data is being returned.

Content-Range: items=0-19/50
X-Content-Range: items=0-19/50

This indicated that the first 20 of 50 items are being returned.

Response format for parcel is broken

In version 0.6.0:

{
    "percid": "11372_A_0005_P_000_00",
    "sectie": {
        "afdeling": {
            "naam": "AARTSELAAR  2 AFD",
            "id": 11372,
            "gemeente": {
                "naam": "Aartselaar",
                "id": 11001
            }
        },
        "id": "A"
    },
    "capakey": "11372A0005/00P000",
    "bounding_box": "{\"coordinates\":[[[149254.71962500364,204170.85500000045],[149328.96400000155,204170.85500000045],[149328.96400000155,204255.58281249925],[149254.71962500364,204255.58281249925],[149254.71962500364,204170.85500000045]]],\"type\":\"Polygon\",\"crs\":{\"type\":\"link\",\"properties\":{\"href\":\"http://www.opengis.net/def/crs/EPSG/0/31370\"}}}",
    "centroid": "{\"coordinates\":[149291.84181250259,204213.21890624985],\"type\":\"Point\",\"crs\":{\"type\":\"link\",\"properties\":{\"href\":\"http://www.opengis.net/def/crs/EPSG/0/31370\"}}}",
    "id": "0005/00P000"
}

In version 0.5.1:

{
    "percid": "11372_A_0005_P_000_00",
    "sectie": {
        "afdeling": {
            "naam": "AARTSELAAR  2 AFD",
            "id": 11372,
            "gemeente": {
                "naam": "Aartselaar",
                "id": 11001
            }
        },
        "id": "A"
    },
    "capakey": "11372A0005/00P000",
    "bounding_box": [
        149254.719625004,
        204170.855,
        149328.964000002,
        204255.582812499
    ],
    "centroid": [
        149291.841812503,
        204213.21890625
    ],
    "id": "0005/00P000"
}

The API of version 0.5 should be maintained. This might also mean that the problem is present in CRABpy 0.8.0 and that we've broken the API there as well. Should be fixed before we start integrating the new version in exsiting applications.

Caching tween volledig herwerken

https://github.com/OnroerendErfgoed/dossierdata/issues/232

Samengevat:

We implementeren optie 1. Standaard staat de tween aan, maar je kunt ze makkelijk afzetten. (Dit om backwards compatible te zijn. We kunnen er ook een opt-in van maken, maar dan moet dat SemVer gewijs duidelijk zijn)
Indien de tween actief is, dan hanteert deze een lijst van routes die cacheable zijn. Zorg dat het eenvoudig is om de lijst van routes uit te breiden in een bepaalde toepassing.
HTTP responses >= 400 krijgen geen ETag

Ik zou op PUT en POST die ETag nog wel meesturen. Ik zie er mss ook niet direct het nut van in maar ik durf niet te garanderen dat we daar al niet ergens van uitgaan.

Update pycountry to latest version

Updating Pycountry to a version > 1.20 breaks CRABpy_pyramid. Think we should update to the latest version (semVer compatible) and pin to a version > 1.20 in setup.py

Maximum op aantal resultaten in percelen

https://github.com/OnroerendErfgoed/crabpy_pyramid/blob/master/crabpy_pyramid/utils.py#L10

Blijkbaar is dit enkel als er geen Range headers worden meegestuurd.

Add deelgemeenten

Add a service that exposes a list of deelgemeenten.

drop sort parameter list_gemeenten_by_provincie

Drop the sort parameter from this view since it is no longer supporter in crabpy 0.4.2

Update to CRABpy 0.11.0

Update once released.

crabpy updaten naar 0.8.2

Ter info en voor toepassing voor vele projecten: Vrijdag werd een update van crabpy naar 0.8.2 uitgevoerd.
De capakey rest service van Informatie Vlaanderen is niet langer toegankelijk op de URL die we tot 0.8.1 gebruikten. Daardoor faalden zoekopdrachten op het geoportaal. Wellicht dient dit in nog tal van projecten te worden aangepast, al heb ik geen idee voor welke projecten dit absoluut noodzakelijk is.

Release notes crabpy 0.8.2: https://github.com/OnroerendErfgoed/crabpy/releases

Indien van toepassing - upgrade naar pyramid-tm 2.2.1

Naar aanleiding van Pylons/pyramid_tm#67

Add a list of countries

Add rest endpoints, exposing stuff from the pycountry dataset:

/crab/landen: A list of countries, containing the ISO 3166 country code and a label in Dutch (if available).
/crab/landen/BE: More information on a certain country.

Example JSON responses:

List

[
 {
  'id': BE,
  'naam': "België"
 }
]

Detail

{
 'id': 'BE',
 'alpha2': 'BE',
 'alpha3': 'BEL',
 'naam': "België",
}

Switchen van wsgicors naar wsgicoers

Indien van toepassing. In de requirements wsgicors dependency verwijderen en toevoegen wsgicoers==0.8.2. In prodcution.ini use = egg:wsgicors#middleware vervangen door use = egg:wsgicoers#middleware.

Allow passing custom crab wsdl

http://crabpy.readthedocs.io/en/latest/_modules/crabpy/client.html?highlight=wsdl

This parameter should be available in crabpy_pyramid

0.7.0 Error handling capakey

When doing capakey requests for a single perceel, GatewayResourceNotFoundExceptions are not being handled.

/cc @cedrikv

Implement subadressen

Implement Subadressen as added to CRABpy in OnroerendErfgoed/crabpy#34.

Routes wanted:

list_subadressen_by_huisnummer, /crab/huisnummers/{huisnummer_id}/subadressen
get_subadres_by_id, /crab/subadressen/{subadres_id}

These should be added to crabpy_pyramid.routes.crab.

Renderers should be similar to those for Huisnummer. In list mode id, status and label (=actual Subadres). In get mode the full object.

Can you develop this in a branch and send a PR when ready?

Make compatible with CRABpy 0.8.0

Upgrade naar pyramid_oeauth==1.4.0

Indien van toepassing, anders sluiten. Zorgt ervoor dat na afmelden van werkrelatie veranderd kan worden.

0.6.1 Filtering empty settings

Sometimes empty settings are provided, eg.:

crabpy.proxy.http =
crabpy.proxy.https =

This leads to errors downstream in urllib. Should be possible to filter theses settings out at initialisation.

Allow passing of proxy info to the capakey gateway

Allowing setting proxy info through the settings file.

Upgrade naar pyramid_oeauth==1.3.0

Indien van toepassing, anders sluiten. Wel zorgen dat Pyramid versie >= 1.5.

Add get_capakey_gateway method

Add a way to get a capakey gateway.

Upon startup the settings should be used to create a capakey gateway and add it to the pyramid registry. This means that the following settings need to be passable:

capakey.user
capakey.password
capakey.wsdl

A get_capakey_gateway() directive should be added to the configurator.

A capakey_gateway property should be added to the request

Fix voor gewijzigde volgorde feed na release feedgen==0.6.0

Indien hier van toepassing, dus feed aanwezig in project. Anders mag deze direct dicht. Zie https://github.com/OnroerendErfgoed/premieaanvragen/issues/297 voor wat meer achtergrond. De fix die Bart daar heeft toegepast werkt voor alle versies van feedgen dus gewoon overnemen.

Inleiding: wat is het doel van de applicatie
Architectuur: Datamodel, gebruikte componenten, Interacties
ontwikkelinfo: hoe het project opzetten als ontwikkelaar