Giter Site home page Giter Site logo

archiefbeheercomponent's People

Stargazers

 avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

Forkers

gemeenteutrecht

archiefbeheercomponent's Issues

Ik wil als record manager verschillende versies van een vernietigingslijst op kunnen slaan,

zodat anonimisering plaats kan vinden. Soms kan de beperkte informatie die in principe op een vernietigingslijst straat toch meer vertellen dan de bedoeling is. In dat geval wil ik gegevens uit de vernietigingslijst kunnen weglaten.
[aanvullen met voorbeelden; wordt deze gebruikt in de verklaring van vernietiging?]

[Thema uit conceptofferte: 2. Proces-verbaal/rapport genereren van vernietiging]
[Thema uit conceptofferte: 5. verschillende versies van rapport mogelijk maken (anonimiseren)]

Implement destruction of the cases in a destruction list

When the list is finally approved, the actual destruction starts.

This means that the hard DELETE calls to zaken must be made, and the related documents. Documents may only be deleted if they are no longer related to any other non-archived cases. See the VNG standard on what it means to "delete a zaakdossier".

The actual destruction must be fault-tolerant - failure while deleting a case is expected. Any outcome must be logged/reported in django-timeline-logger.

When destruction starts, the status of the DestructionList must be flipped to in progress. When it's completed, it must be flipped to completed.

After completion, the record manager should be notified of the outcome.


Technical consideration

Confirmation should start a celery job performing the actual destruction, this will be long-running and should be kept out of the request-response cycle.

Implement landing page for reviewers of destruction lists

The record manager created the destruction list and assigned/scheduled the reviewers for this list. When such a reviewer logs in, they should be directed to their landing page, with the following elements:

  • Filters
  • An overview of destruction lists matching the filters
  • A notifications area

This page is only avaible for roles with can_review_destruction enabled.

image


Filters

The filters should allow for:

  • destruction lists that have been reviewed ("beoordeeld")
  • destruction lists that haven't been reviewed ("te beoordelen") (default)
  • both

overview of destruction lists matching the filters

The name, creation date and author of the each destruction list should be visible. If the list has not been reviewed yet, there should be a call-to-action (button or link) to review that particular list.

Notification area

This is the same as in #5

Add filter on einddatum in Zaken API

For selection Destruction list items it'd be convenient to filter zaken on einddatum, but there is no such filter in Zaken OAS

einddatum, einddatum_gte etc. filters should be added to the Zaken API

Implement destruction list creation for the record manager

A record manager is responsible for the creation of destruction lists. The record manager has the knowledge to "plan" the process that happens after creation.

For the record manager, it's important that they can select the cases in an efficient way that are to be destroyed. After creation, the list is "sent" to other roles for review.

The creation page must minimally provide:

  • data filters
  • a presentation of data matching the filters
  • the option to "freeze" this list into a collection for destruction

The below wireframes give an impression of these core elements, a more detailed description is found below.

image


Filters

The following filters should be available (and none of them are required). Combining filters works in an AND fashion - they reduce the result set further on top of each other.

  • zaaktypen: a multi-select of available zaaktypen in the configured catalogi API(s). Record managers should be able to select a subset of zaaktypen (one or more), and only see zaken belonging to those zaaktypen.
  • start date: should filter zaken on startdatum__gte - only include zaken started on or after the selected date
  • end date: should filter zaken on einddatum__gte - only include zaken finished on or after the selected date (@JurjenBert - is this right?)

The resultaten filter in the wireframes can be left out. The Action radio select should be present, but disabled (as we currently only implement destruction and not transfer of records).

Cases list

The main panel shows the list of zaken matching the selection criteria. The displayed columns should default to "omschrijving", "zaaktype", "startdatum", "einddatum" and "archiefactiedatum". The end-user should be able to configure which columns to display (nice to have).

Cases must always be limited to cases where the archiefactiedatum is in the past (or current date), and have archiefnominatie=vernietigen.

Every row represents a single zaak. There is a checkbox to include the zaak on the list for every zaak. The table header should have a checkbox for "select-all" functionality.

When clicking a zaak-row, detail information should be shown, but only if the user-role has can_view_case_details

A case can only be on a single list - this means that if any other lists have the case as scheduled, it should be presented in the result list as "grayed out"/disabled.

the option to "freeze" this list into a collection for destruction

Currently this is represented in a button in the top right, showing a count of the zaken, but this should be made more accessible/explicit.

Clicking this button then brings up the interface to "finalize" the destruction list.

image

  • It should show how many cases are included
  • It must have a form field to give a name
  • it must have the ability to assign reviewers - the record manager knows who those should be
    • reviewers are added by selecting a user
    • the order of reviewers is important
    • for each selected reviewer, their assigned role (Role model) should be displayed

A confirmation button then causes this data to persist and the process to start.


Side effects

  • When a list is created, a notification should be sent to the next assignee (=first reviewer). This should be a notification in their notifs. area AND an e-mail to the user with a link to the review page for that particular list.

  • audit records should be created in django-timeline-logger that log the creation of the list. additionally, the list of zaak-urls included should probably be logged in the extra_data as well


Technical considerations

This page is highly dynamic in user-interaction, so it will probably make sense to set up own API-endpoints and implement the bulk of the page with React (components). This also allows us to re-use some components from the NLX team (@sergei-maertens follow up)

Als record manager wil ik kunnen zoeken naar afgesloten zaken waarvan de archiefactiedatum nog niet berekend kan worden

zodat ik inzicht krijg in achterstallige administratie.

[thema uit concept offerte: 1 Ophalen van zaken zonder berekende archiefactiedatum & bewerken archiefparameters]

Tasks

  • Knop toevoegen naast "Opstellen": "Toon zaken zonder archiefactiedatum"
  • Dit niewe scherm toont zaken met: Resultaat moet gezet zijn + Zaak afgesloten + Geen archiefactiedatum
  • Sorteer de lijst altijd archiefactiedatum (null is last) + zaaktype naam

Zie ook: #54 voor verder invulling van nieuwe scherm.

Implement the destruction list review page for reviewers

A reviewer lands on this url/page after following a link from a notification or clicking through from the landing page. The page should only be available for roles with can_review_destruction enabled.

The main elements on the page are:

  • general information (name, creation, author)
  • overview of cases on the list
  • approval button
  • widget/panel with case-detail information (only if the role has can_view_case_details set)

image


Happy flow - approved

If there are no remarks on the contents of the list, the reviewer approves it by clicking the approval button. This invokes:

  • creating a list review object (setting the correct author and status etc.)
  • create a django-timeline-logger record
  • assign the list to the next reviewer or
  • start the destruction process if there are no other reviewers

Unhappy flow - remove items

A reviewer can click a row, which brings up the details (if they have the permission!) and the actions to "exclude" the item from the list or change it.

If they exclude it from the list, the list item should be marked as removed, and a log record of this needs to be created. Optionally the reviewer can enter a reason, which is tracked in the list item review model.

The list is then assigned to the original author again, who re-submits it to the reviewers.

Unhappy flow - edit items

Similar to the previouse case, they can also propose a change or give a remark that's sufficient for the record manager. The record manager then makes the change, and marks the record as removed from the list. A log record of the proposed change should be created.


Side effects

Whatever the outcome of review is, a notification to the original owner (record manager) must be sent (both notifs. area and email).

As a record manager I want to see if and why archive data has been changed when selecting cases for a destruction list,

so that I can take the adapted archive data into consideration when compiling lists.
The addition of a column containing the reason for changing (see #17 ) might be enough, although cases with changed archive should stand out in the selection.

During or after closing a zaak, a user can sometimes change archive data. The user will provide a reason for changing the data. The record manager should be able to see which cases deviate from what is configured in the corresponding zaaktype. The record manager should also be to see the reason for changing the data.

Small improvements to destruction list detail page

Todo:

  • implement additional permissions: display page only for DL author and assignees, enable updating onlyt for the author who is assigned to DL at the moment
  • add X-Audit-* header when updating zaken
  • add button to cancel the destruction list (nice to have)

When compling a vernietigingslijst as a record manager I want to compile lists in the context of the relationships cases have with other cases,

so that I can dispose of data properly.

Check needed with Openzaak and with our record managers how this should work.

I think is should work as follows: you can destroy a bijdragezaak or vervolgzaak when the hoofdzaak still exists. I dont think you can destroy a hoofdzaak when is still has bijdragezaken. In the current RMA solution in gemeente Utrecht, there is simply an indicator per case that signifies: this case is related to one or more other case(s).

As functioneel beheerder i would like to be able to generate testcases for destruction,

so that i can easily generate testcases that can be used for testing purposes. Typically testcases will destroyed after each test.

To save time it would be convenient to automatically generate a number of cases that are closed and have archivedata that make them suitable for destruction.

If this is realistic: it would be good to add some documents, and different types of archive data.

The alternative is preparing the cases by hand, which is not a big problem.

Apply permission checks to the list-items endpoint

See comment:

Please add an extra permission check to the rma.destruction.api.FetchListItemsView endpoint. We want to prevent users logging in with ADFS who are not assigned to any role to be able to view the content of lists, as they expose some details on zaken.

Additionally, I think that depending on the role permission (can_view_case_details), the zaak dict needs to only include the fields that are used in the table, and no extra details. Better to use an allow-list and be explicit about it.

Maybe we can define a mapping from role permission to a list of attributes that are are allowed?

Ik wil als functioneel beheerder kunnen configureren welke kolommen getoond worden,

op verschillende schermen in het proces en in een opgeslagen vernietigingslijst. Zodat de betrokkenen in het vernietigingsproces de beschikking hebben over precies die informatie die ze nodig hebben.
Denk aan gegevens als zaakomschrijving, selectielijstcategorie, vigerende selectielijst. Omdat de gegevens die een record manager enigszins variëren van gemeente tot gemeente, moet een beheerder aan kunnen geven wat relevant is voor de desbetreffende implementatie van de vernietigingsapp.

[thema uit conceptofferte : 3. Additionele velden configureerbaar maken en toevoegen aan de vernietigingslijst]

Als functioneel beheerder wil ik weten hoe vernietiging van gerelateerde data buiten de ZRC en DRC zijn werk gaat,

zodat ik relevante user stories op kan stellen op dit vlak.
Ik denk in ieder geval aan notificaties over vernietiging naar aangesloten systemen, zodat ook in die systemen vernietigd kan worden. Met welke notificaties moet een leverancier van een ander systeem uit de voeten kunnen?

[thema uit concept offerte: 8. Vernietigen van gerelateerde data in andere systemen documenteren en inhaak-systeem opzetten.]

Clearification

Het moet duidelijk worden hoe andere applicaties/componenten moeten omgaan met gegevens bij een zaak, als deze zaak wordt vernietigd. Samengevat:

  1. VL uitvoeren
  2. Zaak wordt vernietigd
  3. Andere applicaties ontvangen een notificatie dat de zaak is vernietigd
  4. Andere applicaties moeten dan hun gegevens ook vernietigen

Dit documenteren.

Implement record manager landing page

When a record manager logs in to the application, they should be redirect to their landing page.

The landing page must contain the following elements:

  • A list of authored destruction lists (clicking through leads to a detail page)
  • A button to start a new destruction list
  • An area for notification display

The wireframe below sketches out the elements - this should serve as inspiration but the exact layout needs to be user-friendly.

image


Detailed specification

List of authored destruction lists

For each destruction list presented, the following data must be visible:

  • name
  • current status
  • an icon/indicator that feedback/review has been given and/or actions for the record-manager are required

The list should be sorted by most-recent first.

A button to start a new destruction list

The button should be clear and take you to a new page dedicated to creation.

An area for notification display

Notifications should be ordered by most-recent first. Clicking notifications should take you to the right page if relevant, otherwise they should present their content in an expanded view. It should be possible to delete notifications. The notification data model is WIP.

Implement auth data model

Authorizations are to be role based, and roles will have a prominent place in the process workflow as well. Therefore we need a number of models in the accounts app to coordinate this:

User model

This is the standard user model from default project, but extended with a foreign key to Role. Note that m2m is not applicable here, as a user can only have one role to fulfill.

Usernames and e-mails must be unique. Users will eventually be provisioned via SCIM, but for now ADFS login will be used.

Role model

  • name: name of the role
  • organisatie_onderdeel: URLField, optional (will be implemented later for data filtering)

Permission fields:

  • can_start_destruction: BooleanField - designates whether people in this role can create a "vernietigingslijst", which is the starting point for the "records destruction" process.
  • can_review_destruction: BooleanField - designates if people in this role can review created record destruction lists and approvate/reject them
  • can_view_case_details: BooleanField - designates whether people in this role can view the contents of zaken listed on the record destruction lists

Als record manager wil ik -vanuit een overzicht van afgesloten zaken zonder archiefactiedatum- ontbrekende archiefgegevens kunnen aanvullen

zodat de archiefactiedatum berekend kan worden. In voorkomende gevallen wil ik ook archiefgegevens aan kunnen passen

[thema uit concept offerte: 1 Ophalen van zaken zonder berekende archiefactiedatum & bewerken archiefparameters]

Tasks

  • Knop toevoegen naast "Opstellen": "Toon zaken zonder archiefactiedatum"
  • Toon lijst van zaken zoals bij opstellen MAAR met de volgende wijzigingen:
  • + kolom: zaaknummer
    • zoekveld op zaaknummer => #254
  • + actie bij klikken: Instelen van archiefactiedatum IN OPEN ZAAK (en daarmee verdwijnt deze zaak dus uit deze lijst).

Als record manager wil ik dat er een verklaring van vernietiging (of proces verbaal) wordt gegenereerd over de vernietiging van documenten

zodat ik kan voldoen aan de archiefwet. De verklaring van vernietiging (of proces verbaal) is het bewijs van daadwerkelijke vernietiging. Hierin is opgenomen: een specificatie van het vernietigde (de vernietigingslijst), op grond waarvan vernietigd is (de selectiecategorie) en de wijze waarop vernietigd is (een beschrijving hoe technisch is vernietigd; hierbij is het van belang duidelijk te maken hoe ervoor is gezorgd dat onherstelbaar is vernietigd).
graag oppakken met #62

[Thema uit conceptofferte: 2. Proces-verbaal/rapport genereren van vernietiging]

--

Clearification

There should be like a template document with a table as attachment. This template document should be configuratble. Columns for the attachment are:

image

  • SAD should be gemeentearchivaris.

Guestimate

  • Uniek kenmerk = Zaak.identificatie
  • Beschrijving = Zaak.omschrijving
  • Looptijd = Zaak.Einddatum - Zaak.Startdatum
  • VCS = Zaak.Zaaktype.Selectielijstprocestype.Nummer
  • Toelichting = App.DestructionList.toelichting
  • Opmerkingen = App.DestructionList.comments
  • Reactie zorgdrager = Comment from process owner

Ik wil als record manager de verklaring van vernietiging kunnen aggregeren,

zodat een toezichthouder relevante informatie op de verklaring ziet. Aggregatie zal bijvoorbeeld plaatsvinden op het niveau van zaaktype en selectielijst categorie, periode: in plaats van een vernietigingslijst met daarop bijvoorbeeld 1000 zaaknummers van zaaktype x, wordt o.a., aangegeven dat er 1000 zaken van zaaktype x zijn vernietigd en onder welke selectielijstcategorie dat is gebeurd.
[gewenste mogelijkheden voor aggregatie moeten verder worden uitgewerkt]

[Thema uit conceptofferte: 2. Proces-verbaal/rapport genereren van vernietiging]

While destroying a list item, the status changes are not persisted to database.

Most notably:

https://github.com/GemeenteUtrecht/record-management-app/blob/master/src/rma/destruction/tasks.py#L59 and https://github.com/GemeenteUtrecht/record-management-app/blob/master/src/rma/destruction/tasks.py#L47 - the FSM state changes happen, but the instance does not commit to database. This leads to information loss that the item is being processed (could be useful for progress presentation), but more importantly the failure isn't recorded.

Implementation of destruction list creation - step 2

todo:

  • show zaak details if there are related permissions
  • format back-end form errors in react
  • make reviewers number dynamic
  • add debounce and loading status for zaken
  • disable zaken which are already included in other destruction lists moved to #37
  • get rid off getFullUrl and use consumerjs
  • use all the states in root node (move selectAll from ZakenTable to the parent node)

Step 1 - #6

Create login page for users

At the moment for login admin log is used. The separate login page for application users should be created with ADFS support

Als gebruiker in het vernietigingsproces wil ik via de mail notificaties kunnen ontvangen over de voortgang van het vernietigingsproces,

zodat ik in kan loggen op de applicatie wanneer er werk voor mij. Zo hoef ik niet proactief te controleren of er werk voor mij klaar staat. Het liefst wil ik zelf aan kunnen geven of ik notificaties ontvang, anders kan dit door een functioneel beheerder op het niveau van gebruikersrol worden ingesteld.

[thema uit conceptofferte 6. Configuratie van notificaties toevoegen]

Clearification

Als er VLs aan mij zijn toegewezen/teruggewezen als reviewer, wil ik direct een mail ontvangen (er zit iets in je werkvoorraad).

Improve UX of zaaktype-selection

When creating a destruction list, zaaktypen can be used as filters.

We expect many (versions) of zaaktypen, so the current multi-select is not a user friendly approach. The widget should be replaced with something more convenient, based on checkboxes:

- [ ] {Zaaktype X omschrijving}
  - [ ] Version YYYY-MM-DD - {zaaktype identificatie)
  - [ ] Version YYYY-MM-DD - {zaaktype identificatie)
- [ ] {Zaaktype Y omschrijving}
  - [ ] Version YYYY-MM-DD - {zaaktype identificatie)
  - [ ] Version YYYY-MM-DD - {zaaktype identificatie)
  - [ ] Version YYYY-MM-DD - {zaaktype identificatie)

Checking the left-most checkbox selects all the versions nested under it.

Implement record destruction data model

The data model for the "record destruction" is the core of the application.

Record destruction is the process of deleting data that has reached its expiry/end-of-life time. In this app, we focus on "Zaakgericht archiveren", which means we only consider cases as possible records to destroy. Record destruction is permanent, and a multi-stage process with different roles signing off and providing feedback/input. Sometimes some records need to be altered, where the expiry time can be changed for example. The described data model drives the process, and provides (the hooks for) audit logging.


DestructionList

The destruction list models a collection of records to be destroyed. It is created by a record manager, and progresses through approval states, collecting feedback along the way. It is the subject of application.

Required data fields:

  • name: a name given to the list by the record manager
  • author: the User responsible for creating the list (in the role of record manager)
  • created: timestamp of list creation
  • end: timestamp of when the list was
  • assignee: the currently assigned User
  • status: the current state, with possible options:
    • in progress ("onderhanden")
    • processing ("wordt uitgevoerd")
    • completed ("uitgevoerd")

For the status field, django-fsm is appropriate to dictate the state transitions and side effects (e.g. when it changes to completed -> end should be set)


DestructionListItem

This represents a single item on the destruction list, in practice, this is a Zaak reference. Items can be removed from the list, or require edits.

Required fields:

  • destruction_list: FK to the DestructionList that it's "on"
  • zaak: URLField, pointing to a Zaak resource in a Zaken API
  • status: individual status of a single item, there are three possible choices:
    • scheduled: the item is on the list and scheduled for future destruction
    • removed: the item was originally on the list, but has been removed from the list because of any reason (during the review phases)
    • processing: the item is currently being destroyed
    • destroyed: the item is succesfully destroyed
    • failed: destruction of the item did not succeed

Possible status transitions are:

  • scheduled -> removed
  • scheduled -> processing -> destroyed
  • scheduled -> processing -> failed

DestructionListReview

Similar to github pull request reviews - this models the review of a proposed destruction list as a single entity. More specific reviews on particular items are tracked in a separate model.

Required data fields:

  • destruction_list: FK reference to the DestructionList this review belongs to
  • author: FK to the User who performed the review
  • created: timestamp tracking when the review was created
  • text: textfield with no particular markup/structure to enter review hints from the author to the record manager
  • status: choices, either approved or changes requested

DestructionListItemReview

Models a review/suggestion/feedback on a single item on the destruction list.

Required fields:

  • destruction_list_review: FK to the review instance for the destruction list. This allows you to figure out the author of a particular item review
  • destruction_list_item: FK to the DestructionListItem subject of the review
  • text: free text field for reviewers, no particular structure
  • suggestion: choices, either remove or change_and_remove.

As record manager i want to be able to abort the destruction process,

so that the proces doesnt have to be completed if for example the comments from de proces owner are so extensive, that a new list is appropriate. The cases that were on the VL can now free to be selected for another VL.

Similarly the archivaris can provide feedback to the record manager which makes cancelling the process the most logic step for the record manager. For example: the archivaris notes that none of the cases on the VL should not be destroyed but should instead be bewaard (whats that in english?)

BTW: this is not the happy flow and should not occur much. If this occurs the record manager isn't doing his or her work very well.

Als record manager wil ik dat na afronden van het vernietigingsproces een bevestiging van vernietiging verzonden worden naar in ieder geval de toezichthouder,

zodat ik kan voldoen aan wettelijke verplichtingen.

  • Ik wil met de bevestiging van het afronden van het proces een verklaring van vernietiging meesturen
    [aanvullen: in welk bestandsformaat moet dit?]
    graag oppakken met #55

[thema uit conceptofferte 6. Configuratie van notificaties toevoegen]

Clearification

An email should be sent to the "archivaris/toezichthouder" (existing role) with a list of all zaken that were actually destroyed. Technically, each DELETE call that returned HTTP 200).

Ofcourse, an email address should be part of the user/role.

When compiling a vernietigingslijst as a record manager, I want to be able to see the colums that I need to take into consideration,

when compiling the vernietigingslijst (ie. zaaknummer, Zaaktype, Resultaat, Selectielijstcategorie, Archiefactiedatum, Reden wijziging, Datum uitvoering, Log, Relatie, Omschrijving, Einddatum).

! Not all these columns are relevent, certainly not in a MVP. In case of further development I will have to consult record managers to ascertain which columns are necessary.

Ik wil als record manager het proces van vernietiging kunnen annuleren,

zodat ik in voorkomende gevalle opnieuw kan beginnen. De zaken die voorkwamen op deze lijst zijn weer beschikbaar voor opname op nieuwe lijsten.

[overig - proces]

Clearification

Record manager creates a destruction list, and wants to cancel/delete the destruction list (JUST THE LIST). THis will cause the zaken on that list to be available again to be put onto new destroying lists. This should be possible at all times (until actual destruction ofcourse).

The cancellation should trigger a notification to the reviewer (if any).

Als functioneel beheerder wil ik achteraf kunnen reconstrueren hoe de afhandeling van het vernietigingsproces is verlopen,

zodat achteraf onderzocht kan worden hoe de vernietiging is verlopen.
Het gaat hier enerzijds om het vaststellen welke gegevens allemaal worden vastgelegd (de audittrail) en de toegankelijkheid ervan.

In aanvulling op de gegevens uit de verklaring kan worden gedacht aan informatie over:

  • wie op welk moment welke stappen in het vernietigingsproces heeft uitgevoerd
  • welke opmerkingen zijn toegevoegd
  • Welke notificaties naar andere systemen zijn verzonden
  • ?

[Thema uit conceptofferte: 2. Proces-verbaal/rapport genereren van vernietiging]

Clearification

Aan het VL rapport een audit trail toevoegen wie wat gedaan heeft.
Don't mention person names, just roles and times/dates.
Only for the PDF.

Support JSX in JS linter

As a developer I want a JS linter to support JSX

The possible solution is to replace JSHint with ESLint

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.