mycsina/disa-backend

Installation

Prerequisites

• Python 3.11 or higher
• poetry

Installing dependencies

You can install the dependencies by running the following command:

  poetry install

Running the server

You can start the server by running the following command:

  poetry run hypercorn start:app --reload

Testing

The postman collection used to test the API is available here

To facilitate testing, there is a toy database included in the test folder. To use it, add the following environment variable:

  export TEST=1

It comes with:

• A user:

{
  "name": "test",
  "password": "$2b$12$iFMxI.xziyqiG7lS8jBnB.NLhU7aNTwlizXR8KrRil.zVrMncrwpi",
  "cmd_token": null,
  "token": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiI5MTBkYzBjMC1jMGNhLTQ5NjctODkzMS05MmVkNmM5OGE0OGUiLCJleHAiOjE3MTI3OTQ1Njh9.B1edr-57h9LbID6BZvL5gCpM8GrANQRU1jT6K0CVZ2M",
  "id": "910dc0c0c0ca4967893192ed6c98a48e",
  "email": "[email protected]",
  "nic": "237583734",
  "role": "USER"
}

• A simple collection:

{
  "name": "README.md",
  "share_state": "private",
  "access_from_date": null,
  "id": "69c7cd429a94437bb222be8df0d1d112",
  "owner_id": "910dc0c0c0ca4967893192ed6c98a48e"
}

• A complex collection:

{
  "name": "c2023-adv-06.tar.gz",
  "share_state": "private",
  "access_from_date": null,
  "id": "9872c9b88f5f4eb99290613fafc948f9",
  "owner_id": "910dc0c0c0ca4967893192ed6c98a48e"
}

Development

Migrations

To create a migration, run the following command:

  poetry run alembic revision --autogenerate -m "migration message"

To apply the migration, run the following command:

  poetry run alembic upgrade head

To downgrade the migration, run the following command:

  poetry run alembic downgrade -1

Need help?

@mycsina on most platforms online

disa-backend's Issues

Create notion of collection permission

Implement collection download

Create collection in Paperless-ngx

Upload documents to paperless-ngx and group them either by tags or correspondents (tags seem preferable, unless we use correspondents as a kind of user profile)

Return manifest

The manifest should be returned on an endpoint to be available in the frontend.

Manifest format

[
   {
       "path": ...,
       "hash": ...
   },
   .....
]

hash: sha256 hash of the file
path: the path of the file relative to what the user entered

Save the manifest signature

Verifying manifest hash agains the given blockchain address

Add submission date and last change attributes to collection by user endpoint

When getting collections by user it would be useful to get this 2 extra attributes with enfasis on the first one.

Enpoint mentioned:
collections/user

Note: it is available in collections/info but I would like to avoid 2 different endpoints in dashboard

Implement comprehensive logging

Implement testing based on the Postman collection endpoints

We need testing atp, @mariiajoao :)

Integrate with Archivematica

sam_client.py has already been done, just need to change the logic behind uploads and also the database structure so we can use Archivematica as the storage solution

Create needed issues

As preparation for the next presentation, estimate the work needed to be done until this is able to be included in the MVP and create the necessary issues 🫡

Investigate why paperless parsers aren't working correctly.

Paperless doesn't accept all file types, as it needs a parser for each file type. There's a clumsy implementation in paperless-ngx/code_parser but currently it doesn't work

Docs on custom parsers:
https://docs.paperless-ngx.com/development/#making-custom-parsers

Change Collection name

As a User it would be useful to change the submitted collection's names for better organization (the .tar archive name could be viewable still)

Maybe consider making collection name unique to avoid confusion for the end-user.

Create share link endpoint and implementation

Change route params from type to function sig

Allows anonymous access to certain endpoints, while retaining user information if he is logged in

Verifying manifest against the given files