samagra-development / quml2xform Goto Github PK

View Code? Open in Web Editor NEW

2.0 3.0 2.0 1.74 MB

QuML ODK Interop

JavaScript 0.46% Dockerfile 0.54% TypeScript 99.00%

interop odk-xforms quml

quml2xform's Introduction

quml2xform

Description

QuML to ODK form conversion tool written in Nest framework.

Installation

$ yarn install

Running the app

Create .env file (can be copied from .env.example) and configure all the variables as per need & development environment.

Using docker

Simply hit docker-compose up -d, or use manual installation as per below steps:

# development
$ yarn run start

# watch mode
$ yarn run start:dev

# production mode
$ yarn run start:prod

Note: For Manual installation, make sure the python tool is installed system wide: https://github.com/XLSForm/pyxform#running-the-latest-release-of-pyxform. We are using this tool to generate XML from XLSX form using command line utility available from the tool: xls2xform path_to_XLSForm [output_path]

Routes

For Single Request

POST {{url}}/quml-to-odk
{
    "randomQuestionsCount": 5,
    "board": "CBSE",
    "grade": "Class 6",
    "subject": "Mathematics",
    "competency":"Data Handling",
    "qType": "MCQ"
}

For Bulk requests

POST {{url}}/quml-to-odk/bulk
[
    {
        "randomQuestionsCount": 5,
        "board": "CBSE",
        "grade": "Class 6",
        "subject": "Mathematics",
        "competency":"Data Handling",
        "qType": "MCQ"
    },
    ...
]

The params randomQuestionsCount, board, grade, subject, competency can all be sent as per need & the filters will be applied under the hood. qType for now is limited to "MCQ" only.

To generate forms via CSV-JSON

A JSON body generated from the CSV should be passed as per below format:

POST {{url}}/quml-to-odk/via-json?randomQuestionsCount=3&board=State (Haryana)
[
  {
    "Grade": "Class 6",
    "Subject": "English",
    "Question": "How old was Tansen when he went away with Swami Haridas?",
    "option 1": "Twelve years old&nbsp;",
    "option 2": "Ten years old&nbsp;",
    "option 3": "Nine years old&nbsp;",
    "option 4": "Eleven years old&nbsp;",
    "CorrectAnswer(1/2/3/4)": 2,
    "Competencies": "ENG605 Students can respond to questions by gathering information from notices, charts, diagrams, etc.",
  },
    ...
]

CSV to JSON conversion tool: https://csvjson.com/csv2json

To generate forms via ZIP URLs

A JSON body generated from the CSV should be passed as per the format below:

POST {{url}}/xslx-to-odk/via-json
[
    {
        "url": "http://URL-TO-DOWNLOAD-ZIP",
        "grade": 1,
        "subject": "Hindi"
    },
    ...
]

Form IDs upload on Hasura

Ensure the tables competencies & workflow_refids_mapping exists on Hasura DB. Migration files exists at path hasura-migrations/
Configure variables HASURA_DUMP_FORMS_MAPPING, HASURA_GRAPHQL_URL & HASURA_ADMIN_SECRET

That's it. Now form IDs mapping will be uploaded on Hasura in the above mentioned tables.

Generated Files

XLSX files will be generated at path: ./gen/xlsx
XML files will be generated at path: ./gen/xml
Image files will be generated at path: ./gen/images

Postman Collection link

https://www.getpostman.com/collections/4b5106be932558e0b96a

quml2xform's People

Contributors

Stargazers

Watchers

Forkers

choxx

quml2xform's Issues

What to do with the default labels?

Referring to the template shared earlier: https://docs.google.com/spreadsheets/d/1GIcHghGYUR1FZ31BD-LCXBNMzr1xdCmQ/edit#gid=1148196142

The intro for the form is currently hard coded along with a few other text/labels. Is that fine? Ref: https://github.com/choxx/quml2xform/blob/78f277a8076fea7354f9b8a7d9e7989a51373c8e/src/quml-to-odk/quml-to-odk.service.ts#L194

Issue in bulk xlsx file upload to ODK

Issue in updating zip XLSX files form ID and bulk upload on ODK because of directories, hidden files, and system-generated files

Rendering inline HTML

Also Image questions need to be handled separately. Same is the case with formatting. ODK allows markdown based formatting. So html to makrdown conversion needs to happen as well.
https://github.com/Sunbird-Ed/program-service/blob/8dfbf3efb65e886933bb0d788dd5281000d1aa07/src/service/print/pdf.js#L209
https://github.com/Sunbird-Ed/program-service/blob/8dfbf3efb65e886933bb0d788dd5281000d1aa07/src/service/print/pdf.js#L274
This method can be directly used.

I tweaked the first part and am now excluding tags from stripping out: ref

For the second, I'm not sure where exactly & how to use this function

Please also elaborate how to handle media-based questions.

UPDATE (2022-08-10): Image-based questions & other issues have been resolved now. Rest pending is the rendering of inline HTML support in Android phones (which is to be clarified with Android dev separately as discussed)

Upload XML to ODK Service

For the last part of the task, I cannot make it work yet. There are several service files being used for the handler: https://github.com/samagra-comms/uci-apis/blob/8d90af5563d551a6dded5078f9878542788a72f6/v2/uci/src/modules/form/form.service.ts#L65 e.g. TelemetryService.

Shall I include all these files/classes/services in the current module & use them as it is? Are any 3rd party dev credentials required for that?

Also, I'm finding it quite tricky to call the uploadForm() function with the file as the first argument. Could you please share any reference for initializing the object from the raw XML file to be passed into this?

Naming convention & path of the generated XSLX & XML forms

Currently, the files are being generated at the configured paths (via .env file) and they are being named with current Unix timestamps. Is that fine?
References:

ODK Central form upload support

Current tool only support uploading the forms & media files to ODK Aggregate.