decoded-cipher/mgu-result-scraper

A project designed to automate the process of fetching result screenshots from the MGU result portal, sending them to students via email as secure PDFs, and generating consolidated marklists and result analysis for teachers.

MGU Result Scraper

This is a simple project to get the result screenshots of a list of students from Mahatma Gandhi University result portal.

🌟 Introduction

First of all, Mahatma Gandhi University doen't have a fixed official URL for accessing all results. They keep on changing as the results get published. Two prominant links (active as of now) for accessing results are listed down below:

The accuracy of these links are questionable as time passes. But the element heirarchy within result pages remains the same always.

• UG Exam Results
• PG Exam Results

🌟 Script in Action (Demo)

🌟 Installation

1. Clone this repository

git clone https://github.com/decoded-cipher/mgu-result-bot.git

2. Install NPM packages

npm install

3. Update the data.json file.

Update the file with Permenent Registration Numbers (PRN) and the Exam ID. Exam ID (option value) can be obtained by inspecting the Examination <select> tag on result portal.

{
    "exam_id" : "58",
    "prn" : [
        "203242210987",
        " ... ",
        "203242211046"
    ]
}

🌟 Run Project

This project can be made run by executing the following command via terminal.

npm start

mgu-result-scraper's Issues

Make project Python-free

We need to find an alternative to the Python child processes, that are used to add borders to graph images and to print as PDFs. The Python dependencies are causing issues while trying to run on other computers due to version issues and lack of the Python dev environment.

Common minor fixes to all result docs

• Add appropriate headings and subheadings to all result sheets to transform them into a complete document.
• There's a typo in the header - college name. Fix that too

Discrepancy in Subject List / Names / Count

Regarding semesters offering open-courses or elective papers, each student's Subject list would differ. This happens mostly on UG Streams.
Here's such an example in which, Malayalam, Hindi, or French could be opted as language paper.

OBE Sheet : Warning issue

Sometimes, the OBE Sheet shows warning-like issues, once we open it. It tries to retrieve most of the data and overwrite the already saved version. This issue needs to be addressed, as it needs human intervention to complete the process.

Feature plans for Version 4.0 (v4.0)

Build a web interface for the users to interact with the system

• Add a terminal to showcase the logs of ongoing processes using Xterm.js
• Add a process indicator/visualizer using D3.js in the form of a flowchart would elevate the UX to a whole new level
• There must be multiple ways to feed the Student PRNs into the system.
  • Fields to enter start_PRN and end_PRN, so that the system could iterate all along
  • Provision to upload XLSX/CSV Sheets with Email Addresses
• Uploading XLSX/CSV should also trigger the generatePDF and sendEmail functionality by default.
  Use File Pond for uploading files, just because it is aesthetically pleasing.

Sem details is not added for ug

Sem details of ug is not added to DB in this project

New parameter added on PG Result page

A new " Course " parameter has been added to the PG Results page. This causes the PG results, not to be accessed directly.

Discrepancy on Subject-wise Grade Distribution Table

The discrepancy on the "Overall Grade Distribution" table on the "Result Analysis Sheet" is due to the missing Grade cell on the result. But this also happens on the "Subject-wise Grade Distribution", which is of no clue at all...

Add Declaimer towards the bottom of all results

Disclaimer:

This MGU Result Scrapper project is currently in an active development phase. We do not assert the absolute accuracy of all compiled results. Users are advised to use this tool for obtaining an overall idea rather than relying solely on its outcomes. In the event of any discrepancies discovered among the compiled results, we encourage users to promptly report them for further investigation. Your feedback is crucial in enhancing the accuracy and reliability of this project.

NaN among ESA & ISA

Among the ESA and ISA values, there happened to be a row with a NaN value. This is also causing issues on OBE Datasheet. This wasn't in the picture till data, as we weren't using these values before.

Result Graph PDF is a total mess now

The page-wise alignment of the graphs within the Result Graph sheet is broken. It also makes the graph images stretch vertically, making it aesthetically non-pleasing and useless.

Bug in generating analytics

When I try to run the program the analytics is not working properly all other function is done correctly and all the data is inserted into DB

Error thrown for B.Com results - Data Normalization

No more details are available, as of now.

Feature plans for Version 5.0 (v5.0)

Here, we'll be focusing more on deriving better Analytics and Inferences from the data.

• Side-by-side comparison with previous results. Feel free to get inspired by Fi Money App spending pattern inferences
• Embed the system into our Unified Digital Platform Interface so that anyone with proper credentials can use this system on the cloud.
• Build a secure API interface so that colleges can integrate this into their LMS, probably into Student's Profiles

Handle Missing Data

While fetching the data, there happened to have multiple results with partial data

• Screenshots & HTML with half the table rows
• Missing Graded & Grade Points (We must calculate the missing Overall & Subject Grades, if the student has cleared all subjects)

Multi-page Screenshot PDF

Generate a PDF with all the result screenshots together as a multi-page PDF (need not be a secure one) to be sent out to teachers, via Email.

Incorrect PDF Metadata (especially, Document Title)

• Document Titles are the same as "Consolidated Marklist", for every PDF.
• Metadata for the same is also incorrect