vishesh-baghel/glide

You can see how Glidee-bot works by installing it from here

glide's Issues

Hit the heap memory limit

INFO (probot): Total 37602 files fetched from repository: vishesh-baghel/Chat-app for installation id: 45484616
INFO (probot): Received Total 37602 files for processing from repository: vishesh-baghel/Chat-app with installation id: 45484616

<--- Last few GCs --->

[109427:0x5b03790] 116166 ms: Scavenge 1967.3 (2068.1) -> 1966.9 (2078.6) MB, 4.93 / 0.00 ms (average mu = 0.882, current mu = 0.440) allocation failure;
[109427:0x5b03790] 116175 ms: Scavenge 1973.7 (2078.6) -> 1972.7 (2079.6) MB, 5.18 / 0.00 ms (average mu = 0.882, current mu = 0.440) allocation failure;
[109427:0x5b03790] 116184 ms: Scavenge 1974.6 (2079.6) -> 1973.8 (2101.4) MB, 8.34 / 0.00 ms (average mu = 0.882, current mu = 0.440) allocation failure;

<--- JS stacktrace --->

FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory
1: 0xc9e850 node::Abort() [/home/vishesh.baghel/Documents/workspace/personal-projects/glide/node_modules/node/bin/node]
2: 0xb720ff [/home/vishesh.baghel/Documents/workspace/personal-projects/glide/node_modules/node/bin/node]
3: 0xec1a70 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, v8::OOMDetails const&) [/home/vishesh.baghel/Documents/workspace/personal-projects/glide/node_modules/node/bin/node]

Add pull request event listener to make a comment on every pull request

On adding this event listener, the bot will follow the below-given steps:

• From the event payload, the bot will get the installation ID and the affected file paths.
• The bot will query the DB to get the risk scores using query parameters such as filePath, repoName, owner, etc.
• Then, the bot will construct a good-looking comment to create on the PR
• After commenting, the bot will get to the terminal stage of its lifecycle.

risk score is going beyond 1

The score is not bounded between 0 and 1. check the algorithm again as well as the logistics function

Add a collection in db to store all the webhook events coming from the github.

This is important to make the app more reliable in case the processing fails in between a job

Risk score optimisation

Feature: risk scores would be more refined: The bot will follow these steps after every new installation

It needs to prepare all the files with associated risk scores beforehand to react to PR events.
To do so, it needs to fetch all repositories on which the bot has been installed.
Then, iterate on each repository and get all files.
For each file, first calculate the risk score and then save it.
Now, the bot can give the files with their respective scores in every PR.

Add AI to enhance the comment generation

Write hashnode article and github readme for the project

Tasks:

• research about the writing techniques and best practices
• write article
• proof read it
• enhance it using chat gpt
• add github readme

Add a scheduler to fetch and update all the predicted risk scores in the 'File' model

This scheduler will be responsible for all the retraining of the predictor model. This scheduler will run every 10 minutes to update any residual job in the DB.
The jobs collection schema would be like this:
{
installationId,
owner,
repoName,
filePath,
status: completed, not-completed.
}

Every 10 minutes, the scheduler will query the DB with the jobs with the status as not-completed. It will fetch the mindsDB with parameters given in the schema to get the unique filePath predicted score and then finally it will update the predicted risk score in the File model.

After a file is renamed, the old named file is not deleted from the DB.

Add an handler for this scenario in the pull request closed listener to delete the old files which are renamed

smee client stops receiving events from github

check this solution: probot/smee-client#179

secretOrPrivateKey must be an asymmetric key when using RS256

The bot is not processing the events after receiving them from the github

After investigating this issue. I found this behavior:

• The bot processes the events after starting the app but stops it after 5 minutes.
• Any event sent after 5 minutes, is received by the app successfully and a 200 post log is printed but the further event processing doesn't happen.

Take owner name from repository instead of installation object in the response payload while processing repositories

Batch query on mindsDB is returning the same set of files again and again

Currently on executing a batch query on mindsDB with only installationId as a parameter is returning the same set of files which is expected behaviour from the DB point of view.

But, the bot is overriding the updated predicted scores on the files which is unexpected behaviour. Their should be something to identify in the mindsDB table to make sure the bot gets new rows.

Check if pagination feature is present in mindsdb javascript sdk to solve this issue

Handle errors when the app tries to fetch details from a private repository

This issue was first discovered from the logs tracks on sentry. It is not a blocker for the workflow but still needs to be properly handled

add post installation page

Add comments on the important functions to let the other contibutors know about the implementation details

Add database and save details for every installation

Tasks:

• listen to installation events to get the data
• Add mongoDb to the app and make the required models for saving the installation details

Delete files from the db when file status is 'deleted' in the files fetched after receiving the pull request closed event

setup mindsdb in local environment

Not able to calculate risk score due to error while sorting the commit list

WARN (probot): Commit date not specified
ERROR (probot): a.date.getTime is not a function
TypeError: a.date.getTime is not a function
at /home/vishesh.baghel/Documents/workspace/personal-projects/glide/lib/services/riskScoreService.js:7:54
at Array.sort ()
at calculateRiskScore (/home/vishesh.baghel/Documents/workspace/personal-projects/glide/lib/services/riskScoreService.js:7:17)
at /home/vishesh.baghel/Documents/workspace/personal-projects/glide/lib/services/repositoryService.js:129:98
at Array.map ()
at /home/vishesh.baghel/Documents/workspace/personal-projects/glide/lib/services/repositoryService.js:121:60
at step (/home/vishesh.baghel/Documents/workspace/personal-projects/glide/lib/services/repositoryService.js:33:23)
at Object.next (/home/vishesh.baghel/Documents/workspace/personal-projects/glide/lib/services/repositoryService.js:14:53)
at fulfilled (/home/vishesh.baghel/Documents/workspace/personal-projects/glide/lib/services/repositoryService.js:5:58)

Not able to find the files on the tree

ERROR (probot): Error while fetching resource details for resource url: GET /repos/{owner}/{repo}/git/trees/{tree_sha}
ERROR (probot): Error occurred while fetching all files for vishesh-baghel/codespace-demo and installation id: 45484616
ERROR (HttpError): Not Found
HttpError: Not Found
at /home/vishesh.baghel/Documents/workspace/personal-projects/glide/node_modules/octokit/node_modules/@octokit/request/dist-node/index.js:124:21
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async sendRequestWithRetries (/home/vishesh.baghel/Documents/workspace/personal-projects/glide/node_modules/@octokit/auth-app/dist-node/index.js:398:12)
at async requestWithGraphqlErrorHandling (/home/vishesh.baghel/Documents/workspace/personal-projects/glide/node_modules/octokit/node_modules/@octokit/plugin-retry/dist-node/index.js:71:20)
at async Job.doExecute (/home/vishesh.baghel/Documents/workspace/personal-projects/glide/node_modules/bottleneck/light.js:405:18)
status: 404
request: {
method: "GET"
url: "https://api.github.com/repos/vishesh-baghel/codespace-demo/git/trees/main?recursive=true"
headers: {
"accept": "application/vnd.github.v3+json",
"user-agent": "octokit.js/3.1.2 octokit-core.js/5.0.2 Node.js/20.10.0 (linux; x64)",
"authorization": "token [REDACTED]"
}
request: {}
}

Connect mongodb to mindsdb

Show only top 10 files, which require more attention when a PR is opened

Add performance monitoring in the prod and dev environment apps

Add branch and file sha to the 'File' model

Add a new data model for training the predictor model

This is needed because the native query in the mindsdb editor is not working as expected. I am continuously getting errors while training the model. Fortunately I understood the reason why I was getting the errors.

I was trying to run the MongoDB native query as db.collection.find({}, {commits: 0}) but the commits array was not being excluded. I finally decided to keep the query as db.collection.find({}) and to make a new model which doesn't have the commits array.

The model's training completed successfully. I think the mindsdb's lightwood ai model doesn't process the dictionaries or arrays as a datatype for a column while training a regression model.

Add 'issue.closed' webhook event listener to persist them for risk score calculation for ongoing repository history maintainance

Add github app details in the app settings

Tasks:

• Add app description
• Add home page url
• Add OAuth(optional)
• Add post installation destination
• Add logo
• List the app on the marketplace

Make demo video

Tasks:

• record the video
• edit it
• add the video on the github readme

Deploy mindsdb on cloud

show risk scores and predicted scores without slicing in a demo comment along with the real comment

Add installation_repositories.added listener to process newly added or removed repositories

It is important to handle removed repositories because it may happen that the bot won't be able to fetch the details of the removed repository.
use response.repositories_added to get the list. same for removed repos.
`
"repositories_added": [
{
"id": 732309770,
"node_id": "R_kgDOK6YpCg",
"name": "lombok",
"full_name": "vishesh-baghel/lombok",
"private": false
}
],
"repositories_removed": [

],
`

create a trigger in mindsdb to retrain the model every time data is updated after receiving pull request closed event

ref: https://docs.mindsdb.com/sql/create/trigger

Add risk-score calculation module

Add the logic to update the risk scores stored in the DB

Add 'fromExponential' to convert all the predicted scores from scientific notation to number

Currently, if the predicted scores stored in the db are shown in the PR comment, then it will show the first two decimals places. for eg-
if this is stored in db: 1.3621763272908538e-8, then 1.36 would be shown in the comment which is wrong

Github api is tighly coupled with the app which makes it brittle to extenal changes

Make a normalized interface for the app and map the external apis to the central interface of the app

Add langchain api

Todo:

• remove demo only comment generation code
• turn off scheduled jobs for now
• turn off predicted score for now
• add basic langchain api integration

Add one more function in netlify that will be used for uat testing