jsdelivr/bot

DEPRECATED The jerk of a bot that checks PRs and responds in comments

jsDelivr bot

The bot which validates and comments on PRs, squashs commits and merges submissions.

Why

Couple reasons: we wanted something even faster than @jimaek (he usually took a about a minute whereas the bot is under a second), bigger than @jimaek (checks more things than a human can), better than @jimaek (can squash commits before merging), and stronger than @jimaek (the bot defers to other mods for help if unsure) =D

Why not travis/jenkins etc

Could of be done, but it would have been much, much slower to validate and be difficult to support issue comment hooks (actions when a mod wants a pr squashed) and auto merging. The nice thing about bot is it only interacts with the Github API and avoids cloning the repo most of the time.

How

Born out of a drunk weekend adventure to replace @jimaek with the github API, github3.py, and the jsDelivr API

Usage

Note: squashing requires git-extras

pip install -r requirements.txt

Adjust config.yml as necessary (tokens, user name, repo clone path, etc)

Run server on port 9000

./run.py

#Background
nohup ./run.py &

./cli <pull-request-number>

bot's Issues

error on submitting bullreset.css

.

Fail for update.json without info.ini

jsdelivr/jsdelivr#6158

Delete branch whenever a pull request is closed/merged

Delete a branch from @jsdelivrbot whenever one of their pull requests is closed or merged.

AMD: Require explicit definitions

Discussed in irc briefly, warn when a define([name], [deps], fn) doesn't provide name

jsdelivr/jsdelivr#779

Stop complaining about unescaped UTF-8

The whole point of a global CDN like jsdelivr is that it allows anyone around the world to load content quickly and conveneintly. This feature will attract projects from all around the world. Some of those projects might come with translations in multiple languages. Others might be designed to cater to users in some locales but not others. Both of these involve putting non-English strings in source files. And yet the bot complains about it.

Should we try to please the bot by replacing all of our non-English strings with backslash-escaped hex codes? Those things are impossible for humans to read.

UTF-8 is the de facto standard of internationalization on the web. Anything encoded in valid UTF-8 should be kosher.

Add Exception for @2x suffix for image files.

It would be good if the bot doesn't consider @2x as the illegal character as it's a suffix for Retina Optimized Images and it's a requirement for the browsers to automatically pull the optimized version of the images.

Here's one PR which had these type of images and the bot thought it's an issue. Thanks!

detect minified files

ref
Phase 1 Check sourceMappingURL
/(\/\*+[\s\S]*?sourceMappingURL\s*=[\s\S]*?\*\/)|(\/\/.*?sourceMappingURL\s*=.*?[\r\n])/g
If there is a .map file pointed to, then likely this is a minified file.
Other tests should be skipped if true.

Phase 2: remove comments (where possible without trashing code)
This RegEx test should work: /(^(\/\*+[\s\S]*?\*\/)|(\/\*+.*\*\/)|^\/\/.*?[\r\n])[\r\n]*/gm on this test code:

Match full line & blank newlines after: 
//! https://github.com/tomByrer/web-engeneering-notes/blob/master/banner-tips.md#mini-banner

hunk
/**
* usage: https://github.com/jsdelivr/bot/issues/1#issuecomment-42943418
**/
junk
/*! script.node.js github.com/owner/scriptjs 0.7.0 (c)2014 License GPL */
funk
End line comment=! /*** I like stars ***/

left/*middle*/right
trunk
// project-legacy.js github.com/owner/project 0.1.0 (c)2014 @licence MIT
stunk
/* @preserve this comment please? */
sunk
// @License to @Preserve: none
bunk
/* important info but not enough to keep in minified code */
punk
// comment
edge-case (works if have space between **)
/**/
space
/*
*/

Unsure what to do:
//# sourceMappingURL=jquery.mobile-1.4.2.min.map
var x=y;// Great coding!  Have to keep this comment because of above example...
var twitterHandle = '@preserve';  //oops

Should Fail:
https://user:[email protected]/is-url.js
/!escape
<!-- remove this comment -->
"/*"

True; EOF test:
/* test */

Bonus: grabs newlines after comment.

Phase 3

If ( comments-removed.match(/\n\r/g).length > ( comments-removed.length / 80 ) {
  not mified 
}

Respecting pre-release releases

Currently doens't sort pre-release versions correctly jsdelivr/jsdelivr#3867

No info.ini buddy error

I believe this because the API didnt update yet? jsdelivr/jsdelivr#3575

Can we fix it or make the error cleaner? Some users were confused and thought they made some mistake

A modular library issue

The AniJS library it's composed by a Core and others modules which We call helpers, like scrollReveal, and now DOM. Please we need also add these important modules.

Please take a look to jsdelivr/jsdelivr#1631

Best Regards, Dariel

UTF-8 encoded files seems be interpreted as binary

After forcing the usage of UTF-8 format in all JavaScript files I get messages like in the pool request:

Expected js/i18n/grid.locale-ja.js to be static content; found binary
Expected js/i18n/grid.locale-ja.min.js to be static content; found binary
Expected js/i18n/grid.locale-ja.min.map to be static content; found binary

and the merging of the new version will be done many days later.

I suppose that it's a bug in jsdelivrbot, which seems to be incompatible with the --charset "UTF-8" option of closureCompiler. It seems to have problems even with the source JavaScript files in UTF-8 format.

If it's not a bug, then I would like to know what can I do to prevent stopping of automatic pull request by jsdelivrbot.

Thanks in advance.

Bot thinks *.min.js is a binary file

See jsdelivr/jsdelivr#737. That file is definitely a text file.

basePath is optional

No need to show a warning. Lets discuss it if you want.

jsdelivr/jsdelivr#3429

Incorrect flagging?

I got a message from the jsdeliverbot, but I'm not really sure why.

jsdelivr/jsdelivr#744

add feature to ignore old directory/file structure before certain version

Because the structure may be changed, we can't control or make sure the structure will be the same forever, once it changes, e.g. rename javascript to js, CI bot will not success and fail in all the after version, but that doesn't make sense, we still need a human begin to check the CI message, I think we should support this feature to ignore the old structure before a certain version, so it won't make all the after version not pass CI bot, and the version may be set in info.ini

woff2 is acceptable

We can whitelist the file format jsdelivr/jsdelivr#3592

For new project include stats on first comment

Ease collab work with some links and stats for references repos:

• Direct link to repo
• Versions
• Age of repo (day of first commit)
• Stars + watchers
• ???

Failed to warn about deleted files

jsdelivr/jsdelivr#10695

hostname doesn't match fastly.net

I just submitted a PR for my human-date library and got this error:

Looks like the info.ini has a few problems:
    hostname 'www.npmjs.com' doesn't match either of 'a.sni.fastly.net', 'a.sni.global-ssl.fastly.net'

I didn't see anything about fastly anyhwere in the CONTRIBUTING section or here in existing issues. Please advise on next course of action.

JSON incorrectly passing

The bot considers this as correct

{
  "packageManager": "github",
  "name": "jdenticon",
  "repo": "dmester/jdenticon",
  "files": {
    "basePath": "dist/",
    "exclude": "jdenticon.js"
  }
}

while its not and libgrabber will fail.
The correct is "exclude": ["file"]

Merge a pull request with squash option

@jimaek want the bot to do a squash merge commit when doing auto merging of pull requests?

https://developer.github.com/v3/pulls/#merge-a-pull-request-merge-button

The bot doesn't seem to work for the lipis/bootstrap-social

I realised about the existence of the project when after pushing a new release for the lipis/flag-icon-css it was generating a new PR: https://github.com/jsdelivr/jsdelivr/pulls?utf8=%E2%9C%93&q=is%3Apr+flag-icon-css+

But it doesn't seem to do anything when I'm pushing to lipis/bootstrap-social. Is there anything I have to do so the bot will start picking up on this project as well?

Scan files for malicious code

As mentioned in the Gitter chat room...

Here are some ideas to start with:

• Scan for domains listed on blacklists (such as this, this, this, this, this or just Google it)
• Complain when it detects obfuscated code (eval(function(p,a,c,k,e,r){...}), eval(function(p,a,c,k,e,d){...}), eval(function(w,h,a,t,e,v,e,r){...}) or even just eval(function(...){...}))
• Run code through this service

Too restrictive homepage url validation for info.ini

Creating a PR for adding files of version 2.0.5 of the bigscreen is rejected because the already existing
info.ini file uses http://brad.is/coding/BigScreen/ as homepage which is reported as invalid because it does not fit to the expected hostname schema.

Error response from bot:

Looks like the info.ini or update.json has some problems hostname 'brad.is' doesn't match either of 'www.ampersandnbspsemicolon.com', 'ampersandnbspsemicolon.com'

Validate repo and files in update.json

Validate repo exists, mainfile exists and files match for the last tag when provided an update.json

Example issue jsdelivr/jsdelivr#5313

Ideas for tests

• files/$projectName should contain only 1 file. info.ini
• mainfile parameter should be correct. The file specified should actually exists.

I will add more later as more PRs come in and I see the mistakes :)

Better error when wrong file name

A user added info.txt instead of info.ini and he got this error

Might be nothing but I have some concerns about the files in your submission...

    string indices must be integers, not str

We need to add a better error explaining better what is wrong.

Link to this repo

Add a small note in the bot's comments linking to this repo to allow people to report bugs.

Bot failing with SSL error

From jsdelivr/jsdelivr#10085

[Errno 1] _ssl.c:510: error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure

Can't find website?

Clearly it works: https://gist.github.com/jsdelivrbot/f23a66e08d3c46e5e06b

Validate json

It would help a lot if the bot could validate update.json and comment any errors it finds.

Allow the bot to merge libgrabber's commits

If its possible it would help a lot to add a parameter to update.json.
Something like "auto-merge": true.
It can be enabled per project and if the bot sees it then it will automatically merge the PR for that project without waiting for a mod.

info.ini error where info.ini is present

The bot says "viewability has no info.ini file buddy"

jsdelivr/jsdelivr#5068

Any idea what's wrong?

Bot complains about missing mainfile from previous versions

jsdelivr/jsdelivr#773

The missing file is from a previous version (the initial version, actually) and has nothing to do with the commit in the PR.

Ensure info.ini exists for project

don't show tuple index out of range

Unable to update package because of case sensitivity?

jsdelivr/jsdelivr#8511

There have been significant changes in ScrollReveal v3.0.0, and the desired filename is now all lowercase. I have updated my PR to reflect these changes, but I am still encountering errors.

I figured I’d leave the camelCase v2.3.2 filename for legacy support, but perhaps its causing a problem?

Please advise!

I can't update my file

Hi,

There is an issue with my pull request (jsdelivr/jsdelivr#1002) : it's saying I cannot update a file I've uploaded. However there is an issue with that file (JS compressor that didn't do correctly its job) and I need to fix it.

Can you help please?

Thanks

Support new mainfile format

Support .mainfile to set a specific version mainfile

If github supplied check tag exists

Check https://api.github.com/repos/:owner/:repo/tags to ensure referenced version exists
http://github3py.readthedocs.org/en/latest/repos.html#github3.repos.tag.RepoTag
http://github3py.readthedocs.org/en/latest/repos.html#github3.repos.repo.Repository.iter_tags

Trying to figure out what this means and what to do about ti

First time I have tried to submit and the bot reported this issue 1044

Might be nothing but I have some concerns about the files in your submission...
 Unescaped unicode characters in angular-all.min.js (at pos 41664) 
 Unescaped unicode characters in angular-all.min.js (at pos 42871)

This is an uglified file but I don't see anything suspicious at that location etc.

I have looked at a few other people that had the same message and can't see anything wrong with their submits either.

What's the genesis of this issue and what should I do about it?

Clarify config

Hi there,

I wanna confirm that the auth_token and post_secret is secret of what?
https://github.com/jsdelivr/bot/blob/master/config.yml#L5-L6

I guest auth_token is for GitHub auth, what about post_secret? For Gist only? Thanks.

Possible incorrect flagging of minimized JS file

Not sure if this is an issue/bug or not, but the bot flagged a file in this PR for angular-dragdrop.min.js as not minimized, but it definitely is.

There are perhaps excessive comments in the top of the file (seems like licensing mainly) but the JS code itself is definitely minimized.

Bot failes very often

Lately a lot of PRs are not checked at all by the bot. I think it happens when multiple PRs arrive at the same time but it could be something else completely.

Either @megawac or @MartinKolarik please check because its very important. We heavily rely on this bot to merge

Long forgotten main files

What can we do to make these errors disappear jsdelivr/jsdelivr#3601 ? We created a custom mainfile a long time ago so there is no point showing these errors for ever

adding new min.js file leads to a reject

Hi,

just added a new version of piwik JS to jsdelivr repo and got an reject from the bot, complaining about changing the structure.

But if there was no min.js file, how should one add it without changing the structure?

Comment commands

It would be great if the bot could react to commands in comments.
For example:

jsDelivr mod: @jsdelivrbot: squash commits and all commits in that PR will get squashed into a single one

Scenario: The bot created an update PR for a project. All mods are asleep.
The developer of the project that is getting updated can comment:
Author: @jsdelivrbot: merge this PR

Letter Case

Are we going to enforce Letter Case? For example, jsdelivr/jsdelivr#782. The original file is called sigma.layout.forceAtlas2.min.js but the bot would prefer sigma.layout.forceatlas2.min.js. I'd probably prefer to stick with the original naming of the files, but what do y'll think?

Warn on files being deleted

Marks js files as binary

Example jsdelivr/jsdelivr#1182
But it happens often

2 commit rule

We can create a warning when the user has more than 2 commits and then show him the correct command to squash his commits.

2 because = info.ini + update.json to allow users to use the web interface