mainfile for typeahead.js references nonexistent file

[jsdelivr]

mainfile for typeahead.js references nonexistent file

[jimaek]

I am really sorry about that.
Twitter removed typeahead.min.js from their dist :(

We could change mainfile to typeahead.bundle.min.js but it would break older versions so we cant do that.
Or we can leave mainfile the same and create a file named typeahead.min.js with the contents of typeahead.bundle.min.js. Would that work?

[ryanpitts]

I've already changed the reference on our site to directly get typeahead.bundle.min.js, so we're OK either way. That seems like a solution that would work, though.

[jimaek]

I pushed a fix, typeahead.bundle.min.js renamed as typeahead.min.js.
Thank you for reporting this and I am very sorry for the problem.
I will make sure we apply this kind of fixes proactively in the future!

Btw if your website needs any other libraries let me know and we will host them.
If you have any other feedback feel free to ping me.

[tomByrer]

What should be a long-term solution for this please? Likely to happen more as repos age.

• add version boundries to mainfile
• add 'fallback' 2nd file to mainfile (eg if 1st doesn't exist, use 2nd)
• alias pointing to current mainfile?

[jimaek]

A fallback mainfile is an interesting idea. I think it would be the easiest way to fix this issue.
What do you think?

[ryanpitts]

Only drawback is that adding a fallback mainfile creates extra manual intervention each time you update, and if an organization makes changes to a library, they may well end up changing the name of that fallback file at the same time.

[jimaek]

No, the fallback mainfile will be simply a parameter in info.ini. Not an actual file.
When /g/ receives a project name it reads info.ini for the path of the main file.
If that file is not existent I can make it to read the next fallback mainfile parameter instead of showing a 404 error.

This way old versions will work with the first mainfile parameter and the new versions will work with the second parameter.

For example in this case it would be:

author = "twitter"
github = "https://github.com/twitter/typeahead.js"
homepage = "http://twitter.github.com/typeahead.js/"
description = "typeahead.js is a fast and fully-featured autocomplete library"
mainfile = "typeahead.min.js"
mainfile2 = "typeahead.bundle.min.js"

[ryanpitts]

That would be nice!

On Tuesday, April 22, 2014 at 3:11 PM, Dmitriy Akulov wrote:

No, the fallback mainfile will be simply a parameter in info.ini. Not an actual file.
When /g/ receives a project name it reads info.ini for the path of the main file.
If that file is not existent I can make it to read the next fallback mainfile parameter instead of showing a 404 error.
This way old versions will work with the first mainfile parameter and the new versions will work with the second parameter.
For example in this case it would be:
author = "twitter" github = "https://github.com/twitter/typeahead.js" homepage = "http://twitter.github.com/typeahead.js/" description = "typeahead.js is a fast and fully-featured autocomplete library" mainfile = "typeahead.min.js" mainfilef = "typeahead.bundle.min.js"

—
Reply to this email directly or view it on GitHub (#33 (comment)).

[tomByrer]

if an organization makes changes to a library, they may well end up changing the name of that fallback file at the same time

That's the point. Eg the old info.ini:
mainfile=typeahead.min.js
new info.ini should be:
mainfile=typeahead.min.js, typeahead.bundle.min.js
Where hopefully what ever script looks at the info.ini, it would verify if typeahead.min.js exists, IF NOT THEN use typeahead.bundle.min.js I'm not sure if it has to be more complicated that this.

[jimaek]

Failover mainfile is now supported jsdelivr/jsdelivr@dbaf215

[GeorgeErickson]

Couldn't this still lead to serving an different main-file than expected for a old version?

[jimaek]

It reads them in order. At first mainfile and if that is missing then mainfile2.

[pnommensen]

@jimaek What if new versions of a library include both the mainfile and mainfile2 but the default file should be mainfile2?

[jimaek]

Then we change mainfile to the new default file and set the old one to mainfile2. This way old versions wont break

[tomByrer]

Then we change mainfile to the new default file and set the old one to mainfile2. This way old versions wont break

Still can break it the new mainfile had shipped with old versions.
If for example if a lib has a full-featured.js version, & a limited-function.js (browser-only) version. For v1, that lib listed the full-featured version as mainfile. For v2 there were some API changes, so limited-function.js becomes mainfile & the full-featured.js, which still ships, is now mainfile2. But when people still on v1 look for mainfile, they get the wrong version.
An edge-case, but I see the possibility soon & more common years down the road.

Only way to ensure mainfile is the real mainfile for that version is to list in each version array a mainfile key/value. A bit of over-engineering, but can save headaches down the road.

[jimaek]

I don't like the idea of maintaining the version/mainfile values for ever.

Dont forget that people can always set manually the file they need in /g/.

[GeorgeErickson]

But a change like @tomByrer mentions could cause a url like http://cdn.jsdelivr.net/g/[email protected] to return a different file after an update.

[jimaek]

@GeorgeErickson Not sure what you mean.

[GeorgeErickson]

Rev 1

A library (called example) starts out with a layout like this:

1.0.0
  main.combined.js
  main.js
mainfile=main.combined.js

The url //cdn.jsdelivr.net/g/[email protected] would return the file main.combined.js

Rev 2

An update comes in that changes the mainfile:

1.0.0
  main.combined.js
  main.js
2.0.0
  main.combined.js
  main.js
mainfile=main.js

The url //cdn.jsdelivr.net/g/[email protected] now returns a different file main.js, which would break things for anyone who had that url.

[pnommensen]

@GeorgeErickson Yeah, that's basically the inverse of what I posted earlier. The only way I see this working, in all situations, is the option for a mainfile override per version as you mentioned in Gitter.

[tomByrer]

^ exactly.
Now this might not come up that much right now. But with 700,000 JS repos on GitHub, I think in a few short years this will happen more often. Might be a bit over-engineering, but is it better to fix things after or before it breaks?

[jimaek]

Ok I see the problem. But still I don't think that maintaining version/mainfile for each project is a good solution. Updating these values on each version update sounds like a bad idea.

[tomByrer]

Updating these values on each version update sounds like a bad idea.

Seems like both webdesigns use the API. If the API would track per version what the current mainfile is, then updating would be automated. It would prevent a full API JSON/db rebuild from scratch; old db has to be kept & only new updates incrementally added.

[GeorgeErickson]

@jimaek - We could also do a hybrid approach.

If a .mainfile is inside a versions directory, its value overrides the mainfile from the info.ini.

[GeorgeErickson]

Here's an example layout https://github.com/GeorgeErickson/jsdelivr/tree/design_idea/files/isotope

[jimaek]

@GeorgeErickson Yes but then we would have to add this .mainfile to each version. Thats a pain both for us and users that want to update a project.

[pnommensen]

@jimaek No we don't, just for the versions where the mainfile is different than what's specified in the project's info.ini. It wouldn't be many libraries where the structure changed.

[GeorgeErickson]

@pnommensen - exactly 👍

It should also be easy to write a test to notify when a .mainfile is needed.

[GeorgeErickson]

@jimaek - you would only ever need to add a .mainfile (for each old version) if you change the mainfile.

[tomByrer]

you would only ever need to add a .mainfile (for each old version) if you change the mainfile

I think it would be more consistent & easier to use, simply by adding mainfile key:value pair for every version. Hybrid approach seems neat at first, but requires additional logic (eg slower rendering) when consumed.

Really, all I'm/We're asking is a db un-pivot of the mainfile value. Database managers do this all the time.

BTW, this thread should be tagged v2.

[jimaek]

@pnommensen If the author does the change, the change will apply to all future versions. So we would need to include that mainfile.ini to all new versions after the change was done.

[pnommensen]

@jimaek Well not necessarily, we can change the info.ini to reflect the new mainfile for future versions and then add a .mainfile in the directory of the prior/old versions of the library.

[jimaek]

@pnommensen Interesting.
Does everybody agree on this solution?

[tomByrer]

add a .mainfile in the directory of the prior/old versions of the library

If you expect the APIdb to be rebuilt from scratch everytime you run the db-builder, then yes this is best solution.
But I still don't understand why the API-db needs to be rebuilt from scratch every time?

[jimaek]

@bebraw is the one to answer that. He makes the decisions regarding our API.

[GeorgeErickson]

@jimaek - 👍

[bebraw]

Currently I just consume whatever you have at PHP index (later GitHub). If you can describe how that should map to API, I'm fine with that. I would hate having to detect changes to files on API level given that's very brittle way to deal with it.

[jimaek]

Since we plan moving away from PHP I was wondering if this solution would work for you when the API starts pulling its own info via Github.

[bebraw]

Ok. I can go with .mainfile.

[jimaek]

Pushed live mainfile per version jsdelivr/jsdelivr@b966f18
Mainfile2 is now not supported.

[tomByrer]

Can you work with mainfile with no dot @bebraw ?

[bebraw]

@tomByrer Yes. It doesn't make any difference.

Can someone describe how the new scheme should map to the API? What sort of output do you expect?

[tomByrer]

Maybe:
http://api.jsdelivr.com/v1/jsdelivr/libraries?name=jquery =

[
  {
    "name": "jquery",
    "_id": "xkp7FUejl36",
    "created": "2014-05-04T14:01:20.336Z",
    "zip": "jquery.zip",
    "lastversion": "2.1.1",
    "description": "jQuery is....",
    "homepage": "http://jquery.com/",
    "github": "https://github.com/jquery/jquery",
    "author": "jQuery Foundation",
    "assets": [
      {
        "version": "3.0.0-future-beta",
        "mainfile": "jquery.amd.min.js",
        "files": [
          "jquery.amd.js",
          "jquery.amd.min.js",
          "jquery.js",
          "jquery.min.js",
        ]
      },
      {
        "version": "2.1.1",
        "mainfile": "jquery.min.js",
        "files": [
          "jquery.js",
          "jquery.min.js",
          "jquery.min.map"
        ]
      },
      {
        "version": "2.1.0",
        "mainfile": "jquery.min.js",
        "files": [
          "jquery.js",
          "jquery.min.js",
          "jquery.min.map"
        ]
      },
...

Basically, we hope to just pivot mainfile. So if mainfile changes between versions, we won't be caught off guard & serve wrong file.

[bebraw]

@tomByrer Ok. So mainfile will disappear from base level (ie. it's always per version)? In addition mainfile2 will go away as well?

[tomByrer]

Mainfile2 is now not supported.

Not sure if they want to keep mainfile temporarily until they switch to new webdesign. Perhaps this should be v2 of API, & new webdesign use v2?

[MartinKolarik]

As @tomByrer says, removing mainfile from the base level is not a good idea as it can be a breaking change for any 3rd party using the API. I'd just add mainfile for each version and remove base level mainfile with v2.

[jimaek]

Yes mainfile at base should remain and be the parameter in info.ini.
Under each version we can have the mainfile parameter read from the file if it exists.
We can also add the mainfile parameter to all versions that dont have the mainfile file and it would equal to the one in info.ini