anutron/mootools-depender

A server side dependency builder for MooTools scirpts

mootools-depender's Issues

Depender URLs can go far beyond the GET limit

IE8 has a hard limit of 2,083 characters allowed in a url.
Cf. http://support.microsoft.com/kb/208427

We use Depender at work and recently discovered this error.
Running IE8 normally this issue did not happen. But Running IE8 with either Selenium or dynaTrace Ajax surfaced the issue.
None of our customers have complained afaik, but I have no way of knowing if this issue is happening in the wild or not.

Here's an example of the kind of URL that is causing the issue:

/depender/build?client=true&require=help/Help&exclude=Core/$util,Core/Native,Core/Array.each,Core/Array,Core/String,Core/Function,Core/Number,Core/Hash.base,Core/Hash,Core/Class,Core/Class.Extras,Core/Window,Core/Document,Core/Element,Core/Event,Core/Element.Event,depender_client/Depender.Client,Core/Options,Core/Cookie,Core/DomReady,Core/Element.Dimensions,Core/Element.Style,Core/Chain,Core/Events,Core/Fx,Core/Fx.CSS,Core/Fx.Tween,Core/Browser,Core/Request,Core/Request.HTML,Core/Selectors,Core/MooTools,More/MooTools.More,More/Element.Delegation,More/Class.refactor,More/Element.Measure,More/Class.Occlude,More/HtmlTable,More/Drag,More/HtmlTable.Resize,More/Class.Binds,More/Element.Position,More/IframeShim,More/Mask,More/Spinner,More/String.QueryString,More/URI,clientcide/dbug,clientcide/Clientcide,clientcide/Class.ToElement,More/Element.Shortcuts,More/Element.Pin,clientcide/StyleWriter,clientcide/StickyWin,More/String.Extras,clientcide/StickyWin.UI,clientcide/StickyWin.UI.Pointy,clientcide/StickyWin.PointyTip,Widgets/UI,Slick/Slick.Parser,Widgets/UI.Sheet,Widgets/UI.Widget,Color/Color,More/Table,ART/ART,ART/ART.Element,ART/ART.Container,ART/ART.Path,ART/ART.VML,ART/ART.SVG,ART/ART.Base,Widgets/ART.Widget,More/Log,More/Keyboard,Widgets/Stacker,Touch/Touch,Widgets/ART.Keyboard,Widgets/ART.Popup,More/Elements.From,More/Fx.Scroll,Core/Fx.Morph,More/Fx.Reveal,clientcide/Collapsible,Widgets/ART.Sheet,ART/ART.Shape,ART/ART.Rectangle,Widgets/ART.Box,Widgets/Press,ART/ART.Font,ART/Moderna,Widgets/ART.Button,ART/Moderna.Bold,Widgets/ART.Glyphs,Widgets/ART.Window,Widgets/ART.Alerts,Core/JSON,Behavior/Element.Data,Behavior/Behavior,Widgets/Behavior,Widgets/Element.Data,Widgets/Behavior.ArtButton,Widgets/ART.Input,Widgets/ART.Menu,Widgets/ART.Search,Widgets/ART.Check,Widgets/ART.Radio,Widgets/Behavior.ArtInput,More/Fx.Elements,Widgets/ART.SplitView,Widgets/Behavior.SplitView,More/Fx.Accordion,More-Behaviors/Behavior.FxAccordion,More/Form.Request,More-Behaviors/Behavior.FormRequest,More/Lang,More/Date.English.US,More/Date,More/Element.Forms,More/Form.Validator.English,More/Form.Validator,More/Form.Validator.Inline,More-Behaviors/Behavior.FormValidator,More/HtmlTable.Sort,More/HtmlTable.Zebra,More/Keyboard.Extras,More/HtmlTable.Select,More/HtmlTable.Tree,More-Behaviors/Behavior.HtmlTable,More/OverText,More-Behaviors/Behavior.OverText,More-Behaviors/Behavior.Resizable,More/Drag.Move,More/Sortables,More/Scroller,More-Behaviors/Behavior.Sortable,clientcide/Autocompleter.Observer,clientcide/Autocompleter,clientcide/Autocompleter.Local,Core/Request.JSON,clientcide/Autocompleter.Remote,JFrame/Behavior.Autocomplete,JFrame/Behavior.CollapsingElements,JFrame/ContextMenu,JFrame/Behavior.ContextMenu,JFrame/Behavior.DataGroupToggle,JFrame/Behavior.FilterInput,FitText/FitText,JFrame/Behavior.FitText,JFrame/Behavior.FlashMessage,Widgets/Behavior.HtmlTable,JFrame/Behavior.HtmlTableCheckSelected,JFrame/Behavior.HtmlTableChromeHack,JFrame/Event.Mock,JFrame/Behavior.HtmlTableClickSelected,JFrame/Behavior.HtmlTableKeyboard,JFrame/Behavior.HtmlTableLiveTreeKeyboard,JFrame/Behavior.HtmlTableMultiSelectMenu,JFrame/Behavior.HtmlTableRestore,JFrame/Behavior.HtmlTableUpdate,JFrame/Behavior.InputFilterHtmlTableZebra,JFrame/MultiChecks,JFrame/Behavior.MultiChecks,clientcide/PostEditor,JFrame/Behavior.PostEditor,JFrame/Behavior.SelectWithOther,JFrame/ART.SideBySideSelect,JFrame/Behavior.SideBySideSelect,JFrame/Behavior.SizeTo,JFrame/Behavior.SplitViewPostFold,JFrame/Behavior.SplitViewScroller,JFrame/Behavior.SubmitOnChange,clientcide/TabSwapper,JFrame/Behavior.Tabs,More/Tips,clientcide/Tips.Pointy,JFrame/Behavior.Tips,More/Date.Extras,JFrame/Behavior.Timer,Core/Fx.Transitions,JFrame/FlashMessage,JFrame/JFrame.Keys,JFrame/MooTools.Config,JFrame/JFrame,More/Class.Refactor,JFrame/JFrame.Error,JFrame/JFrame.Request,JFrame/JFrame.Alert,Behavior/DashSelectors,JFrame/PartialUpdate,JFrame/JFrame.PartialRefresh,JFrame/JFrame.Prompt,JFrame/JFrame.AutoRefresh,JFrame/JFrame.DoubleClickDelegate,JFrame/JFrame.FormRequest,JFrame/JFrame.AjaxLoad,JFrame/JFrame.CheckAllOrNone,JFrame/JFrame.ConfirmAndPost,JFrame/JFrame.FakeRefresh,JFrame/JFrame.HideParent,JFrame/JFrame.LivePath,JFrame/JFrame.Nav,JFrame/JFrame.NoScroll,JFrame/JFrame.PromptAndPost,JFrame/JFrame.Refresh,JFrame/JFrame.RefreshWithParams,JFrame/JFrame.SubmitLink,JFrame/JFrame.SplitViewLinkers,JFrame/JFrame.Container,Widgets/ART.History,Widgets/ART.Browser,Widgets/ART.SolidWindow,JFrame/JFrame.ToggleHistory,JFrame/Events.Relay,JFrame/JFrame.Browser,hue-shared/Hue,hue-shared/Hue.User,hue-shared/Hue.Dock,hue-shared/Hue.Desktop,hue-shared/Hue.Desktop.BackgroundManager,clientcide/StickyWin.Drag,hue-shared/Hue.Desktop.Config,hue-shared/Hue.JFrame.Target,hue-shared/StickyWin.UI.Solid,hue-shared/Hue.SolidWindow,More/Fx.Move,hue-shared/ThumbTack,hue-shared/Fx.Shake,hue-shared/Hue.Login,hue-shared/Hue.Profiler,hue-shared/Hue.Request,hue-shared/CCS.JFrame,hue-shared/Hue.JFrame,hue-shared/CCS.JFrame.DeprecatedFilters,hue-shared/CCS.JBrowser,hue-shared/CCS.Desktop.FlashMessage,hue-shared/DesktopLoader

Note that it is 5,115 characters long (with the full path that I removed here). That is more than double the hard URL limit in IE.

pep-8

Code at the moment is not pep-8 compatible. E.g. it uses 2 spaces instead of 4.

Repository failure

Here's what I get when trying to update cloned copy of the repository after first download.

git submodule update --init
Submodule 'libs/clientcide' (git://github.com/anutron/clientcide.git) registered for path 'libs/clientcide'
Submodule 'libs/core' (git://github.com/mootools/mootools-core.git) registered for path 'libs/core'
Submodule 'libs/more' (git://github.com/mootools/mootools-more.git) registered for path 'libs/more'
Cloning into 'libs/clientcide'...
remote: Counting objects: 3299, done.
remote: Compressing objects: 100% (1281/1281), done.
remote: Total 3299 (delta 2028), reused 3145 (delta 1930)
Receiving objects: 100% (3299/3299), 2.03 MiB | 440 KiB/s, done.
Resolving deltas: 100% (2028/2028), done.
Submodule path 'libs/clientcide': checked out '3af5341e2ff994b7023d012963e199278060753c'
Cloning into 'libs/core'...
remote: Counting objects: 23158, done.
remote: Compressing objects: 100% (7380/7380), done.
remote: Total 23158 (delta 15785), reused 22365 (delta 15139)
Receiving objects: 100% (23158/23158), 3.62 MiB | 313 KiB/s, done.
Resolving deltas: 100% (15785/15785), done.
fatal: reference is not a tree: 7a8ddc74333aabd0d696ece842e9534449f7b80d
Cloning into 'libs/more'...
remote: Counting objects: 19088, done.
remote: Compressing objects: 100% (5348/5348), done.
remote: Total 19088 (delta 14373), reused 17990 (delta 13545)
Receiving objects: 100% (19088/19088), 2.89 MiB | 462 KiB/s, done.
Resolving deltas: 100% (14373/14373), done.
Submodule path 'libs/more': checked out 'dc90ba75705728df385b9e6672767fc1d59cead2'
Unable to checkout '7a8ddc74333aabd0d696ece842e9534449f7b80d' in submodule path 'libs/core'

exclude mootool core

this results in

//Contents: Core, More, Array, String, Function, Number, Hash, Class, Class.Refactor blar blar

ie I exclude mootools core I build but I get mootools core;

UTF-8 Encoding with Mootools More Form Validation (PHP)

When outputting a JS file that contains the mootools more form validation components, the UTF-8 encoding doesn't come down as UTF (and thus is garbled) since the charset isn't defined for the Content-Type header.

Possible fix (works for me), change this line:

header("Content-Type: application/x-javascript");

to:

header("Content-Type: application/x-javascript; charset=UTF-8");

core.py

I could be missing something, but why are there so many lines that try to deal with None for package_ymls and script_jsons? Why not just set the package_ymls and scipt_jsons in the init_ declaration to [] as default values?

def init(self, package_ymls=[], script_jsons=[]):
"""
package_ymls is list of filenames; script_jsons
is list of (library_name, filename) pairs.
"""
if package_ymls is None: # remove
package_ymls = [] # remove
if script_jsons is None: # remove
scripts_json = [] #remove

self.packages = {}
self.unqualified_components = {}

self.script_json_packages = []

package_ymls = package_ymls or [] # remove

URL link on http://mootools.net/blog/2009/11/09/mootools-depender-a-build-tool-for-mootools-javascript-libraries/ links to the wrong Github URL

Your blog posting on http://mootools.net/blog/2009/11/09/mootools-depender-a-build-tool-for-mootools-javascript-libraries/ links the GitHub repository of http://github.com/mootools/mootools-depender/.

It looks like the correct URL is http://github.com/anutron/mootools-depender

rq: Export Script.json from UI

The depender UI would be awesome for creating your script.json files in the first place!
oh yeah!

Split mootools-depender into several packages

At the moment mootools-depender implements several separate functions:

1. Combine js files into one according to dependency information;
2. infrastructure to serve the result file;
3. public interface to generate mootools builds

I think these functions are totally separate and it will be better if mootools-depender will be split into several packages.

Combining files into 1 don't have to be related to django at all. It also don't have to be tied to exact serving infrastructure and the public interface. I think it is a wrong direction to move all the functionality into django management commands and it will be better to create a separate python package from 'core.py' and upload it to pypi. This way it can be used with other python frameworks, for automated build systems, programmatically with other js-combining packages like django-compress and django-compressor, etc.

As for file serving: serving files via django itself it is a very niche implementation. Nginx is much better at serving files and there are mature django apps that makes generation of combined js files easy and robust (django-compress, django-compressor). They handle cache issues (auto-versioning), compression (via YUI or any other compression tool), debugging (e.g. don't combine or compress files when DEBUG=True), but they don't handle dependency resolution. I think most django developers would prefer one of these implementation because they already use them and they are proven, stable and used in high-load. Using something that requires django to serve files feels scary and suitable only for development or for small sites.

So my proposal it to split core.py into separate python package (e.g. js-depender) that can be installed via pip install and then used stand-alone, with any python framework and with any compressing/combining utility (like mootools-depender, django-compress or django-compressor).

What do you think?