etianen / django-require Goto Github PK
View Code? Open in Web Editor NEWA Django staticfiles post-processor for optimizing with RequireJS.
License: BSD 3-Clause "New" or "Revised" License
A Django staticfiles post-processor for optimizing with RequireJS.
License: BSD 3-Clause "New" or "Revised" License
The default REQUIRE_ENVIRONMENT is now "auto", and can also be set to a path to an Environment class.
I've successfully run collectstatic
with STATICFILES_STORAGE
set to require.storage.OptimizedCachedStaticFilesStorage
, but it's not clear to me how I would tell requirejs to use the fingerprinted files.
Would I need to put each file (with its fingerprint) explicitly in my config's paths
? And if so, would it make sense to extend OptimizedCachedStaticFilesStorage
to do this?
Or is there another solution for this I don't know about?
I'd like to build a module without almond.js. Currently, django-require forces almond.js for every standalone module.
How about something like this?
diff --git a/require/storage.py b/require/storage.py
index 7de04db..fcf4ced 100644
--- a/require/storage.py
+++ b/require/storage.py
@@ -119,8 +119,8 @@ class OptimizedFilesMixin(object):
module_build_js_path = env.resource_path("module.build.js")
env.run_optimizer(
module_build_js_path,
- name = "almond",
- include = standalone_module,
+ name = standalone_config.get("name", "almond"),
+ include = standalone_config.get("include", standalone_module),
out = env.build_dir_path(standalone_config["out"]),
baseUrl = os.path.join(env.compile_dir, require_settings.REQUIRE_BASE_URL),
)
My JS/CSS assets (which are served off of CloudFront are not being Gzipped). I am pushing the assets to S3 using:
class OptimizedCachedS3BotoStorage(FolderNameHackMixin, OptimizedFilesMixin, CachedFilesMixin, S3BotoStorage):
I've been reading up on it a bit, and it seems that if you need to gzip it before your push to cloudfront, remove the .gz part and set the content type to gzip on the asset in S3. This should obviously be handled automatically, but it seems crazy that the collectstatic pushing it to S3 doesn't handle it automatically. Any thoughts about how to integrate it or what setting is needed for this?
In the repo https://github.com/requirejs/example-multipage-shim the recommendation for using RequireJS with the shim config is to write script tags like:
// https://github.com/requirejs/example-multipage-shim/blob/master/www/page1.html
<script src="js/lib/require.js"></script>
<script>
//Load common code that includes config, then load the app
//logic for this page. Do the require calls here instead of
//a separate file so after a build there are only 2 HTTP
//requests instead of three.
require(['./js/common'], function (common) {
//js/common sets the baseUrl to be js/ so
//can just ask for 'app/main1' here instead
//of 'js/app/main1'
require(['app/main1']);
});
</script>
django-require comes with a tag you can use like:
{% require_module 'main' %}
which outputs:
<script src="/static/js/require.js" data-main="/static/js/main.js"></script>
That's great, but I use the shim config, so my Django template looks like:
<script>
require(['{% static 'js/requirejs_config.js' %}'], function () {
require(['home/js/home']);
});
</script>
This works fine except that I haven't gotten the r.js optimizer to work on it. I cannot easily rewrite my script tag to use the require_module
tag. Thus I cannot use the r.js support provided by django-require. Is there a simple way to convert my template to usable by django-require?
For what it's worth my RequireJS config looks like:
requirejs.config({
waitSeconds: 60,
baseUrl: '/static',
paths: {
'jquery': 'js/lib/jquery',
'ember': 'js/lib/ember-1.0.0-rc.6.1',
'handlebars': 'js/lib/handlebars-1.0.0-rc.4',
'underscore': 'js/lib/underscore'
},
shim: {
'underscore': {
exports: '_'
},
'ember': {
deps: ['jquery', 'handlebars'],
exports: 'Ember'
},
'handlebars': {
exports: 'Handlebars'
}
}
});
This is currently unimplementable, since collectstatic does not pass the verbosity option to post_process, despite being documented as doing so...
Hi, is it possible to somehow make django-require include a CSS file with a filename of the format filename.abc123.css
when using the OptimizedCachedStaticFilesStorage
, like it does with javascript files? I see that minified CSS is being served, but using the normal filename of the format filename.css
, so there is potential for caching issues.
I'm tyring to hash file names and append to requirejs's config.
since build.txt has the .js
files it processed, I could use it. but TemporaryCompileEnvironment deletes the directory it seems.
I don't see a way of overriding the environment class or storage to save the build.txt.
Thanks!
When I run collectstatic
, the process seems to get most of the way through my files but often fails due to the following exception. Otherwise the output looks fine!
[...]
Post-processed 'js/lib/tiny_mce/plugins/contextmenu/editor_plugin_src.js' as 'js/lib/tiny_mce/plugins/contextmenu/editor_plugin_src.js
Post-processed 'js/lib/tiny_mce/plugins/directionality/editor_plugin.js' as 'js/lib/tiny_mce/plugins/directionality/editor_plugin.js
Traceback (most recent call last):
File "/Users/eric/.virtualenvs/philly_living_lots/bin/django-admin.py", line 5, in <module>
management.execute_from_command_line()
File "/Users/eric/.virtualenvs/philly_living_lots/lib/python2.7/site-packages/django/core/management/__init__.py", line 453, in execute_from_command_line
utility.execute()
File "/Users/eric/.virtualenvs/philly_living_lots/lib/python2.7/site-packages/django/core/management/__init__.py", line 392, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/Users/eric/.virtualenvs/philly_living_lots/lib/python2.7/site-packages/django/core/management/base.py", line 229, in run_from_argv
stderr.write('%s: %s' % (e.__class__.__name__, e))
File "/Users/eric/.virtualenvs/philly_living_lots/lib/python2.7/site-packages/django/core/management/base.py", line 68, in write
self._out.write(force_str(style_func(msg)))
IOError: [Errno 35] Resource temporarily unavailable
I'm using django-pipeline to compile/compress/cache bust my SASS files.
Since it uses
STATICFILES_STORAGE = 'pipeline.storage.PipelineCachedStorage'
in the settings, and you want it to be changed to one of your storage module. This way it either can't do its job, or your module can't do theirs with compiling the required javascript.
Any idea on how to fix this? I've been fighting with this for 4 days, It seems I'll implement my own module at the end, as I'm starting to lose it.
I'm trying to build a multi-page app with django-require.
My goal would be to have it build an almond standalone module on collectstatic with all my 3rdparty libraries and my page modules included, but only parse/execute the parts that the specific page module refers to, using something like this in the page:
{% require_module 'common' %}
<script>
require(['common'], function(common) {
require(['app/page1']);
});
</script>
Now I'm getting an error in the browser, because my libraries do not depend on the page modules, so the page modules were never included in the standalone module:
Uncaught Error: undefined missing app/page1
So I tried to include the page-specific modules in the almond build. However, if I specify modules
in my app.build.js
, I get an r.js error:
Error: If the "modules" option is used, then there should be a "dir" option set and "out" should not be used since "out" is only for single file optimization output.
My next attempt was to only build the common libs and modules into the standalone module, and keep the page-specific modules separate, so using multiple {% require_module %}
tags like this:
{% require_module 'common' %} {# this is declared in REQUIRE_STANDALONE_MODULES #}
{% require_module 'app/page1' %} {# this is not! #}
However, then I suddenly get both almond and require.js into my page, and I'll probably have to maintain two almost-identical app.build.js
es. It also doesn't execute the code in the page module, but I haven't further investigated why.
Has anyone succeeded in using django-require for multi-page apps and are there any best practices how to set this best up, so it works both in production (after collectstatic) and in DEBUG mode?
Can we update the template tag to output with an async attribute? Or if not backwards compatible perhaps a keyword on the template tag 'async' to do so.
http://requirejs.org/docs/start.html
For browsers that support it, you could also add an async attribute to the script tag.
e.g.
<script src="static/js/require.a3f22c5ad833.js" data-main="static/js/search.bf074024de9e.js">
becomes
<script src="static/js/require.a3f22c5ad833.js" data-main="static/js/search.bf074024de9e.js" async defer>
https://www.igvita.com/2014/05/20/script-injected-async-scripts-considered-harmful/
Hi there,
Forgive me if this is not the best place for this, but this is more of a bunch of questions that aren't easily asked over Twitter rather than an issue report.
I'm trying to figure out exactly what the staticfiles storage is doing. Here's the behaviour I'm seeing:
collectstatic --link --noinput
, all css/js are linked, over and over again. With the default STATICFILES_STORAGE
, I don't get this issue. I assume this is not expected?optimizeCss: "none"
, I am still seeing post-processing being done on CSS files. Am I thinking of two different things here? It feels like I am.Due to some deprecated CSS files, I am not able to complete a collectstatic
command without errors. Generally a bad image path in a css file, e.g.:
ValueError: The file 'images/small-arrow-left.png' could not be found with <require.storage.OptimizedCachedStaticFilesStorage object at 0x107dc4cd0>.
At this point, I simply want django-require to ignore CSS entirely. When I run r.js optimizer by its lonesome, I don't have this issue, so not sure exactly what I'm missing.
My settings that I've defined are as follows:
REQUIRE_BASE_URL = "."
REQUIRE_BUILD_PROFILE = 'app.build.js'
REQUIRE_JS = "js/require.js"
REQUIRE_DEBUG = DEBUG
REQUIRE_ENVIRONMENT = 'node'
REQUIRE_STANDALONE_MODULES = {}
Thanks for the excellent app! I believe it'll be very beneficial for us in using require-js modules around the site, just trying to better understand it :)
I have a js file that I'd like to exclude from optimization: /js/vendor/openlayers/OpenLayer.js. How do i specify this using the REQUIRE_EXCLUDE setting? I tried just using the filename itself, the full path, and many variations of the relative path but nothing seems to work. Here's my require config:
########## REQUIRE CONFIGURATION
# The baseUrl to pass to the r.js optimizer.
REQUIRE_BASE_URL = "js"
# The name of a build profile to use for your project, relative to REQUIRE_BASE_URL.
# A sensible value would be 'app.build.js'. Leave blank to use the built-in default build profile.
REQUIRE_BUILD_PROFILE = "app.build.js"
# The name of the require.js script used by your project, relative to REQUIRE_BASE_URL.
REQUIRE_JS = "vendor/require.js"
# A dictionary of standalone modules to build with almond.js.
# See the section on Standalone Modules, below.
REQUIRE_STANDALONE_MODULES = {}
# Whether to run django-require in debug mode.
REQUIRE_DEBUG = True
# A tuple of files to exclude from the compilation result of r.js.
REQUIRE_EXCLUDE = ("build.txt", "OpenLayers.js",)
# The execution environment in which to run r.js: node or rhino.
REQUIRE_ENVIRONMENT = "node"
########## END REQUIRE CONFIGURATIO
In general, the "node" environment is simply faster and better, whereas the "rhino" environment is an order of magnitude slower. However, whilst java is available on almost all machines, node isn't.
A REQUIRE_ENVIRONMENT setting of "auto" would check if node exists, and if so, use that. If not, it will use rhino.
The default REQUIRE_ENVIRONMENT would become "auto".
When ./manage.py require_init
is called without specifying a --dir
option if the STATICFILES_DIR
setting is left blank an unhelpful error messages is displayed:
AttributeError: 'NoneType' object has no attribute 'endswith'
This exception should probably be suppressed and an error message should state that a target directory must be specified.
When I'm trying to build on collectstatic almond try to find jquery as file and not as path on app define
this is my app build js
{
baseUrl: './javascripts/',
name: 'main',
include: ['main'],
mainConfigFile: 'main.js',
paths: {
'jquery': 'libs/jquery/jquery-1.9.1',
'underscore': 'libs/underscore/underscore',
'backbone': 'libs/backbone/backbone',
'loader': 'libs/backbone/loader',
'backbone-queryparams': 'libs/backbone/plugins/backbone.queryparams',
'bootstrap': 'libs/bootstrap/bootstrap',
'text': 'libs/requirejs/text',
'flowplayer': 'libs/flowplayer/flowplayer',
'jquery-ui': 'libs/jquery/plugins/jquery-ui-1.10.1.custom',
'jquery.ui.widget': 'libs/jquery/plugins/jquery.ui.widget',
'iframe-transport': 'libs/jquery/plugins/jquery.iframe-transport',
'fileupload': 'libs/jquery/plugins/jquery.fileupload',
'meio-mask': 'libs/jquery/plugins/jquery.meio.mask',
'underscore-ead': 'libs/underscore/underscore-ead',
'templates': '..'
},
shim: {
'backbone': {
deps: ['jquery', 'underscore'],
exports: 'Backbone'
},
'backbone-queryparams': ['backbone'],
'jquery-ui': ['jquery'],
'iframe-transport': ['jquery'],
'fileupload': ['jquery', 'jquery.ui.widget', 'iframe-transport']
},
waitSeconds: 30
}
I'm using django-require with a single REQUIRE_STANDALONE_MODULES
and a build profile. In the build profile, I set the paths
configuration like this:
paths: {
"jquery": "libs/jquery-1.12.1"
},
Now when I run the app in DEBUG mode (i.e. without having built the JS file and with require.js instead of almond.js), it doesn't use the build profile and can't find the jquery
dependency.
Is there an elegant way to reuse the paths between the two build configurations?
It does work when using something like this, but I'd prefer to avoid the duplication:
{% require_module 'entrypoint' %}
<script>
requirejs.config({
paths: {
"jquery": "libs/jquery-1.12.1"
}
});
</script>
This is because the default build profile specifies the closure compiler, which isn't available for node.
Solutions:
But which is best?
My build config file contains "preserveLicenseComments: false" which strips out comments and license headers. This leaves me with a big blob of code, which is great but it also needs a license header. I assumed wrap could be used like this:
wrap: {
startFile: "header.txt",
endFile: "footer.txt",
},
But due to requirejs/r.js#132 this won't work either and jrburke suggested to "use a post-r.js build step to add the header back after final optimizations" before closing that ticket. Is this something we can add to django-require? I'll see if i can come up with a patch.
1.0.3 says requirejs was updated to 2.1.4 but this is also the case for 1.0.2 if you look at the changelog. I think 1.0.3's changelog entry should reference to 2.1.5, the latest requirejs.
When I run collectstatic command, I get the following error
Post-processed 'js\main.js' as 'js/main.62646e75ef2d.js'
Post-processed 'js\test.js' as 'js/test.5ee2469fc455.js'
Post-processed 'js\test2.js' as 'js/test2.e2c752f413a8.js'
Post-processed 'cms\PIE.htc' as 'cms/PIE.a219e20e2678.htc'
Post-processed 'cms\config.rb' as 'cms/config.a49346017e25.rb'
Post-processing 'admin/css/base.css' failed!
Traceback (most recent call last):
File "./manage.py", line 10, in <module>
execute_from_command_line(sys.argv)
File "c:\dev\test_project\lib\site-packages\django\core\management\__init__.py", line 399, in execute_from_command_line
utility.execute()
File "c:\dev\test_project\lib\site-packages\django\core\management\__init__.py", line 392, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "c:\dev\test_project\lib\site-packages\django\core\management\base.py", line 242, in run_from_argv
self.execute(*args, **options.__dict__)
File "c:\dev\test_project\lib\site-packages\django\core\management\base.py", line 285, in execute
output = self.handle(*args, **options)
File "c:\dev\test_project\lib\site-packages\django\core\management\base.py", line 415, in handle
return self.handle_noargs(**options)
File "c:\dev\test_project\lib\site-packages\django\contrib\staticfiles\management\commands\collectstatic.py", line 167, in handle_noargs
collected = self.collect()
File "c:\dev\test_project\lib\site-packages\django\contrib\staticfiles\management\commands\collectstatic.py", line 124, in collect
raise processed
ValueError: The file 'img/nav-bg.gif' could not be found with <require.storage.OptimizedCachedStaticFilesStorage object at 0x0000000003727C88>.
However, when I turn off the OptimizedCachedStaticFilesStorage
and run the following commands manually, which from what I can tell is effectively the same thing, it works just fine.
manage.py collectstatic -c
r.js -o customerportal/static/js/libs/require/app.build.js appDir=../../static/ baseUrl=js dir=../../static2/
Hi, I need to speed up my deployments. Therefore I want to exclude some scripts from optimization, for example, 'ckeditor/ckeditor/ckeditor.js' and all its dependencies and plugins. Currently it takes minutes to run collectstatic
, so is there any best practice to add whole paths to the REQUIRE_EXCLUDE
setting, like 'ckeditor'
.
How should I optimize my less files?
Is it possible to optimize js files using django-require and
optimize less/css files using django-compress/django-pipeline?
I have no idea how to optimize requirejs modules using django-compressor/django-pipeline, nor, less files with r.js
On PyPi the download URL is:
https://github.com/downloads/etianen/django-require/django-require-1.0.2.tar.gz
Which makes setup tools sad. Updating it to:
https://github.com/etianen/django-require/archive/release-1.0.2.tar.gz
would fix it.
Thanks!
Using the "removeCombined" setting in a build profile has no effect. I suppose this happens because it copies the built assets into the original directory, without emptying it first.
Hi,
I am trying to compile standalone modules residing in different subdirectories, each being a different subsystem, independent of each other.
Since each of the standalone modules needs its own configuration, the REQUIRE_BASE_URL
should be different for them, or else the compilation part will fail, telling that it can't find files that are actually in subdirectories. But since I can't change the baseUrl
to their respective subdirectories, the build will fail.
This could be fixed with introducing a separate BASE_URL
setting to each REQUIRE_STANDALONE_MODULES
section, which can be relative to the REQUIRE_BASE_URL
setting.
Could you please implement this?
Rhino with r.js often runs out of stack: mozilla/rhino#67. This can be fixed by passing the -Xss option to java. -Xss1M fixed the issue for me.
I'm not sure this is something django-require should fix, but it'd be easy to add the option to RhinoEnvironment.
Trying to build the standalone modules requires one to run collectstatic, which can be a length process since it uglifyies2 all the files again (even if not changed). That may be required behaviour since you might change an uglify2 compressor option.
But trying to build and test a standalone module makes it hard since you got to wait and run collectstatic.
Would be nice to be able to just run the standalone module builder on it's own.
Just found this project, it looks pretty useful! What do you think about factoring out the Django dependency in a couple places so that one could run_optimizer()
outside of a Django project? While I use Django pretty extensively, I have gotten in the habit of maintaining a strict modular separation between my client and server libraries, in part because there are a few projects where I can only use one or the other.
My build process already includes a call to execute r.js via subprocess
but I'm interested in replacing it with yours (mostly so I can get some of your AutoEnvironment
magic). To accomplish this without a major refactoring, I'm thinking of something like the following API:
os.environ['REQUIRE_SETTINGS'] = "app.require_settings"
from require import optimize
optimize()
Then LazySettings
might get a bit lazier:
class LazySettings(object):
@cached_property
def config(self):
if 'DJANGO_SETTINGS_MODULE' in os.environ:
from django.conf import settings
return settings
else:
return load_module(os.environ['REQUIRE_SETTINGS'])
@property
def REQUIRE_BASE_URL(self):
return getattr(self.config, "REQUIRE_BASE_URL", "js")
# ...
TemporaryCompileEnvironment
would probably need to move out of storage
, and maybe have a non-temporary version that used whatever was in the build profile as directory paths. optimize()
would just be a thin convenience wrapper to initialize and run a compile environment.
What do you think? I'm happy to submit a PR for this change, but I thought I'd run it by you first to see a.) if this feature would be useful to others and b.) if the above would be the right way to do it.
Template:
{% load require %}
{% require_module 'main' %}
In the latest version of Django this produces:
<script src="/path/to/require.js" data-main="/path/to/main.js"></script>
It's been over a year since the resources
directory has been updated with newer versions of require.js, r.js etc. Can these be updated to the latest versions?
I see commits for python 3 compat but setup.py doesn't list it as supported. This is important for people on pypi searching for python 3 compatibility.
When I set REQUIRE_BASE_URL = ""
, it defaults to "js"
because an empty string is falsey. As a workaround I tried REQUIRE_BASE_URL = "../static"
and REQUIRE_BASE_URL = "../static/"
but they cause some bizarre errors.
Hopefully it should just be as simple as checking for an empty string before checking for a truthy/falsey value.
The tests.py module should ideally test both, but should skip the rhino/node tests when that environment is not available.
When using this storage ran into an issue where it hashed and gave a different name for jquery. Since this jquery was different to one in my path it loaded a second jquery. Fixed it by not using {% static jquery.js %}
but instead used STATIC_URL. Although how are you mean to use jquery, requirejs and the OptimizedCachedStaticFilesStorage together? Since require.js has it's paths and moduleid is part of the name how do we get them all tied together?
Using django-require v1.0.11.
Testing with the official storage:
STATICFILES_STORAGE = 'require.storage.OptimizedManifestStaticFilesStorage'
I have a single standalone module defined:
REQUIRE_STANDALONE_MODULES = {
'main': {
'out': 'main-built.js',
'build_profile': 'app.build.js',
}
}
I run collectstatic to run the r.js optimizer on my bundle and I get a js/main-built.19882680b8d2.js
in my STATIC_ROOT
directory. I can also see the corresponding entry in the staticfiles.json
:
"js\\main-built.js": "js/main-built.19882680b8d2.js"
So far, so good!
Now I reference this in my template:
{% require_module "main" %}
I set REQUIRE_DEBUG = False
and load my page in the browser.
I expected the following line to be generated:
<script src="http://localhost:8000/static/js/main-built.19882680b8d2.js"></script>
Instead, I got this:
<script src="http://localhost:8000/static/js/main-built.js"></script>
Did I miss something? What do I have to do for require_module
to return hashed URLs?
The current one doesn't show up right. For converting from markdown to rst:
pandoc -f markdown -t rst -o README.rst README.md
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.