File depending on a file which depends on a task will always be run

[jakejs]

File depending on a file which depends on a task will always be run

[mde]

This is interesting. I think I understand what you're trying to accomplish -- but first off, using a fake "force" no-op task to cause your download of bar-file to happen seems less than optimal. Also, if it's actually downloaded each time, won't it always overwrite what's on the filesystem? It's always going to appear to be a newer file. Or are you handling this in the task somewhere? Looks like what you want is some sort of etags-aware, remote file-task. Am I understanding this correctly?

[curvedmark]

Exactly! I didn't think it could be a feature built into jake.

You are right about bar-file being etag-aware. I wrote a custom node script that downloads bar-file. It checks etag (and fallbacks to last-modified if etag is not supported) for the file's freshness, and only downloads a new copy when necessary.

If jake could support remote file-task, that would be awesome!

I'd also like to elaborate on my previous sample Jakefile. Since it's a valid Jakefile, I believe it exhibits some defect jake currently has.

Here is my another take on the file:

var fs = require('fs');

task('default', ['foo-file'], function() {});

file('foo-file', ['bar-file'], function() {
    console.log('building foo-file');

    fs.writeFile('./foo-file', 'foo-file content');
});

file('bar-file', ['force'], function() {
    console.log('building bar-file');

    try {
        fs.statSync('./bar-file');
    } catch (e) {
        fs.writeFile('./bar-file', 'bar-file content');
    }
});

task('force', function() {});

If both foo-file and bar-file don't exist before this jakefile is executed, the output should be building bar-file building foo-file on its first execution, and just building bar-file on its second execution (since bar-file is older than foo-file, the foo-file task doesn't need to be run).

However, with jake 0.1.12, the output is building bar-file building foo-file for both executions.

[mde]

Okay, I've verified that this behavior is exactly what Rake does. It makes sense that if any file task has a non-file dependency, it will always run. If that task runs, any tasks that depends on it will run too. Also made the task-action callback optional. Now you can just define task('foo'); and it will be a no-op.

[curvedmark]

If that task runs, any tasks that depends on it will run too
I'm not sure it's a correct behavior.

It's a file task, which means the task is supposed to generate/modify a file. When other tasks depend on this task, they actually depend on the file the task generates/modifies, not the task itself, so whether the task will always run shouldn't affect how these task run. These task should only check the file's existence or mtime. For they depend on a file, not a task.

What do you think?

[mde]

Okay, I've taken another look at this -- looks like Rake is doing the right thing, but Jake is not. Our problem is that Jake parses the dependency-tree all up-front for file tasks, whereas it ought to do it incrementally as execution occurs .This will require a substantial change to how dependency-parsing and task-running works. I'll leave this issue open, but it will take a while for this to get fixed.

[curvedmark]

Thanks for deciding to fix it. :)

[mde]

Fixed in master 88690ff. Please verify, and I'll do a release.

[curvedmark]

Just tested against 7fa59fa with this Jakefile, and it throws Error: ENOENT, No such file or directory 'foo-file'. Looks like jake needs to use try & catch to check file's existence in jake.js line 266, not sure though.

var fs = require('fs');

task({default: ['foo-file']});

file({"foo-file": ['bar-file']}, function() {
    console.log('building foo-file');
});

file({"bar-file": ['force']}, function() {
    console.log('building bar-file');
});

task('force');

[mde]

Yes, I was assuming that the file-task should actually end up generating a file, and throw if it doesn't. But maybe you're right, a more reasonable behavior would be just to treat it like a normal task that has to run every time, and use the current time as the mod-time for the purposes of any file tasks higher up the tree. The fix is in 0278c80.

[curvedmark]

Tested against 0278c80 with this Jakefile. On first execution, foo-file task is executed, but not bar-file task. On second execution, none is executed.

var fs = require('fs');

task({default: ['foo-file']});

file({"foo-file": ['bar-file']}, function() {
    console.log('building foo-file');

    fs.writeFile('foo-file', 'foo-file content');
});

file({"bar-file": ['force']}, function() {
    console.log('building bar-file');

    try {
        fs.statSync('bar-file');
    } catch (e) {
        fs.writeFile('bar-file', 'bar-file content');
    }
});

task('force');

[curvedmark]

Do you think I should fire another issue for adding remote file task support?

[mde]

Okay, I've verified with HEAD (5c0a7f9) that it runs both bar and foo the first time, and only bar subsequent times:

mdes-mbp:jake mde$ node ./bin/cli.js -f ./tests/Jakefile4 -t -C /Users/mde/work/jake
building bar-file
building foo-file
mdes-mbp:jake mde$ node ./bin/cli.js -f ./tests/Jakefile4 -t -C /Users/mde/work/jake
building bar-file

As for remote files -- you can sure add another issue, but there's so much I still have to do with PackageTask that I'm not sure how long it'll be before I can get to it.

You could use this trick of modifying a local stub-file when the remote file has changed, or if you want, you could try doing an implementation of remoteFile yourself. :) I'm happy to give you some direction on where to look in the code if you'd like to take a stab at it.

[curvedmark]

I'm definitely interested in implementing it. Where should I start to look?

(Should we head to some irc channel?)

[mde]

The globalized API calls (task, file, etc.) are defined in lib/api.js. Maybe take a look at how file defers to createTask, and think about how to implement a new task-type. In the longer term, I'm planning on changing from generic Task instances with a type property, to Task with sub-classing, e.g., FileTask, DirectoryTask, etc. I've starte a #jake IRC channel on Freenode, if you want to talk more.